2024-04-26T03:30:32.593 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-26T03:30:32.593 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi/7674443 --name yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi --description rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{default} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} --verbose --owner scheduled_yuriw@teuthology -- /home/teuthworker/archive/yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi/7674443/orig.config.yaml 2024-04-26T03:30:32.634 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T03:30:32.723 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi/7674443 branch: wip-yuri2-testing-2024-04-24-0914-squid description: rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{default} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} email: yweinste@redhat.com first_in_suite: false job_id: '7674443' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-yuri2-testing-2024-04-24-0914-squid ceph: conf: global: mon client directed command retry: 5 mon election default strategy: 1 ms inject socket failures: 2500 ms tcp read timeout: 5 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon scrub interval: 300 osd: bluestore block size: 96636764160 bluestore compression algorithm: zlib bluestore compression mode: aggressive bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd debug reject backfill probability: 0.3 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd max backfills: 3 osd max markdown count: 1000 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - slow request - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED sha1: ddeaa91797339a19172c3036ff48cca58c12f448 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: ddeaa91797339a19172c3036ff48cca58c12f448 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri2-testing-2024-04-24-0914-squid sha1: ddeaa91797339a19172c3036ff48cca58c12f448 owner: scheduled_yuriw@teuthology priority: 99 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - osd.0 - osd.4 - osd.8 - osd.12 - - mon.b - osd.1 - osd.5 - osd.9 - osd.13 - - mon.c - osd.2 - osd.6 - osd.10 - osd.14 - - mgr.x - osd.3 - osd.7 - osd.11 - osd.15 - client.0 seed: 1848 sha1: ddeaa91797339a19172c3036ff48cca58c12f448 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-yuri2-testing-2024-04-24-0914-squid suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_ddeaa91797339a19172c3036ff48cca58c12f448/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: ddeaa91797339a19172c3036ff48cca58c12f448 targets: smithi028.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMGSA8wL+tzr81Ty8AFOYZ/VZPP0hQA/31h6+l9a3of3/3q7foqrrN1IEF7uwrbRWhaHalBEgBrqInqu0oYTjDg= smithi042.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLmTDwfubsCA2qsSwg0HoXpo1FxmcM1ACY9Kyrv1mcT2zdktDFxFCg1Z5/UtTfRJTkTC+gK6Arv1lFcFejRR67o= smithi066.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJrHq4izvgILDjyK2UbhRKaS+aQoC3LdnaJp7pcHbiE2W+Md+Hn0aQJtzgBciGIqEvVj5cMwdWhIjMrle/rN0HM= smithi092.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCBimfGZaOgOmcVYzVt14hT9zje/dFFyZ+b2S7egISjZzQp49FrINmAsYOOeOaFUs4m7HgNIMUR6moCu3clamJw= tasks: - install: null - ceph: conf: osd: debug monc: 20 - thrashosds: aggressive_pg_num_changes: false chance_pgnum_grow: 1 chance_pgnum_shrink: 1 chance_pgpnum_fix: 1 min_in: 8 timeout: 1200 - rados: clients: - client.0 ec_pool: true erasure_code_profile: c: 2 crush-failure-domain: osd k: 4 m: 3 name: shecprofile plugin: shec objects: 50 op_weights: append: 100 copy_from: 50 delete: 50 read: 100 rmattr: 25 rollback: 50 setattr: 25 snap_create: 50 snap_remove: 50 write: 0 ops: 400 write_append_excl: false teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-26_01:23:56 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-26T03:30:32.724 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_ddeaa91797339a19172c3036ff48cca58c12f448/qa; will attempt to use it 2024-04-26T03:30:32.725 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_ddeaa91797339a19172c3036ff48cca58c12f448/qa/tasks 2024-04-26T03:30:32.725 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-26T03:30:32.726 INFO:teuthology.task.internal:Checking packages... 2024-04-26T03:30:32.751 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'ddeaa91797339a19172c3036ff48cca58c12f448' 2024-04-26T03:30:32.751 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-26T03:30:32.751 INFO:teuthology.packaging:ref: None 2024-04-26T03:30:32.752 INFO:teuthology.packaging:tag: None 2024-04-26T03:30:32.752 INFO:teuthology.packaging:branch: wip-yuri2-testing-2024-04-24-0914-squid 2024-04-26T03:30:32.752 INFO:teuthology.packaging:sha1: ddeaa91797339a19172c3036ff48cca58c12f448 2024-04-26T03:30:32.752 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-yuri2-testing-2024-04-24-0914-squid 2024-04-26T03:30:32.891 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2457.gddeaa917 2024-04-26T03:30:32.892 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-26T03:30:32.897 INFO:teuthology.task.internal:no buildpackages task found 2024-04-26T03:30:32.897 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-26T03:30:32.902 INFO:teuthology.task.internal:Saving configuration 2024-04-26T03:30:32.917 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-26T03:30:32.922 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T03:30:32.947 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi028.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi/7674443', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 03:24:37.040197', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMGSA8wL+tzr81Ty8AFOYZ/VZPP0hQA/31h6+l9a3of3/3q7foqrrN1IEF7uwrbRWhaHalBEgBrqInqu0oYTjDg='} 2024-04-26T03:30:32.971 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi042.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi/7674443', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 03:24:37.044130', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLmTDwfubsCA2qsSwg0HoXpo1FxmcM1ACY9Kyrv1mcT2zdktDFxFCg1Z5/UtTfRJTkTC+gK6Arv1lFcFejRR67o='} 2024-04-26T03:30:32.991 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi066.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi/7674443', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 03:24:37.042219', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJrHq4izvgILDjyK2UbhRKaS+aQoC3LdnaJp7pcHbiE2W+Md+Hn0aQJtzgBciGIqEvVj5cMwdWhIjMrle/rN0HM='} 2024-04-26T03:30:33.010 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi092.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi/7674443', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 03:24:37.037913', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCBimfGZaOgOmcVYzVt14hT9zje/dFFyZ+b2S7egISjZzQp49FrINmAsYOOeOaFUs4m7HgNIMUR6moCu3clamJw='} 2024-04-26T03:30:33.011 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-26T03:30:33.017 INFO:teuthology.task.internal:roles: ubuntu@smithi028.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'osd.0', 'osd.4', 'osd.8', 'osd.12'] 2024-04-26T03:30:33.017 INFO:teuthology.task.internal:roles: ubuntu@smithi042.front.sepia.ceph.com - ['mon.b', 'osd.1', 'osd.5', 'osd.9', 'osd.13'] 2024-04-26T03:30:33.017 INFO:teuthology.task.internal:roles: ubuntu@smithi066.front.sepia.ceph.com - ['mon.c', 'osd.2', 'osd.6', 'osd.10', 'osd.14'] 2024-04-26T03:30:33.017 INFO:teuthology.task.internal:roles: ubuntu@smithi092.front.sepia.ceph.com - ['mgr.x', 'osd.3', 'osd.7', 'osd.11', 'osd.15', 'client.0'] 2024-04-26T03:30:33.017 INFO:teuthology.run_tasks:Running task console_log... 2024-04-26T03:30:33.163 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f28b4c1ef70>, signals=[15]) 2024-04-26T03:30:33.163 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-26T03:30:33.240 INFO:teuthology.task.internal:Opening connections... 2024-04-26T03:30:33.240 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi028.front.sepia.ceph.com 2024-04-26T03:30:33.243 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T03:30:33.321 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi042.front.sepia.ceph.com 2024-04-26T03:30:33.322 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T03:30:33.406 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi066.front.sepia.ceph.com 2024-04-26T03:30:33.407 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T03:30:33.487 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi092.front.sepia.ceph.com 2024-04-26T03:30:33.488 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T03:30:33.564 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-26T03:30:33.572 DEBUG:teuthology.orchestra.run.smithi028:> uname -m 2024-04-26T03:30:33.591 INFO:teuthology.orchestra.run.smithi028.stdout:x86_64 2024-04-26T03:30:33.591 DEBUG:teuthology.orchestra.run.smithi028:> cat /etc/os-release 2024-04-26T03:30:33.650 INFO:teuthology.orchestra.run.smithi028.stdout:NAME="CentOS Stream" 2024-04-26T03:30:33.650 INFO:teuthology.orchestra.run.smithi028.stdout:VERSION="9" 2024-04-26T03:30:33.650 INFO:teuthology.orchestra.run.smithi028.stdout:ID="centos" 2024-04-26T03:30:33.650 INFO:teuthology.orchestra.run.smithi028.stdout:ID_LIKE="rhel fedora" 2024-04-26T03:30:33.650 INFO:teuthology.orchestra.run.smithi028.stdout:VERSION_ID="9" 2024-04-26T03:30:33.651 INFO:teuthology.orchestra.run.smithi028.stdout:PLATFORM_ID="platform:el9" 2024-04-26T03:30:33.651 INFO:teuthology.orchestra.run.smithi028.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T03:30:33.651 INFO:teuthology.orchestra.run.smithi028.stdout:ANSI_COLOR="0;31" 2024-04-26T03:30:33.651 INFO:teuthology.orchestra.run.smithi028.stdout:LOGO="fedora-logo-icon" 2024-04-26T03:30:33.651 INFO:teuthology.orchestra.run.smithi028.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T03:30:33.651 INFO:teuthology.orchestra.run.smithi028.stdout:HOME_URL="https://centos.org/" 2024-04-26T03:30:33.651 INFO:teuthology.orchestra.run.smithi028.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T03:30:33.651 INFO:teuthology.orchestra.run.smithi028.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T03:30:33.651 INFO:teuthology.orchestra.run.smithi028.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T03:30:33.652 INFO:teuthology.lock.ops:Updating smithi028.front.sepia.ceph.com on lock server 2024-04-26T03:30:33.669 DEBUG:teuthology.orchestra.run.smithi042:> uname -m 2024-04-26T03:30:33.687 INFO:teuthology.orchestra.run.smithi042.stdout:x86_64 2024-04-26T03:30:33.688 DEBUG:teuthology.orchestra.run.smithi042:> cat /etc/os-release 2024-04-26T03:30:33.746 INFO:teuthology.orchestra.run.smithi042.stdout:NAME="CentOS Stream" 2024-04-26T03:30:33.746 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION="9" 2024-04-26T03:30:33.746 INFO:teuthology.orchestra.run.smithi042.stdout:ID="centos" 2024-04-26T03:30:33.746 INFO:teuthology.orchestra.run.smithi042.stdout:ID_LIKE="rhel fedora" 2024-04-26T03:30:33.746 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION_ID="9" 2024-04-26T03:30:33.746 INFO:teuthology.orchestra.run.smithi042.stdout:PLATFORM_ID="platform:el9" 2024-04-26T03:30:33.746 INFO:teuthology.orchestra.run.smithi042.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T03:30:33.746 INFO:teuthology.orchestra.run.smithi042.stdout:ANSI_COLOR="0;31" 2024-04-26T03:30:33.746 INFO:teuthology.orchestra.run.smithi042.stdout:LOGO="fedora-logo-icon" 2024-04-26T03:30:33.747 INFO:teuthology.orchestra.run.smithi042.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T03:30:33.747 INFO:teuthology.orchestra.run.smithi042.stdout:HOME_URL="https://centos.org/" 2024-04-26T03:30:33.747 INFO:teuthology.orchestra.run.smithi042.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T03:30:33.747 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T03:30:33.747 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T03:30:33.747 INFO:teuthology.lock.ops:Updating smithi042.front.sepia.ceph.com on lock server 2024-04-26T03:30:33.769 DEBUG:teuthology.orchestra.run.smithi066:> uname -m 2024-04-26T03:30:33.787 INFO:teuthology.orchestra.run.smithi066.stdout:x86_64 2024-04-26T03:30:33.787 DEBUG:teuthology.orchestra.run.smithi066:> cat /etc/os-release 2024-04-26T03:30:33.845 INFO:teuthology.orchestra.run.smithi066.stdout:NAME="CentOS Stream" 2024-04-26T03:30:33.845 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION="9" 2024-04-26T03:30:33.845 INFO:teuthology.orchestra.run.smithi066.stdout:ID="centos" 2024-04-26T03:30:33.845 INFO:teuthology.orchestra.run.smithi066.stdout:ID_LIKE="rhel fedora" 2024-04-26T03:30:33.845 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION_ID="9" 2024-04-26T03:30:33.845 INFO:teuthology.orchestra.run.smithi066.stdout:PLATFORM_ID="platform:el9" 2024-04-26T03:30:33.845 INFO:teuthology.orchestra.run.smithi066.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T03:30:33.845 INFO:teuthology.orchestra.run.smithi066.stdout:ANSI_COLOR="0;31" 2024-04-26T03:30:33.846 INFO:teuthology.orchestra.run.smithi066.stdout:LOGO="fedora-logo-icon" 2024-04-26T03:30:33.846 INFO:teuthology.orchestra.run.smithi066.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T03:30:33.846 INFO:teuthology.orchestra.run.smithi066.stdout:HOME_URL="https://centos.org/" 2024-04-26T03:30:33.846 INFO:teuthology.orchestra.run.smithi066.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T03:30:33.846 INFO:teuthology.orchestra.run.smithi066.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T03:30:33.846 INFO:teuthology.orchestra.run.smithi066.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T03:30:33.846 INFO:teuthology.lock.ops:Updating smithi066.front.sepia.ceph.com on lock server 2024-04-26T03:30:33.869 DEBUG:teuthology.orchestra.run.smithi092:> uname -m 2024-04-26T03:30:33.887 INFO:teuthology.orchestra.run.smithi092.stdout:x86_64 2024-04-26T03:30:33.887 DEBUG:teuthology.orchestra.run.smithi092:> cat /etc/os-release 2024-04-26T03:30:33.944 INFO:teuthology.orchestra.run.smithi092.stdout:NAME="CentOS Stream" 2024-04-26T03:30:33.945 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION="9" 2024-04-26T03:30:33.945 INFO:teuthology.orchestra.run.smithi092.stdout:ID="centos" 2024-04-26T03:30:33.945 INFO:teuthology.orchestra.run.smithi092.stdout:ID_LIKE="rhel fedora" 2024-04-26T03:30:33.945 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_ID="9" 2024-04-26T03:30:33.945 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM_ID="platform:el9" 2024-04-26T03:30:33.945 INFO:teuthology.orchestra.run.smithi092.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T03:30:33.945 INFO:teuthology.orchestra.run.smithi092.stdout:ANSI_COLOR="0;31" 2024-04-26T03:30:33.945 INFO:teuthology.orchestra.run.smithi092.stdout:LOGO="fedora-logo-icon" 2024-04-26T03:30:33.945 INFO:teuthology.orchestra.run.smithi092.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T03:30:33.945 INFO:teuthology.orchestra.run.smithi092.stdout:HOME_URL="https://centos.org/" 2024-04-26T03:30:33.945 INFO:teuthology.orchestra.run.smithi092.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T03:30:33.945 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T03:30:33.946 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T03:30:33.946 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server 2024-04-26T03:30:34.069 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-26T03:30:34.080 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-26T03:30:34.086 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-26T03:30:34.087 DEBUG:teuthology.orchestra.run.smithi028:> test '!' -e /home/ubuntu/cephtest 2024-04-26T03:30:34.089 DEBUG:teuthology.orchestra.run.smithi042:> test '!' -e /home/ubuntu/cephtest 2024-04-26T03:30:34.092 DEBUG:teuthology.orchestra.run.smithi066:> test '!' -e /home/ubuntu/cephtest 2024-04-26T03:30:34.095 DEBUG:teuthology.orchestra.run.smithi092:> test '!' -e /home/ubuntu/cephtest 2024-04-26T03:30:34.111 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-26T03:30:34.117 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-26T03:30:34.118 DEBUG:teuthology.orchestra.run.smithi028:> test -z $(ls -A /var/lib/ceph) 2024-04-26T03:30:34.147 DEBUG:teuthology.orchestra.run.smithi042:> test -z $(ls -A /var/lib/ceph) 2024-04-26T03:30:34.149 DEBUG:teuthology.orchestra.run.smithi066:> test -z $(ls -A /var/lib/ceph) 2024-04-26T03:30:34.152 DEBUG:teuthology.orchestra.run.smithi092:> test -z $(ls -A /var/lib/ceph) 2024-04-26T03:30:34.176 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-26T03:30:34.275 INFO:teuthology.run_tasks:Running task kernel... 2024-04-26T03:30:34.289 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-04-26T03:30:34.289 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}, 'mgr.x': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-04-26T03:30:34.289 DEBUG:teuthology.orchestra.run.smithi028:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T03:30:34.290 DEBUG:teuthology.orchestra.run.smithi042:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T03:30:34.290 DEBUG:teuthology.orchestra.run.smithi066:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T03:30:34.291 DEBUG:teuthology.orchestra.run.smithi092:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T03:30:34.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T03:30:34.308 DEBUG:teuthology.orchestra.run.smithi092:> uname -r 2024-04-26T03:30:34.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T03:30:34.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T03:30:34.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T03:30:34.310 DEBUG:teuthology.orchestra.run.smithi028:> uname -r 2024-04-26T03:30:34.310 DEBUG:teuthology.orchestra.run.smithi066:> uname -r 2024-04-26T03:30:34.311 DEBUG:teuthology.orchestra.run.smithi042:> uname -r 2024-04-26T03:30:34.365 INFO:teuthology.orchestra.run.smithi092.stdout:5.14.0-437.el9.x86_64 2024-04-26T03:30:34.365 INFO:teuthology.task.kernel:Running kernel on smithi092: 5.14.0-437.el9.x86_64 2024-04-26T03:30:34.365 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum install -y kernel 2024-04-26T03:30:34.368 INFO:teuthology.orchestra.run.smithi028.stdout:5.14.0-437.el9.x86_64 2024-04-26T03:30:34.368 INFO:teuthology.task.kernel:Running kernel on smithi028: 5.14.0-437.el9.x86_64 2024-04-26T03:30:34.368 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum install -y kernel 2024-04-26T03:30:34.370 INFO:teuthology.orchestra.run.smithi042.stdout:5.14.0-437.el9.x86_64 2024-04-26T03:30:34.370 INFO:teuthology.orchestra.run.smithi066.stdout:5.14.0-437.el9.x86_64 2024-04-26T03:30:34.371 INFO:teuthology.task.kernel:Running kernel on smithi042: 5.14.0-437.el9.x86_64 2024-04-26T03:30:34.371 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum install -y kernel 2024-04-26T03:30:34.371 INFO:teuthology.task.kernel:Running kernel on smithi066: 5.14.0-437.el9.x86_64 2024-04-26T03:30:34.371 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum install -y kernel 2024-04-26T03:30:36.636 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-04-26T03:30:36.694 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-04-26T03:30:36.767 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-26T03:30:37.002 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - BaseOS 7.5 MB/s | 8.0 MB 00:01 2024-04-26T03:30:39.402 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - AppStream 13 MB/s | 19 MB 00:01 2024-04-26T03:30:39.488 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - AppStream 14 MB/s | 19 MB 00:01 2024-04-26T03:30:39.622 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - AppStream 10 MB/s | 19 MB 00:01 2024-04-26T03:30:39.753 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - AppStream 9.4 MB/s | 19 MB 00:02 2024-04-26T03:30:44.056 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - CRB 7.1 MB/s | 6.2 MB 00:00 2024-04-26T03:30:44.436 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - CRB 4.5 MB/s | 6.2 MB 00:01 2024-04-26T03:30:44.788 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - CRB 5.3 MB/s | 6.2 MB 00:01 2024-04-26T03:30:44.970 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - CRB 3.8 MB/s | 6.2 MB 00:01 2024-04-26T03:30:45.737 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-04-26T03:30:46.292 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - Extras packages 25 kB/s | 16 kB 00:00 2024-04-26T03:30:46.676 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - Extras packages 49 kB/s | 16 kB 00:00 2024-04-26T03:30:47.138 INFO:teuthology.orchestra.run.smithi066.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-04-26T03:30:47.378 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - Extras packages 12 kB/s | 16 kB 00:01 2024-04-26T03:30:47.737 INFO:teuthology.orchestra.run.smithi092.stdout:Extra Packages for Enterprise Linux 22 MB/s | 21 MB 00:00 2024-04-26T03:30:48.604 INFO:teuthology.orchestra.run.smithi028.stdout:Extra Packages for Enterprise Linux 9.6 MB/s | 21 MB 00:02 2024-04-26T03:30:48.663 INFO:teuthology.orchestra.run.smithi042.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-04-26T03:30:52.322 INFO:teuthology.orchestra.run.smithi066.stdout:lab-extras 48 kB/s | 1.7 kB 00:00 2024-04-26T03:30:52.802 INFO:teuthology.orchestra.run.smithi092.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-04-26T03:30:53.894 INFO:teuthology.orchestra.run.smithi028.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-26T03:30:53.921 INFO:teuthology.orchestra.run.smithi042.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-04-26T03:30:54.253 INFO:teuthology.orchestra.run.smithi066.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-26T03:30:54.253 INFO:teuthology.orchestra.run.smithi066.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-26T03:30:54.344 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-04-26T03:30:54.345 INFO:teuthology.orchestra.run.smithi066.stdout:Nothing to do. 2024-04-26T03:30:54.345 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-04-26T03:30:54.455 DEBUG:teuthology.orchestra.run.smithi066:> echo no | sudo yum reinstall kernel || true 2024-04-26T03:30:54.716 INFO:teuthology.orchestra.run.smithi092.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-26T03:30:54.716 INFO:teuthology.orchestra.run.smithi092.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-26T03:30:54.804 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-26T03:30:54.805 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-04-26T03:30:54.805 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-26T03:30:54.906 DEBUG:teuthology.orchestra.run.smithi092:> echo no | sudo yum reinstall kernel || true 2024-04-26T03:30:54.978 INFO:teuthology.orchestra.run.smithi066.stdout:Last metadata expiration check: 0:00:02 ago on Fri 26 Apr 2024 03:30:52 AM UTC. 2024-04-26T03:30:55.102 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-04-26T03:30:55.118 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:30:55.119 INFO:teuthology.orchestra.run.smithi066.stdout: Package Architecture Version Repository Size 2024-04-26T03:30:55.119 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:30:55.119 INFO:teuthology.orchestra.run.smithi066.stdout:Reinstalling: 2024-04-26T03:30:55.119 INFO:teuthology.orchestra.run.smithi066.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T03:30:55.119 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:30:55.119 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction Summary 2024-04-26T03:30:55.119 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:30:55.119 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:30:55.119 INFO:teuthology.orchestra.run.smithi066.stdout:Total download size: 255 k 2024-04-26T03:30:55.119 INFO:teuthology.orchestra.run.smithi066.stdout:Installed size: 0 2024-04-26T03:30:55.119 INFO:teuthology.orchestra.run.smithi066.stderr:Operation aborted. 2024-04-26T03:30:55.176 INFO:teuthology.orchestra.run.smithi066.stdout:Is this ok [y/N]: 2024-04-26T03:30:55.177 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum reinstall -y kernel || true 2024-04-26T03:30:55.418 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:03 ago on Fri 26 Apr 2024 03:30:52 AM UTC. 2024-04-26T03:30:55.537 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-26T03:30:55.545 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:30:55.546 INFO:teuthology.orchestra.run.smithi092.stdout: Package Architecture Version Repository Size 2024-04-26T03:30:55.546 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:30:55.546 INFO:teuthology.orchestra.run.smithi092.stdout:Reinstalling: 2024-04-26T03:30:55.546 INFO:teuthology.orchestra.run.smithi092.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T03:30:55.546 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:30:55.547 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-04-26T03:30:55.547 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:30:55.547 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:30:55.547 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 255 k 2024-04-26T03:30:55.547 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 0 2024-04-26T03:30:55.548 INFO:teuthology.orchestra.run.smithi092.stderr:Operation aborted. 2024-04-26T03:30:55.604 INFO:teuthology.orchestra.run.smithi092.stdout:Is this ok [y/N]: 2024-04-26T03:30:55.604 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum reinstall -y kernel || true 2024-04-26T03:30:55.693 INFO:teuthology.orchestra.run.smithi066.stdout:Last metadata expiration check: 0:00:03 ago on Fri 26 Apr 2024 03:30:52 AM UTC. 2024-04-26T03:30:55.815 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-04-26T03:30:55.816 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:30:55.816 INFO:teuthology.orchestra.run.smithi066.stdout: Package Architecture Version Repository Size 2024-04-26T03:30:55.816 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:30:55.816 INFO:teuthology.orchestra.run.smithi066.stdout:Reinstalling: 2024-04-26T03:30:55.816 INFO:teuthology.orchestra.run.smithi066.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T03:30:55.816 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:30:55.816 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction Summary 2024-04-26T03:30:55.816 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:30:55.816 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:30:55.817 INFO:teuthology.orchestra.run.smithi066.stdout:Total download size: 255 k 2024-04-26T03:30:55.817 INFO:teuthology.orchestra.run.smithi066.stdout:Installed size: 0 2024-04-26T03:30:55.817 INFO:teuthology.orchestra.run.smithi066.stdout:Downloading Packages: 2024-04-26T03:30:55.858 INFO:teuthology.orchestra.run.smithi028.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-26T03:30:55.858 INFO:teuthology.orchestra.run.smithi028.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-26T03:30:55.861 INFO:teuthology.orchestra.run.smithi042.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-26T03:30:55.861 INFO:teuthology.orchestra.run.smithi042.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-26T03:30:55.949 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-04-26T03:30:55.950 INFO:teuthology.orchestra.run.smithi028.stdout:Nothing to do. 2024-04-26T03:30:55.950 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-04-26T03:30:55.959 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-04-26T03:30:55.960 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-04-26T03:30:55.960 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-04-26T03:30:56.062 DEBUG:teuthology.orchestra.run.smithi028:> echo no | sudo yum reinstall kernel || true 2024-04-26T03:30:56.065 DEBUG:teuthology.orchestra.run.smithi042:> echo no | sudo yum reinstall kernel || true 2024-04-26T03:30:56.095 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:04 ago on Fri 26 Apr 2024 03:30:52 AM UTC. 2024-04-26T03:30:56.211 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-26T03:30:56.211 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:30:56.212 INFO:teuthology.orchestra.run.smithi092.stdout: Package Architecture Version Repository Size 2024-04-26T03:30:56.212 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:30:56.212 INFO:teuthology.orchestra.run.smithi092.stdout:Reinstalling: 2024-04-26T03:30:56.212 INFO:teuthology.orchestra.run.smithi092.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T03:30:56.212 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:30:56.212 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-04-26T03:30:56.212 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:30:56.212 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:30:56.212 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 255 k 2024-04-26T03:30:56.212 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 0 2024-04-26T03:30:56.212 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-04-26T03:30:56.290 INFO:teuthology.orchestra.run.smithi066.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.3 MB/s | 255 kB 00:00 2024-04-26T03:30:56.291 INFO:teuthology.orchestra.run.smithi066.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:30:56.291 INFO:teuthology.orchestra.run.smithi066.stdout:Total 536 kB/s | 255 kB 00:00 2024-04-26T03:30:56.313 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction check 2024-04-26T03:30:56.506 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction check succeeded. 2024-04-26T03:30:56.506 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction test 2024-04-26T03:30:56.511 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction test succeeded. 2024-04-26T03:30:56.512 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction 2024-04-26T03:30:56.582 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:00:03 ago on Fri 26 Apr 2024 03:30:53 AM UTC. 2024-04-26T03:30:56.627 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:00:03 ago on Fri 26 Apr 2024 03:30:53 AM UTC. 2024-04-26T03:30:56.641 INFO:teuthology.orchestra.run.smithi066.stdout: Preparing : 1/1 2024-04-26T03:30:56.708 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-04-26T03:30:56.716 INFO:teuthology.orchestra.run.smithi066.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T03:30:56.717 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T03:30:56.722 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:30:56.723 INFO:teuthology.orchestra.run.smithi028.stdout: Package Architecture Version Repository Size 2024-04-26T03:30:56.723 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:30:56.723 INFO:teuthology.orchestra.run.smithi028.stdout:Reinstalling: 2024-04-26T03:30:56.723 INFO:teuthology.orchestra.run.smithi028.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T03:30:56.723 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:30:56.723 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-04-26T03:30:56.723 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:30:56.723 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:30:56.723 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 255 k 2024-04-26T03:30:56.723 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 0 2024-04-26T03:30:56.724 INFO:teuthology.orchestra.run.smithi028.stderr:Operation aborted. 2024-04-26T03:30:56.754 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-04-26T03:30:56.779 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:30:56.779 INFO:teuthology.orchestra.run.smithi042.stdout: Package Architecture Version Repository Size 2024-04-26T03:30:56.779 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:30:56.779 INFO:teuthology.orchestra.run.smithi042.stdout:Reinstalling: 2024-04-26T03:30:56.779 INFO:teuthology.orchestra.run.smithi042.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T03:30:56.780 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:30:56.780 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-04-26T03:30:56.780 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:30:56.780 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:30:56.780 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 255 k 2024-04-26T03:30:56.780 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 0 2024-04-26T03:30:56.780 INFO:teuthology.orchestra.run.smithi042.stderr:Operation aborted. 2024-04-26T03:30:56.782 INFO:teuthology.orchestra.run.smithi028.stdout:Is this ok [y/N]: 2024-04-26T03:30:56.783 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum reinstall -y kernel || true 2024-04-26T03:30:56.841 INFO:teuthology.orchestra.run.smithi042.stdout:Is this ok [y/N]: 2024-04-26T03:30:56.841 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum reinstall -y kernel || true 2024-04-26T03:30:56.928 INFO:teuthology.orchestra.run.smithi092.stdout:kernel-5.14.0-437.el9.x86_64.rpm 574 kB/s | 255 kB 00:00 2024-04-26T03:30:56.929 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:30:56.929 INFO:teuthology.orchestra.run.smithi092.stdout:Total 355 kB/s | 255 kB 00:00 2024-04-26T03:30:56.943 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-04-26T03:30:57.130 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-04-26T03:30:57.130 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-04-26T03:30:57.135 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-04-26T03:30:57.135 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-04-26T03:30:57.281 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-04-26T03:30:57.297 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:00:04 ago on Fri 26 Apr 2024 03:30:53 AM UTC. 2024-04-26T03:30:57.358 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:00:04 ago on Fri 26 Apr 2024 03:30:53 AM UTC. 2024-04-26T03:30:57.362 INFO:teuthology.orchestra.run.smithi092.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T03:30:57.366 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T03:30:57.419 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-04-26T03:30:57.420 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:30:57.420 INFO:teuthology.orchestra.run.smithi028.stdout: Package Architecture Version Repository Size 2024-04-26T03:30:57.420 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:30:57.420 INFO:teuthology.orchestra.run.smithi028.stdout:Reinstalling: 2024-04-26T03:30:57.420 INFO:teuthology.orchestra.run.smithi028.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T03:30:57.420 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:30:57.420 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-04-26T03:30:57.420 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:30:57.420 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:30:57.420 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 255 k 2024-04-26T03:30:57.420 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 0 2024-04-26T03:30:57.420 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-04-26T03:30:57.430 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T03:30:57.430 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T03:30:57.477 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-04-26T03:30:57.478 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:30:57.478 INFO:teuthology.orchestra.run.smithi042.stdout: Package Architecture Version Repository Size 2024-04-26T03:30:57.478 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:30:57.478 INFO:teuthology.orchestra.run.smithi042.stdout:Reinstalling: 2024-04-26T03:30:57.478 INFO:teuthology.orchestra.run.smithi042.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T03:30:57.479 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:30:57.479 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-04-26T03:30:57.479 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:30:57.479 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:30:57.479 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 255 k 2024-04-26T03:30:57.479 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 0 2024-04-26T03:30:57.479 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-04-26T03:30:57.695 INFO:teuthology.orchestra.run.smithi028.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.8 MB/s | 255 kB 00:00 2024-04-26T03:30:57.696 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:30:57.696 INFO:teuthology.orchestra.run.smithi028.stdout:Total 924 kB/s | 255 kB 00:00 2024-04-26T03:30:57.710 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-04-26T03:30:57.782 INFO:teuthology.orchestra.run.smithi042.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.1 MB/s | 255 kB 00:00 2024-04-26T03:30:57.782 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:30:57.782 INFO:teuthology.orchestra.run.smithi042.stdout:Total 838 kB/s | 255 kB 00:00 2024-04-26T03:30:57.798 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-04-26T03:30:57.890 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-04-26T03:30:57.891 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-04-26T03:30:57.896 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-04-26T03:30:57.896 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-04-26T03:30:57.988 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-04-26T03:30:57.988 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-04-26T03:30:57.994 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-04-26T03:30:57.994 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-04-26T03:30:58.017 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T03:30:58.017 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T03:30:58.043 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-04-26T03:30:58.113 INFO:teuthology.orchestra.run.smithi028.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T03:30:58.113 INFO:teuthology.orchestra.run.smithi028.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T03:30:58.132 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-04-26T03:30:58.147 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T03:30:58.147 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:30:58.147 INFO:teuthology.orchestra.run.smithi066.stdout:Reinstalled: 2024-04-26T03:30:58.147 INFO:teuthology.orchestra.run.smithi066.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-26T03:30:58.147 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:30:58.147 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-04-26T03:30:58.216 INFO:teuthology.orchestra.run.smithi042.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T03:30:58.217 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T03:30:58.324 DEBUG:teuthology.orchestra.run.smithi066:> rpm -q kernel | sort -rV | head -n 1 2024-04-26T03:30:58.371 INFO:teuthology.orchestra.run.smithi066.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-26T03:30:58.372 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-26T03:30:58.372 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-26T03:30:58.372 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-04-26T03:30:58.372 DEBUG:teuthology.orchestra.run.smithi066:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-26T03:30:58.401 INFO:teuthology.orchestra.run.smithi066.stdout:ttyS1 2024-04-26T03:30:58.431 DEBUG:teuthology.parallel:result is None 2024-04-26T03:30:58.745 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T03:30:58.745 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:30:58.745 INFO:teuthology.orchestra.run.smithi092.stdout:Reinstalled: 2024-04-26T03:30:58.745 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-26T03:30:58.745 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:30:58.745 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-26T03:30:58.777 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T03:30:58.777 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T03:30:58.886 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T03:30:58.886 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T03:30:58.949 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q kernel | sort -rV | head -n 1 2024-04-26T03:30:59.036 INFO:teuthology.orchestra.run.smithi092.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-26T03:30:59.037 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-26T03:30:59.037 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-26T03:30:59.037 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2024-04-26T03:30:59.037 DEBUG:teuthology.orchestra.run.smithi092:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-26T03:30:59.068 INFO:teuthology.orchestra.run.smithi092.stdout:ttyS1 2024-04-26T03:30:59.093 DEBUG:teuthology.parallel:result is None 2024-04-26T03:30:59.444 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T03:30:59.444 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:30:59.445 INFO:teuthology.orchestra.run.smithi028.stdout:Reinstalled: 2024-04-26T03:30:59.445 INFO:teuthology.orchestra.run.smithi028.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-26T03:30:59.445 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:30:59.445 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-04-26T03:30:59.582 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T03:30:59.582 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:30:59.582 INFO:teuthology.orchestra.run.smithi042.stdout:Reinstalled: 2024-04-26T03:30:59.582 INFO:teuthology.orchestra.run.smithi042.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-26T03:30:59.582 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:30:59.582 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-04-26T03:30:59.632 DEBUG:teuthology.orchestra.run.smithi028:> rpm -q kernel | sort -rV | head -n 1 2024-04-26T03:30:59.679 INFO:teuthology.orchestra.run.smithi028.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-26T03:30:59.680 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-26T03:30:59.680 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-26T03:30:59.680 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-26T03:30:59.680 DEBUG:teuthology.orchestra.run.smithi028:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-26T03:30:59.715 INFO:teuthology.orchestra.run.smithi028.stdout:ttyS1 2024-04-26T03:30:59.742 DEBUG:teuthology.parallel:result is None 2024-04-26T03:30:59.748 DEBUG:teuthology.orchestra.run.smithi042:> rpm -q kernel | sort -rV | head -n 1 2024-04-26T03:30:59.795 INFO:teuthology.orchestra.run.smithi042.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-26T03:30:59.796 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-26T03:30:59.796 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-26T03:30:59.796 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-26T03:30:59.796 DEBUG:teuthology.orchestra.run.smithi042:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-26T03:30:59.824 INFO:teuthology.orchestra.run.smithi042.stdout:ttyS1 2024-04-26T03:30:59.848 DEBUG:teuthology.parallel:result is None 2024-04-26T03:30:59.848 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-26T03:30:59.856 INFO:teuthology.task.internal:Creating test directory... 2024-04-26T03:30:59.856 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T03:30:59.859 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T03:30:59.891 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T03:30:59.893 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T03:30:59.910 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-26T03:30:59.917 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-26T03:30:59.923 INFO:teuthology.task.internal:Creating archive directory... 2024-04-26T03:30:59.924 DEBUG:teuthology.orchestra.run.smithi028:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T03:30:59.926 DEBUG:teuthology.orchestra.run.smithi042:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T03:30:59.946 DEBUG:teuthology.orchestra.run.smithi066:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T03:30:59.952 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T03:30:59.985 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-26T03:30:59.995 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-26T03:30:59.996 DEBUG:teuthology.orchestra.run.smithi028:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T03:30:59.999 DEBUG:teuthology.orchestra.run.smithi042:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T03:31:00.009 DEBUG:teuthology.orchestra.run.smithi066:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T03:31:00.022 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T03:31:00.026 INFO:teuthology.orchestra.run.smithi028.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T03:31:00.036 INFO:teuthology.orchestra.run.smithi028.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T03:31:00.039 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T03:31:00.049 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T03:31:00.057 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T03:31:00.066 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T03:31:00.070 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T03:31:00.081 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T03:31:00.083 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-26T03:31:00.090 INFO:teuthology.task.internal:Configuring sudo... 2024-04-26T03:31:00.090 DEBUG:teuthology.orchestra.run.smithi028:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T03:31:00.092 DEBUG:teuthology.orchestra.run.smithi042:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T03:31:00.095 DEBUG:teuthology.orchestra.run.smithi066:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T03:31:00.097 DEBUG:teuthology.orchestra.run.smithi092:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T03:31:00.133 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-26T03:31:00.142 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-26T03:31:00.142 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T03:31:00.157 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T03:31:00.159 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T03:31:00.165 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T03:31:00.189 DEBUG:teuthology.orchestra.run.smithi028:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T03:31:00.252 DEBUG:teuthology.orchestra.run.smithi028:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T03:31:00.300 DEBUG:teuthology.orchestra.run.smithi028:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T03:31:00.383 DEBUG:teuthology.orchestra.run.smithi028:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T03:31:00.421 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T03:31:00.422 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T03:31:00.485 DEBUG:teuthology.orchestra.run.smithi042:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T03:31:00.526 DEBUG:teuthology.orchestra.run.smithi042:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T03:31:00.581 DEBUG:teuthology.orchestra.run.smithi042:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T03:31:00.665 DEBUG:teuthology.orchestra.run.smithi042:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T03:31:00.705 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-26T03:31:00.705 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T03:31:00.770 DEBUG:teuthology.orchestra.run.smithi066:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T03:31:00.814 DEBUG:teuthology.orchestra.run.smithi066:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T03:31:00.870 DEBUG:teuthology.orchestra.run.smithi066:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T03:31:00.959 DEBUG:teuthology.orchestra.run.smithi066:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T03:31:00.999 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-26T03:31:01.000 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T03:31:01.068 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T03:31:01.112 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T03:31:01.169 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T03:31:01.256 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T03:31:01.294 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-26T03:31:01.295 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T03:31:01.360 DEBUG:teuthology.orchestra.run.smithi028:> sudo service rsyslog restart 2024-04-26T03:31:01.362 DEBUG:teuthology.orchestra.run.smithi042:> sudo service rsyslog restart 2024-04-26T03:31:01.366 DEBUG:teuthology.orchestra.run.smithi066:> sudo service rsyslog restart 2024-04-26T03:31:01.368 DEBUG:teuthology.orchestra.run.smithi092:> sudo service rsyslog restart 2024-04-26T03:31:01.400 INFO:teuthology.orchestra.run.smithi042.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T03:31:01.402 INFO:teuthology.orchestra.run.smithi028.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T03:31:01.416 INFO:teuthology.orchestra.run.smithi066.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T03:31:01.442 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T03:31:01.825 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-26T03:31:01.834 INFO:teuthology.task.internal:Starting timer... 2024-04-26T03:31:01.834 INFO:teuthology.run_tasks:Running task pcp... 2024-04-26T03:31:01.922 INFO:teuthology.run_tasks:Running task selinux... 2024-04-26T03:31:01.940 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-26T03:31:01.941 DEBUG:teuthology.orchestra.run.smithi028:> sudo service auditd rotate 2024-04-26T03:31:02.003 INFO:teuthology.orchestra.run.smithi028.stdout:Rotating logs: 2024-04-26T03:31:02.005 DEBUG:teuthology.orchestra.run.smithi042:> sudo service auditd rotate 2024-04-26T03:31:02.079 INFO:teuthology.orchestra.run.smithi042.stdout:Rotating logs: 2024-04-26T03:31:02.081 DEBUG:teuthology.orchestra.run.smithi066:> sudo service auditd rotate 2024-04-26T03:31:02.129 INFO:teuthology.orchestra.run.smithi066.stdout:Rotating logs: 2024-04-26T03:31:02.131 DEBUG:teuthology.orchestra.run.smithi092:> sudo service auditd rotate 2024-04-26T03:31:02.178 INFO:teuthology.orchestra.run.smithi092.stdout:Rotating logs: 2024-04-26T03:31:02.180 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-26T03:31:02.180 DEBUG:teuthology.orchestra.run.smithi028:> /usr/sbin/getenforce 2024-04-26T03:31:02.204 INFO:teuthology.orchestra.run.smithi028.stdout:Permissive 2024-04-26T03:31:02.205 DEBUG:teuthology.orchestra.run.smithi042:> /usr/sbin/getenforce 2024-04-26T03:31:02.224 INFO:teuthology.orchestra.run.smithi042.stdout:Permissive 2024-04-26T03:31:02.224 DEBUG:teuthology.orchestra.run.smithi066:> /usr/sbin/getenforce 2024-04-26T03:31:02.245 INFO:teuthology.orchestra.run.smithi066.stdout:Permissive 2024-04-26T03:31:02.245 DEBUG:teuthology.orchestra.run.smithi092:> /usr/sbin/getenforce 2024-04-26T03:31:02.271 INFO:teuthology.orchestra.run.smithi092.stdout:Permissive 2024-04-26T03:31:02.272 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi028.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi042.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi066.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi092.front.sepia.ceph.com': 'permissive'} 2024-04-26T03:31:02.272 DEBUG:teuthology.orchestra.run.smithi028:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T03:31:02.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T03:31:02.298 DEBUG:teuthology.orchestra.run.smithi042:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T03:31:02.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T03:31:02.324 DEBUG:teuthology.orchestra.run.smithi066:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T03:31:02.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T03:31:02.349 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T03:31:02.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T03:31:02.374 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-26T03:31:02.374 DEBUG:teuthology.orchestra.run.smithi028:> sudo /usr/sbin/setenforce permissive 2024-04-26T03:31:02.407 DEBUG:teuthology.orchestra.run.smithi042:> sudo /usr/sbin/setenforce permissive 2024-04-26T03:31:02.431 DEBUG:teuthology.orchestra.run.smithi066:> sudo /usr/sbin/setenforce permissive 2024-04-26T03:31:02.455 DEBUG:teuthology.orchestra.run.smithi092:> sudo /usr/sbin/setenforce permissive 2024-04-26T03:31:02.498 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-26T03:31:02.507 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-26T03:31:02.524 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-26T03:31:02.525 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi028.front.sepia.ceph.com,smithi042.front.sepia.ceph.com,smithi066.front.sepia.ceph.com,smithi092.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-26T03:34:39.978 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi028.front.sepia.ceph.com'), Remote(name='ubuntu@smithi042.front.sepia.ceph.com'), Remote(name='ubuntu@smithi066.front.sepia.ceph.com'), Remote(name='ubuntu@smithi092.front.sepia.ceph.com')] 2024-04-26T03:34:39.979 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi028.front.sepia.ceph.com' 2024-04-26T03:34:39.980 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T03:34:40.056 DEBUG:teuthology.orchestra.run.smithi028:> true 2024-04-26T03:34:40.137 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi028.front.sepia.ceph.com' 2024-04-26T03:34:40.137 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-04-26T03:34:40.138 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T03:34:40.223 DEBUG:teuthology.orchestra.run.smithi042:> true 2024-04-26T03:34:40.299 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-04-26T03:34:40.299 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-04-26T03:34:40.300 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T03:34:40.379 DEBUG:teuthology.orchestra.run.smithi066:> true 2024-04-26T03:34:40.468 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-04-26T03:34:40.469 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-04-26T03:34:40.469 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T03:34:40.543 DEBUG:teuthology.orchestra.run.smithi092:> true 2024-04-26T03:34:40.622 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-04-26T03:34:40.623 INFO:teuthology.run_tasks:Running task clock... 2024-04-26T03:34:40.635 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-26T03:34:40.636 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T03:34:40.636 DEBUG:teuthology.orchestra.run.smithi028:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T03:34:40.639 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T03:34:40.639 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T03:34:40.642 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T03:34:40.642 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T03:34:40.645 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T03:34:40.645 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T03:34:40.671 INFO:teuthology.orchestra.run.smithi028.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T03:34:40.673 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T03:34:40.675 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T03:34:40.686 INFO:teuthology.orchestra.run.smithi028.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T03:34:40.689 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T03:34:40.690 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T03:34:40.691 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T03:34:40.704 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T03:34:40.712 INFO:teuthology.orchestra.run.smithi028.stderr:sudo: ntpd: command not found 2024-04-26T03:34:40.714 INFO:teuthology.orchestra.run.smithi042.stderr:sudo: ntpd: command not found 2024-04-26T03:34:40.717 INFO:teuthology.orchestra.run.smithi066.stderr:sudo: ntpd: command not found 2024-04-26T03:34:40.724 INFO:teuthology.orchestra.run.smithi028.stdout:506 Cannot talk to daemon 2024-04-26T03:34:40.726 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: ntpd: command not found 2024-04-26T03:34:40.726 INFO:teuthology.orchestra.run.smithi042.stdout:506 Cannot talk to daemon 2024-04-26T03:34:40.728 INFO:teuthology.orchestra.run.smithi066.stdout:506 Cannot talk to daemon 2024-04-26T03:34:40.736 INFO:teuthology.orchestra.run.smithi092.stdout:506 Cannot talk to daemon 2024-04-26T03:34:40.738 INFO:teuthology.orchestra.run.smithi028.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T03:34:40.741 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T03:34:40.742 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T03:34:40.750 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T03:34:40.752 INFO:teuthology.orchestra.run.smithi028.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T03:34:40.755 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T03:34:40.756 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T03:34:40.763 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T03:34:40.810 INFO:teuthology.orchestra.run.smithi028.stderr:bash: line 1: ntpq: command not found 2024-04-26T03:34:40.812 INFO:teuthology.orchestra.run.smithi042.stderr:bash: line 1: ntpq: command not found 2024-04-26T03:34:40.814 INFO:teuthology.orchestra.run.smithi066.stderr:bash: line 1: ntpq: command not found 2024-04-26T03:34:40.814 INFO:teuthology.orchestra.run.smithi028.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T03:34:40.814 INFO:teuthology.orchestra.run.smithi028.stdout:=============================================================================== 2024-04-26T03:34:40.815 INFO:teuthology.orchestra.run.smithi028.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T03:34:40.815 INFO:teuthology.orchestra.run.smithi028.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T03:34:40.815 INFO:teuthology.orchestra.run.smithi028.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T03:34:40.815 INFO:teuthology.orchestra.run.smithi028.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T03:34:40.816 INFO:teuthology.orchestra.run.smithi042.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T03:34:40.816 INFO:teuthology.orchestra.run.smithi042.stdout:=============================================================================== 2024-04-26T03:34:40.816 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T03:34:40.816 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T03:34:40.816 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T03:34:40.816 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T03:34:40.818 INFO:teuthology.orchestra.run.smithi066.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T03:34:40.818 INFO:teuthology.orchestra.run.smithi066.stdout:=============================================================================== 2024-04-26T03:34:40.818 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T03:34:40.818 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T03:34:40.818 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T03:34:40.818 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T03:34:40.821 INFO:teuthology.orchestra.run.smithi092.stderr:bash: line 1: ntpq: command not found 2024-04-26T03:34:40.824 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T03:34:40.824 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2024-04-26T03:34:40.824 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T03:34:40.824 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T03:34:40.824 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T03:34:40.824 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T03:34:40.825 INFO:teuthology.run_tasks:Running task install... 2024-04-26T03:34:40.834 DEBUG:teuthology.task.install:project ceph 2024-04-26T03:34:40.835 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'ddeaa91797339a19172c3036ff48cca58c12f448'}} 2024-04-26T03:34:40.835 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'ddeaa91797339a19172c3036ff48cca58c12f448'} 2024-04-26T03:34:40.835 INFO:teuthology.task.install:Using flavor: default 2024-04-26T03:34:40.841 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-26T03:34:40.842 INFO:teuthology.task.install:extra packages: [] 2024-04-26T03:34:40.842 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ddeaa91797339a19172c3036ff48cca58c12f448', 'tag': None, 'wait_for_package': False} 2024-04-26T03:34:40.843 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ddeaa91797339a19172c3036ff48cca58c12f448 2024-04-26T03:34:40.845 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ddeaa91797339a19172c3036ff48cca58c12f448', 'tag': None, 'wait_for_package': False} 2024-04-26T03:34:40.845 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ddeaa91797339a19172c3036ff48cca58c12f448 2024-04-26T03:34:40.846 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ddeaa91797339a19172c3036ff48cca58c12f448', 'tag': None, 'wait_for_package': False} 2024-04-26T03:34:40.847 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ddeaa91797339a19172c3036ff48cca58c12f448 2024-04-26T03:34:40.848 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ddeaa91797339a19172c3036ff48cca58c12f448', 'tag': None, 'wait_for_package': False} 2024-04-26T03:34:40.848 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ddeaa91797339a19172c3036ff48cca58c12f448 2024-04-26T03:34:41.006 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/centos/9/flavors/default/ 2024-04-26T03:34:41.006 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2457.gddeaa917 2024-04-26T03:34:41.074 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/centos/9/flavors/default/ 2024-04-26T03:34:41.074 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2457.gddeaa917 2024-04-26T03:34:41.078 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/centos/9/flavors/default/ 2024-04-26T03:34:41.078 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2457.gddeaa917 2024-04-26T03:34:41.086 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/centos/9/flavors/default/ 2024-04-26T03:34:41.086 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2457.gddeaa917 2024-04-26T03:34:41.181 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T03:34:41.181 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-26T03:34:41.181 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T03:34:41.207 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-26T03:34:41.207 DEBUG:teuthology.orchestra.run.smithi092:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ddeaa91797339a19172c3036ff48cca58c12f448/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T03:34:41.244 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T03:34:41.245 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T03:34:41.245 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T03:34:41.255 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T03:34:41.256 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-26T03:34:41.256 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T03:34:41.258 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T03:34:41.258 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-26T03:34:41.258 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T03:34:41.271 DEBUG:teuthology.orchestra.run.smithi092:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T03:34:41.273 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-26T03:34:41.273 DEBUG:teuthology.orchestra.run.smithi028:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ddeaa91797339a19172c3036ff48cca58c12f448/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T03:34:41.286 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-26T03:34:41.287 DEBUG:teuthology.orchestra.run.smithi066:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ddeaa91797339a19172c3036ff48cca58c12f448/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T03:34:41.288 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-26T03:34:41.288 DEBUG:teuthology.orchestra.run.smithi042:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ddeaa91797339a19172c3036ff48cca58c12f448/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T03:34:41.341 DEBUG:teuthology.orchestra.run.smithi028:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T03:34:41.344 DEBUG:teuthology.orchestra.run.smithi092:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T03:34:41.353 DEBUG:teuthology.orchestra.run.smithi066:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T03:34:41.358 DEBUG:teuthology.orchestra.run.smithi042:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T03:34:41.407 INFO:teuthology.orchestra.run.smithi092.stdout:check_obsoletes = 1 2024-04-26T03:34:41.408 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum clean all 2024-04-26T03:34:41.418 DEBUG:teuthology.orchestra.run.smithi028:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T03:34:41.430 DEBUG:teuthology.orchestra.run.smithi066:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T03:34:41.438 DEBUG:teuthology.orchestra.run.smithi042:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T03:34:41.483 INFO:teuthology.orchestra.run.smithi028.stdout:check_obsoletes = 1 2024-04-26T03:34:41.485 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum clean all 2024-04-26T03:34:41.496 INFO:teuthology.orchestra.run.smithi066.stdout:check_obsoletes = 1 2024-04-26T03:34:41.497 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum clean all 2024-04-26T03:34:41.505 INFO:teuthology.orchestra.run.smithi042.stdout:check_obsoletes = 1 2024-04-26T03:34:41.507 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum clean all 2024-04-26T03:34:41.630 INFO:teuthology.orchestra.run.smithi092.stdout:48 files removed 2024-04-26T03:34:41.657 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-radosgw 2024-04-26T03:34:41.719 INFO:teuthology.orchestra.run.smithi028.stdout:48 files removed 2024-04-26T03:34:41.733 INFO:teuthology.orchestra.run.smithi066.stdout:48 files removed 2024-04-26T03:34:41.746 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install ceph-radosgw 2024-04-26T03:34:41.749 INFO:teuthology.orchestra.run.smithi042.stdout:48 files removed 2024-04-26T03:34:41.760 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum -y install ceph-radosgw 2024-04-26T03:34:41.776 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-radosgw 2024-04-26T03:34:42.272 INFO:teuthology.orchestra.run.smithi092.stdout:ceph packages for x86_64 234 kB/s | 83 kB 00:00 2024-04-26T03:34:42.398 INFO:teuthology.orchestra.run.smithi066.stdout:ceph packages for x86_64 229 kB/s | 83 kB 00:00 2024-04-26T03:34:42.405 INFO:teuthology.orchestra.run.smithi028.stdout:ceph packages for x86_64 215 kB/s | 83 kB 00:00 2024-04-26T03:34:42.469 INFO:teuthology.orchestra.run.smithi042.stdout:ceph packages for x86_64 198 kB/s | 83 kB 00:00 2024-04-26T03:34:42.615 INFO:teuthology.orchestra.run.smithi092.stdout:ceph noarch packages 39 kB/s | 11 kB 00:00 2024-04-26T03:34:42.730 INFO:teuthology.orchestra.run.smithi066.stdout:ceph noarch packages 40 kB/s | 11 kB 00:00 2024-04-26T03:34:42.747 INFO:teuthology.orchestra.run.smithi028.stdout:ceph noarch packages 39 kB/s | 11 kB 00:00 2024-04-26T03:34:42.845 INFO:teuthology.orchestra.run.smithi042.stdout:ceph noarch packages 35 kB/s | 11 kB 00:00 2024-04-26T03:34:42.923 INFO:teuthology.orchestra.run.smithi092.stdout:ceph source packages 6.9 kB/s | 1.9 kB 00:00 2024-04-26T03:34:43.148 INFO:teuthology.orchestra.run.smithi028.stdout:ceph source packages 5.2 kB/s | 1.9 kB 00:00 2024-04-26T03:34:43.213 INFO:teuthology.orchestra.run.smithi066.stdout:ceph source packages 4.2 kB/s | 1.9 kB 00:00 2024-04-26T03:34:43.296 INFO:teuthology.orchestra.run.smithi042.stdout:ceph source packages 4.6 kB/s | 1.9 kB 00:00 2024-04-26T03:34:43.510 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-04-26T03:34:43.967 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-04-26T03:34:44.073 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-26T03:34:44.166 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - BaseOS 8.8 MB/s | 8.0 MB 00:00 2024-04-26T03:34:45.552 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - AppStream 19 MB/s | 19 MB 00:00 2024-04-26T03:34:45.977 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-04-26T03:34:46.299 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - AppStream 17 MB/s | 19 MB 00:01 2024-04-26T03:34:46.759 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - AppStream 13 MB/s | 19 MB 00:01 2024-04-26T03:34:49.736 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-04-26T03:34:50.603 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - CRB 6.6 MB/s | 6.2 MB 00:00 2024-04-26T03:34:50.784 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - CRB 8.0 MB/s | 6.2 MB 00:00 2024-04-26T03:34:51.085 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-04-26T03:34:51.209 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - Extras packages 50 kB/s | 16 kB 00:00 2024-04-26T03:34:52.249 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - Extras packages 58 kB/s | 16 kB 00:00 2024-04-26T03:34:52.334 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 16 kB 00:00 2024-04-26T03:34:53.501 INFO:teuthology.orchestra.run.smithi042.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-04-26T03:34:53.899 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - Extras packages 10 kB/s | 16 kB 00:01 2024-04-26T03:34:54.689 INFO:teuthology.orchestra.run.smithi028.stdout:Extra Packages for Enterprise Linux 9.4 MB/s | 21 MB 00:02 2024-04-26T03:34:54.693 INFO:teuthology.orchestra.run.smithi092.stdout:Extra Packages for Enterprise Linux 6.2 MB/s | 21 MB 00:03 2024-04-26T03:34:56.009 INFO:teuthology.orchestra.run.smithi066.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-26T03:34:58.701 INFO:teuthology.orchestra.run.smithi042.stdout:lab-extras 41 kB/s | 1.7 kB 00:00 2024-04-26T03:34:59.843 INFO:teuthology.orchestra.run.smithi092.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-04-26T03:34:59.969 INFO:teuthology.orchestra.run.smithi028.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-04-26T03:35:00.938 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-04-26T03:35:00.940 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:35:00.940 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-04-26T03:35:00.940 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:35:00.940 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-04-26T03:35:00.941 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-radosgw x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 10 M 2024-04-26T03:35:00.941 INFO:teuthology.orchestra.run.smithi042.stdout:Upgrading: 2024-04-26T03:35:00.941 INFO:teuthology.orchestra.run.smithi042.stdout: librados2 x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 3.4 M 2024-04-26T03:35:00.941 INFO:teuthology.orchestra.run.smithi042.stdout: librbd1 x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 3.1 M 2024-04-26T03:35:00.941 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-04-26T03:35:00.941 INFO:teuthology.orchestra.run.smithi042.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T03:35:00.941 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-base x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 5.4 M 2024-04-26T03:35:00.941 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-common x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 21 M 2024-04-26T03:35:00.941 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-selinux x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 25 k 2024-04-26T03:35:00.941 INFO:teuthology.orchestra.run.smithi042.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T03:35:00.941 INFO:teuthology.orchestra.run.smithi042.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T03:35:00.941 INFO:teuthology.orchestra.run.smithi042.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T03:35:00.941 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs2 x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 746 k 2024-04-26T03:35:00.942 INFO:teuthology.orchestra.run.smithi042.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T03:35:00.942 INFO:teuthology.orchestra.run.smithi042.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T03:35:00.942 INFO:teuthology.orchestra.run.smithi042.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T03:35:00.942 INFO:teuthology.orchestra.run.smithi042.stdout: libradosstriper1 x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 500 k 2024-04-26T03:35:00.942 INFO:teuthology.orchestra.run.smithi042.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T03:35:00.942 INFO:teuthology.orchestra.run.smithi042.stdout: librgw2 x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 5.3 M 2024-04-26T03:35:00.942 INFO:teuthology.orchestra.run.smithi042.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T03:35:00.942 INFO:teuthology.orchestra.run.smithi042.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T03:35:00.942 INFO:teuthology.orchestra.run.smithi042.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T03:35:00.942 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-argparse x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 45 k 2024-04-26T03:35:00.942 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-common x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 133 k 2024-04-26T03:35:00.942 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cephfs x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 161 k 2024-04-26T03:35:00.942 INFO:teuthology.orchestra.run.smithi042.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T03:35:00.943 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rados x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 319 k 2024-04-26T03:35:00.943 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rbd x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 298 k 2024-04-26T03:35:00.943 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rgw x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 100 k 2024-04-26T03:35:00.943 INFO:teuthology.orchestra.run.smithi042.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T03:35:00.943 INFO:teuthology.orchestra.run.smithi042.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T03:35:00.943 INFO:teuthology.orchestra.run.smithi042.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T03:35:00.943 INFO:teuthology.orchestra.run.smithi042.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T03:35:00.943 INFO:teuthology.orchestra.run.smithi042.stdout:Installing weak dependencies: 2024-04-26T03:35:00.943 INFO:teuthology.orchestra.run.smithi042.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T03:35:00.943 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:35:00.943 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-04-26T03:35:00.943 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:35:00.944 INFO:teuthology.orchestra.run.smithi042.stdout:Install 30 Packages 2024-04-26T03:35:00.944 INFO:teuthology.orchestra.run.smithi042.stdout:Upgrade 2 Packages 2024-04-26T03:35:00.944 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:35:00.944 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 60 M 2024-04-26T03:35:00.944 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-04-26T03:35:01.251 INFO:teuthology.orchestra.run.smithi066.stdout:lab-extras 43 kB/s | 1.7 kB 00:00 2024-04-26T03:35:02.002 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-26T03:35:02.004 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:35:02.005 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-04-26T03:35:02.005 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:35:02.005 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-04-26T03:35:02.005 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 10 M 2024-04-26T03:35:02.005 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrading: 2024-04-26T03:35:02.005 INFO:teuthology.orchestra.run.smithi092.stdout: librados2 x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 3.4 M 2024-04-26T03:35:02.005 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1 x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 3.1 M 2024-04-26T03:35:02.005 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-04-26T03:35:02.005 INFO:teuthology.orchestra.run.smithi092.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T03:35:02.005 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 5.4 M 2024-04-26T03:35:02.005 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 21 M 2024-04-26T03:35:02.005 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 25 k 2024-04-26T03:35:02.006 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T03:35:02.006 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T03:35:02.006 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T03:35:02.006 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2 x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 746 k 2024-04-26T03:35:02.006 INFO:teuthology.orchestra.run.smithi092.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T03:35:02.006 INFO:teuthology.orchestra.run.smithi092.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T03:35:02.006 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T03:35:02.006 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1 x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 500 k 2024-04-26T03:35:02.006 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T03:35:02.006 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2 x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 5.3 M 2024-04-26T03:35:02.006 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T03:35:02.006 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T03:35:02.007 INFO:teuthology.orchestra.run.smithi092.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T03:35:02.007 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 45 k 2024-04-26T03:35:02.007 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 133 k 2024-04-26T03:35:02.007 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 161 k 2024-04-26T03:35:02.007 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T03:35:02.007 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 319 k 2024-04-26T03:35:02.007 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 298 k 2024-04-26T03:35:02.007 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 100 k 2024-04-26T03:35:02.007 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T03:35:02.007 INFO:teuthology.orchestra.run.smithi092.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T03:35:02.007 INFO:teuthology.orchestra.run.smithi092.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T03:35:02.007 INFO:teuthology.orchestra.run.smithi092.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T03:35:02.007 INFO:teuthology.orchestra.run.smithi092.stdout:Installing weak dependencies: 2024-04-26T03:35:02.008 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T03:35:02.008 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:35:02.008 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-04-26T03:35:02.008 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:35:02.008 INFO:teuthology.orchestra.run.smithi092.stdout:Install 30 Packages 2024-04-26T03:35:02.008 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrade 2 Packages 2024-04-26T03:35:02.008 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:35:02.008 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 60 M 2024-04-26T03:35:02.009 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-04-26T03:35:02.248 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-04-26T03:35:02.251 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:35:02.251 INFO:teuthology.orchestra.run.smithi028.stdout: Package Arch Version Repository Size 2024-04-26T03:35:02.251 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:35:02.251 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-04-26T03:35:02.251 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-radosgw x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 10 M 2024-04-26T03:35:02.251 INFO:teuthology.orchestra.run.smithi028.stdout:Upgrading: 2024-04-26T03:35:02.251 INFO:teuthology.orchestra.run.smithi028.stdout: librados2 x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 3.4 M 2024-04-26T03:35:02.251 INFO:teuthology.orchestra.run.smithi028.stdout: librbd1 x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 3.1 M 2024-04-26T03:35:02.251 INFO:teuthology.orchestra.run.smithi028.stdout:Installing dependencies: 2024-04-26T03:35:02.251 INFO:teuthology.orchestra.run.smithi028.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T03:35:02.251 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-base x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 5.4 M 2024-04-26T03:35:02.251 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-common x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 21 M 2024-04-26T03:35:02.252 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-selinux x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 25 k 2024-04-26T03:35:02.252 INFO:teuthology.orchestra.run.smithi028.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T03:35:02.252 INFO:teuthology.orchestra.run.smithi028.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T03:35:02.252 INFO:teuthology.orchestra.run.smithi028.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T03:35:02.252 INFO:teuthology.orchestra.run.smithi028.stdout: libcephfs2 x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 746 k 2024-04-26T03:35:02.252 INFO:teuthology.orchestra.run.smithi028.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T03:35:02.252 INFO:teuthology.orchestra.run.smithi028.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T03:35:02.252 INFO:teuthology.orchestra.run.smithi028.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T03:35:02.252 INFO:teuthology.orchestra.run.smithi028.stdout: libradosstriper1 x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 500 k 2024-04-26T03:35:02.252 INFO:teuthology.orchestra.run.smithi028.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T03:35:02.252 INFO:teuthology.orchestra.run.smithi028.stdout: librgw2 x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 5.3 M 2024-04-26T03:35:02.252 INFO:teuthology.orchestra.run.smithi028.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T03:35:02.252 INFO:teuthology.orchestra.run.smithi028.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T03:35:02.252 INFO:teuthology.orchestra.run.smithi028.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T03:35:02.252 INFO:teuthology.orchestra.run.smithi028.stdout: python3-ceph-argparse x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 45 k 2024-04-26T03:35:02.253 INFO:teuthology.orchestra.run.smithi028.stdout: python3-ceph-common x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 133 k 2024-04-26T03:35:02.253 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cephfs x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 161 k 2024-04-26T03:35:02.253 INFO:teuthology.orchestra.run.smithi028.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T03:35:02.253 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rados x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 319 k 2024-04-26T03:35:02.253 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rbd x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 298 k 2024-04-26T03:35:02.253 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rgw x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 100 k 2024-04-26T03:35:02.253 INFO:teuthology.orchestra.run.smithi028.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T03:35:02.253 INFO:teuthology.orchestra.run.smithi028.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T03:35:02.253 INFO:teuthology.orchestra.run.smithi028.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T03:35:02.253 INFO:teuthology.orchestra.run.smithi028.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T03:35:02.253 INFO:teuthology.orchestra.run.smithi028.stdout:Installing weak dependencies: 2024-04-26T03:35:02.253 INFO:teuthology.orchestra.run.smithi028.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T03:35:02.253 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:35:02.253 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-04-26T03:35:02.254 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:35:02.254 INFO:teuthology.orchestra.run.smithi028.stdout:Install 30 Packages 2024-04-26T03:35:02.254 INFO:teuthology.orchestra.run.smithi028.stdout:Upgrade 2 Packages 2024-04-26T03:35:02.254 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:35:02.254 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 60 M 2024-04-26T03:35:02.254 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-04-26T03:35:03.427 INFO:teuthology.orchestra.run.smithi042.stdout:(1/32): ceph-common-19.0.0-2457.gddeaa917.el9.x 10 MB/s | 21 MB 00:02 2024-04-26T03:35:03.469 INFO:teuthology.orchestra.run.smithi042.stdout:(2/32): ceph-selinux-19.0.0-2457.gddeaa917.el9. 612 kB/s | 25 kB 00:00 2024-04-26T03:35:03.478 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-04-26T03:35:03.481 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:35:03.481 INFO:teuthology.orchestra.run.smithi066.stdout: Package Arch Version Repository Size 2024-04-26T03:35:03.481 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:35:03.481 INFO:teuthology.orchestra.run.smithi066.stdout:Installing: 2024-04-26T03:35:03.481 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-radosgw x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 10 M 2024-04-26T03:35:03.481 INFO:teuthology.orchestra.run.smithi066.stdout:Upgrading: 2024-04-26T03:35:03.482 INFO:teuthology.orchestra.run.smithi066.stdout: librados2 x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 3.4 M 2024-04-26T03:35:03.482 INFO:teuthology.orchestra.run.smithi066.stdout: librbd1 x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 3.1 M 2024-04-26T03:35:03.482 INFO:teuthology.orchestra.run.smithi066.stdout:Installing dependencies: 2024-04-26T03:35:03.482 INFO:teuthology.orchestra.run.smithi066.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T03:35:03.482 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-base x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 5.4 M 2024-04-26T03:35:03.482 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-common x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 21 M 2024-04-26T03:35:03.482 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-selinux x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 25 k 2024-04-26T03:35:03.482 INFO:teuthology.orchestra.run.smithi066.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T03:35:03.482 INFO:teuthology.orchestra.run.smithi066.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T03:35:03.482 INFO:teuthology.orchestra.run.smithi066.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T03:35:03.482 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs2 x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 746 k 2024-04-26T03:35:03.483 INFO:teuthology.orchestra.run.smithi066.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T03:35:03.483 INFO:teuthology.orchestra.run.smithi066.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T03:35:03.483 INFO:teuthology.orchestra.run.smithi066.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T03:35:03.483 INFO:teuthology.orchestra.run.smithi066.stdout: libradosstriper1 x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 500 k 2024-04-26T03:35:03.483 INFO:teuthology.orchestra.run.smithi066.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T03:35:03.483 INFO:teuthology.orchestra.run.smithi066.stdout: librgw2 x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 5.3 M 2024-04-26T03:35:03.483 INFO:teuthology.orchestra.run.smithi066.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T03:35:03.483 INFO:teuthology.orchestra.run.smithi066.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T03:35:03.483 INFO:teuthology.orchestra.run.smithi066.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T03:35:03.483 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-argparse x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 45 k 2024-04-26T03:35:03.483 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-common x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 133 k 2024-04-26T03:35:03.483 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cephfs x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 161 k 2024-04-26T03:35:03.483 INFO:teuthology.orchestra.run.smithi066.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T03:35:03.484 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rados x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 319 k 2024-04-26T03:35:03.484 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rbd x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 298 k 2024-04-26T03:35:03.484 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rgw x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 100 k 2024-04-26T03:35:03.484 INFO:teuthology.orchestra.run.smithi066.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T03:35:03.484 INFO:teuthology.orchestra.run.smithi066.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T03:35:03.484 INFO:teuthology.orchestra.run.smithi066.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T03:35:03.484 INFO:teuthology.orchestra.run.smithi066.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T03:35:03.484 INFO:teuthology.orchestra.run.smithi066.stdout:Installing weak dependencies: 2024-04-26T03:35:03.484 INFO:teuthology.orchestra.run.smithi066.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T03:35:03.484 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:35:03.484 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction Summary 2024-04-26T03:35:03.484 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:35:03.484 INFO:teuthology.orchestra.run.smithi066.stdout:Install 30 Packages 2024-04-26T03:35:03.484 INFO:teuthology.orchestra.run.smithi066.stdout:Upgrade 2 Packages 2024-04-26T03:35:03.485 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:35:03.485 INFO:teuthology.orchestra.run.smithi066.stdout:Total download size: 60 M 2024-04-26T03:35:03.485 INFO:teuthology.orchestra.run.smithi066.stdout:Downloading Packages: 2024-04-26T03:35:03.571 INFO:teuthology.orchestra.run.smithi042.stdout:(3/32): libcephfs2-19.0.0-2457.gddeaa917.el9.x8 7.2 MB/s | 746 kB 00:00 2024-04-26T03:35:03.645 INFO:teuthology.orchestra.run.smithi042.stdout:(4/32): libradosstriper1-19.0.0-2457.gddeaa917. 6.7 MB/s | 500 kB 00:00 2024-04-26T03:35:04.253 INFO:teuthology.orchestra.run.smithi042.stdout:(5/32): librgw2-19.0.0-2457.gddeaa917.el9.x86_6 8.7 MB/s | 5.3 MB 00:00 2024-04-26T03:35:04.337 INFO:teuthology.orchestra.run.smithi042.stdout:(6/32): python3-ceph-argparse-19.0.0-2457.gddea 544 kB/s | 45 kB 00:00 2024-04-26T03:35:04.412 INFO:teuthology.orchestra.run.smithi042.stdout:(7/32): python3-ceph-common-19.0.0-2457.gddeaa9 1.7 MB/s | 133 kB 00:00 2024-04-26T03:35:04.479 INFO:teuthology.orchestra.run.smithi042.stdout:(8/32): ceph-base-19.0.0-2457.gddeaa917.el9.x86 1.7 MB/s | 5.4 MB 00:03 2024-04-26T03:35:04.496 INFO:teuthology.orchestra.run.smithi042.stdout:(9/32): python3-cephfs-19.0.0-2457.gddeaa917.el 1.9 MB/s | 161 kB 00:00 2024-04-26T03:35:04.597 INFO:teuthology.orchestra.run.smithi042.stdout:(10/32): python3-rbd-19.0.0-2457.gddeaa917.el9. 2.9 MB/s | 298 kB 00:00 2024-04-26T03:35:04.672 INFO:teuthology.orchestra.run.smithi042.stdout:(11/32): python3-rados-19.0.0-2457.gddeaa917.el 1.6 MB/s | 319 kB 00:00 2024-04-26T03:35:04.697 INFO:teuthology.orchestra.run.smithi042.stdout:(12/32): python3-rgw-19.0.0-2457.gddeaa917.el9. 995 kB/s | 100 kB 00:00 2024-04-26T03:35:04.831 INFO:teuthology.orchestra.run.smithi042.stdout:(13/32): boost-program-options-1.75.0-8.el9.x86 674 kB/s | 107 kB 00:00 2024-04-26T03:35:04.856 INFO:teuthology.orchestra.run.smithi042.stdout:(14/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2024-04-26T03:35:04.881 INFO:teuthology.orchestra.run.smithi042.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 905 kB/s | 45 kB 00:00 2024-04-26T03:35:04.990 INFO:teuthology.orchestra.run.smithi042.stdout:(16/32): librdkafka-1.6.1-102.el9.x86_64.rpm 4.8 MB/s | 662 kB 00:00 2024-04-26T03:35:05.015 INFO:teuthology.orchestra.run.smithi042.stdout:(17/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-04-26T03:35:05.040 INFO:teuthology.orchestra.run.smithi042.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 849 kB/s | 42 kB 00:00 2024-04-26T03:35:05.066 INFO:teuthology.orchestra.run.smithi042.stdout:(19/32): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-04-26T03:35:05.091 INFO:teuthology.orchestra.run.smithi042.stdout:(20/32): qatlib-service-24.02.0-1.el9.x86_64.rp 720 kB/s | 36 kB 00:00 2024-04-26T03:35:05.111 INFO:teuthology.orchestra.run.smithi092.stdout:(1/32): ceph-radosgw-19.0.0-2457.gddeaa917.el9. 3.9 MB/s | 10 MB 00:02 2024-04-26T03:35:05.116 INFO:teuthology.orchestra.run.smithi042.stdout:(21/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-04-26T03:35:05.202 INFO:teuthology.orchestra.run.smithi092.stdout:(2/32): ceph-selinux-19.0.0-2457.gddeaa917.el9. 275 kB/s | 25 kB 00:00 2024-04-26T03:35:05.304 INFO:teuthology.orchestra.run.smithi028.stdout:(1/32): ceph-radosgw-19.0.0-2457.gddeaa917.el9. 3.9 MB/s | 10 MB 00:02 2024-04-26T03:35:05.311 INFO:teuthology.orchestra.run.smithi092.stdout:(3/32): libcephfs2-19.0.0-2457.gddeaa917.el9.x8 6.7 MB/s | 746 kB 00:00 2024-04-26T03:35:05.345 INFO:teuthology.orchestra.run.smithi028.stdout:(2/32): ceph-selinux-19.0.0-2457.gddeaa917.el9. 610 kB/s | 25 kB 00:00 2024-04-26T03:35:05.378 INFO:teuthology.orchestra.run.smithi092.stdout:(4/32): libradosstriper1-19.0.0-2457.gddeaa917. 7.4 MB/s | 500 kB 00:00 2024-04-26T03:35:05.437 INFO:teuthology.orchestra.run.smithi028.stdout:(3/32): libcephfs2-19.0.0-2457.gddeaa917.el9.x8 7.9 MB/s | 746 kB 00:00 2024-04-26T03:35:05.467 INFO:teuthology.orchestra.run.smithi042.stdout:(22/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 824 kB/s | 309 kB 00:00 2024-04-26T03:35:05.529 INFO:teuthology.orchestra.run.smithi028.stdout:(4/32): libradosstriper1-19.0.0-2457.gddeaa917. 5.3 MB/s | 500 kB 00:00 2024-04-26T03:35:05.534 INFO:teuthology.orchestra.run.smithi042.stdout:(23/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 391 kB/s | 26 kB 00:00 2024-04-26T03:35:05.595 INFO:teuthology.orchestra.run.smithi092.stdout:(5/32): ceph-base-19.0.0-2457.gddeaa917.el9.x86 1.7 MB/s | 5.4 MB 00:03 2024-04-26T03:35:05.659 INFO:teuthology.orchestra.run.smithi042.stdout:(24/32): ceph-radosgw-19.0.0-2457.gddeaa917.el9 2.4 MB/s | 10 MB 00:04 2024-04-26T03:35:05.684 INFO:teuthology.orchestra.run.smithi042.stdout:(25/32): liboath-2.6.7-2.el9.x86_64.rpm 323 kB/s | 49 kB 00:00 2024-04-26T03:35:05.720 INFO:teuthology.orchestra.run.smithi092.stdout:(6/32): python3-ceph-argparse-19.0.0-2457.gddea 362 kB/s | 45 kB 00:00 2024-04-26T03:35:05.776 INFO:teuthology.orchestra.run.smithi042.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 577 kB/s | 67 kB 00:00 2024-04-26T03:35:05.854 INFO:teuthology.orchestra.run.smithi092.stdout:(7/32): python3-ceph-common-19.0.0-2457.gddeaa9 997 kB/s | 133 kB 00:00 2024-04-26T03:35:05.910 INFO:teuthology.orchestra.run.smithi042.stdout:(27/32): re2-20211101-3.el9.x86_64.rpm 1.4 MB/s | 192 kB 00:00 2024-04-26T03:35:05.987 INFO:teuthology.orchestra.run.smithi092.stdout:(8/32): python3-cephfs-19.0.0-2457.gddeaa917.el 1.2 MB/s | 161 kB 00:00 2024-04-26T03:35:06.071 INFO:teuthology.orchestra.run.smithi092.stdout:(9/32): librgw2-19.0.0-2457.gddeaa917.el9.x86_6 7.7 MB/s | 5.3 MB 00:00 2024-04-26T03:35:06.088 INFO:teuthology.orchestra.run.smithi028.stdout:(5/32): librgw2-19.0.0-2457.gddeaa917.el9.x86_6 9.5 MB/s | 5.3 MB 00:00 2024-04-26T03:35:06.121 INFO:teuthology.orchestra.run.smithi092.stdout:(10/32): python3-rbd-19.0.0-2457.gddeaa917.el9. 5.8 MB/s | 298 kB 00:00 2024-04-26T03:35:06.138 INFO:teuthology.orchestra.run.smithi028.stdout:(6/32): python3-ceph-argparse-19.0.0-2457.gddea 910 kB/s | 45 kB 00:00 2024-04-26T03:35:06.163 INFO:teuthology.orchestra.run.smithi092.stdout:(11/32): python3-rgw-19.0.0-2457.gddeaa917.el9. 2.3 MB/s | 100 kB 00:00 2024-04-26T03:35:06.177 INFO:teuthology.orchestra.run.smithi066.stdout:(1/32): ceph-base-19.0.0-2457.gddeaa917.el9.x86 2.4 MB/s | 5.4 MB 00:02 2024-04-26T03:35:06.188 INFO:teuthology.orchestra.run.smithi092.stdout:(12/32): python3-rados-19.0.0-2457.gddeaa917.el 1.6 MB/s | 319 kB 00:00 2024-04-26T03:35:06.189 INFO:teuthology.orchestra.run.smithi028.stdout:(7/32): python3-ceph-common-19.0.0-2457.gddeaa9 2.6 MB/s | 133 kB 00:00 2024-04-26T03:35:06.219 INFO:teuthology.orchestra.run.smithi066.stdout:(2/32): ceph-selinux-19.0.0-2457.gddeaa917.el9. 609 kB/s | 25 kB 00:00 2024-04-26T03:35:06.235 INFO:teuthology.orchestra.run.smithi042.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 1.5 MB/s | 839 kB 00:00 2024-04-26T03:35:06.255 INFO:teuthology.orchestra.run.smithi028.stdout:(8/32): ceph-base-19.0.0-2457.gddeaa917.el9.x86 1.5 MB/s | 5.4 MB 00:03 2024-04-26T03:35:06.281 INFO:teuthology.orchestra.run.smithi028.stdout:(9/32): python3-cephfs-19.0.0-2457.gddeaa917.el 1.7 MB/s | 161 kB 00:00 2024-04-26T03:35:06.319 INFO:teuthology.orchestra.run.smithi066.stdout:(3/32): libcephfs2-19.0.0-2457.gddeaa917.el9.x8 7.3 MB/s | 746 kB 00:00 2024-04-26T03:35:06.331 INFO:teuthology.orchestra.run.smithi028.stdout:(10/32): python3-rbd-19.0.0-2457.gddeaa917.el9. 5.9 MB/s | 298 kB 00:00 2024-04-26T03:35:06.344 INFO:teuthology.orchestra.run.smithi042.stdout:(29/32): libarrow-9.0.0-11.el9.x86_64.rpm 3.6 MB/s | 4.4 MB 00:01 2024-04-26T03:35:06.381 INFO:teuthology.orchestra.run.smithi028.stdout:(11/32): python3-rgw-19.0.0-2457.gddeaa917.el9. 2.0 MB/s | 100 kB 00:00 2024-04-26T03:35:06.397 INFO:teuthology.orchestra.run.smithi092.stdout:(13/32): boost-program-options-1.75.0-8.el9.x86 458 kB/s | 107 kB 00:00 2024-04-26T03:35:06.403 INFO:teuthology.orchestra.run.smithi066.stdout:(4/32): libradosstriper1-19.0.0-2457.gddeaa917. 5.9 MB/s | 500 kB 00:00 2024-04-26T03:35:06.439 INFO:teuthology.orchestra.run.smithi092.stdout:(14/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 718 kB/s | 160 kB 00:00 2024-04-26T03:35:06.464 INFO:teuthology.orchestra.run.smithi092.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 680 kB/s | 45 kB 00:00 2024-04-26T03:35:06.465 INFO:teuthology.orchestra.run.smithi028.stdout:(12/32): python3-rados-19.0.0-2457.gddeaa917.el 1.5 MB/s | 319 kB 00:00 2024-04-26T03:35:06.486 INFO:teuthology.orchestra.run.smithi042.stdout:(30/32): thrift-0.15.0-2.el9.x86_64.rpm 2.7 MB/s | 1.6 MB 00:00 2024-04-26T03:35:06.694 INFO:teuthology.orchestra.run.smithi042.stdout:(31/32): librbd1-19.0.0-2457.gddeaa917.el9.x86_ 9.0 MB/s | 3.1 MB 00:00 2024-04-26T03:35:06.814 INFO:teuthology.orchestra.run.smithi092.stdout:(16/32): librdkafka-1.6.1-102.el9.x86_64.rpm 1.7 MB/s | 662 kB 00:00 2024-04-26T03:35:06.820 INFO:teuthology.orchestra.run.smithi066.stdout:(5/32): librgw2-19.0.0-2457.gddeaa917.el9.x86_6 13 MB/s | 5.3 MB 00:00 2024-04-26T03:35:06.839 INFO:teuthology.orchestra.run.smithi092.stdout:(17/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 779 kB/s | 292 kB 00:00 2024-04-26T03:35:06.862 INFO:teuthology.orchestra.run.smithi066.stdout:(6/32): python3-ceph-argparse-19.0.0-2457.gddea 1.1 MB/s | 45 kB 00:00 2024-04-26T03:35:06.873 INFO:teuthology.orchestra.run.smithi092.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 726 kB/s | 42 kB 00:00 2024-04-26T03:35:06.912 INFO:teuthology.orchestra.run.smithi066.stdout:(7/32): python3-ceph-common-19.0.0-2457.gddeaa9 2.6 MB/s | 133 kB 00:00 2024-04-26T03:35:06.915 INFO:teuthology.orchestra.run.smithi028.stdout:(13/32): boost-program-options-1.75.0-8.el9.x86 200 kB/s | 107 kB 00:00 2024-04-26T03:35:06.962 INFO:teuthology.orchestra.run.smithi066.stdout:(8/32): python3-cephfs-19.0.0-2457.gddeaa917.el 3.2 MB/s | 161 kB 00:00 2024-04-26T03:35:06.999 INFO:teuthology.orchestra.run.smithi028.stdout:(14/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 542 kB/s | 45 kB 00:00 2024-04-26T03:35:07.006 INFO:teuthology.orchestra.run.smithi092.stdout:(19/32): qatlib-24.02.0-1.el9.x86_64.rpm 1.3 MB/s | 221 kB 00:00 2024-04-26T03:35:07.012 INFO:teuthology.orchestra.run.smithi066.stdout:(9/32): python3-rados-19.0.0-2457.gddeaa917.el9 6.3 MB/s | 319 kB 00:00 2024-04-26T03:35:07.032 INFO:teuthology.orchestra.run.smithi092.stdout:(20/32): qatlib-service-24.02.0-1.el9.x86_64.rp 227 kB/s | 36 kB 00:00 2024-04-26T03:35:07.057 INFO:teuthology.orchestra.run.smithi092.stdout:(21/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-04-26T03:35:07.062 INFO:teuthology.orchestra.run.smithi066.stdout:(10/32): python3-rbd-19.0.0-2457.gddeaa917.el9. 5.8 MB/s | 298 kB 00:00 2024-04-26T03:35:07.105 INFO:teuthology.orchestra.run.smithi066.stdout:(11/32): python3-rgw-19.0.0-2457.gddeaa917.el9. 2.3 MB/s | 100 kB 00:00 2024-04-26T03:35:07.149 INFO:teuthology.orchestra.run.smithi028.stdout:(15/32): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-04-26T03:35:07.199 INFO:teuthology.orchestra.run.smithi028.stdout:(16/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-26T03:35:07.258 INFO:teuthology.orchestra.run.smithi028.stdout:(17/32): python3-prettytable-0.7.2-27.el9.noarc 720 kB/s | 42 kB 00:00 2024-04-26T03:35:07.288 INFO:teuthology.orchestra.run.smithi066.stdout:(12/32): boost-program-options-1.75.0-8.el9.x86 584 kB/s | 107 kB 00:00 2024-04-26T03:35:07.447 INFO:teuthology.orchestra.run.smithi066.stdout:(13/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2024-04-26T03:35:07.450 INFO:teuthology.orchestra.run.smithi028.stdout:(18/32): qatlib-24.02.0-1.el9.x86_64.rpm 1.1 MB/s | 221 kB 00:00 2024-04-26T03:35:07.500 INFO:teuthology.orchestra.run.smithi092.stdout:[MIRROR] libarrow-9.0.0-11.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/fedora-epel/9/Everything/x86_64/Packages/l/libarrow-9.0.0-11.el9.x86_64.rpm (IP: 140.211.166.134) 2024-04-26T03:35:07.500 INFO:teuthology.orchestra.run.smithi028.stdout:(19/32): qatlib-service-24.02.0-1.el9.x86_64.rp 718 kB/s | 36 kB 00:00 2024-04-26T03:35:07.505 INFO:teuthology.orchestra.run.smithi066.stdout:(14/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 779 kB/s | 45 kB 00:00 2024-04-26T03:35:07.588 INFO:teuthology.orchestra.run.smithi028.stdout:(20/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 539 kB/s | 47 kB 00:00 2024-04-26T03:35:07.697 INFO:teuthology.orchestra.run.smithi066.stdout:(15/32): librdkafka-1.6.1-102.el9.x86_64.rpm 3.4 MB/s | 662 kB 00:00 2024-04-26T03:35:07.720 INFO:teuthology.orchestra.run.smithi092.stdout:[MIRROR] libarrow-9.0.0-11.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/fedora-epel/9/Everything/x86_64/Packages/l/libarrow-9.0.0-11.el9.x86_64.rpm (IP: 140.211.166.134) 2024-04-26T03:35:07.781 INFO:teuthology.orchestra.run.smithi066.stdout:(16/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-04-26T03:35:07.801 INFO:teuthology.orchestra.run.smithi028.stdout:(21/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 120 kB/s | 160 kB 00:01 2024-04-26T03:35:07.848 INFO:teuthology.orchestra.run.smithi066.stdout:(17/32): python3-prettytable-0.7.2-27.el9.noarc 635 kB/s | 42 kB 00:00 2024-04-26T03:35:07.931 INFO:teuthology.orchestra.run.smithi066.stdout:(18/32): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-04-26T03:35:08.007 INFO:teuthology.orchestra.run.smithi066.stdout:(19/32): qatlib-service-24.02.0-1.el9.x86_64.rp 479 kB/s | 36 kB 00:00 2024-04-26T03:35:08.074 INFO:teuthology.orchestra.run.smithi066.stdout:(20/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 712 kB/s | 47 kB 00:00 2024-04-26T03:35:08.201 INFO:teuthology.orchestra.run.smithi028.stdout:(22/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 506 kB/s | 309 kB 00:00 2024-04-26T03:35:08.301 INFO:teuthology.orchestra.run.smithi028.stdout:(23/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 259 kB/s | 26 kB 00:00 2024-04-26T03:35:08.401 INFO:teuthology.orchestra.run.smithi028.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 487 kB/s | 49 kB 00:00 2024-04-26T03:35:08.415 INFO:teuthology.orchestra.run.smithi092.stdout:(22/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 223 kB/s | 309 kB 00:01 2024-04-26T03:35:08.504 INFO:teuthology.orchestra.run.smithi092.stdout:[MIRROR] libarrow-doc-9.0.0-11.el9.noarch.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/fedora-epel/9/Everything/x86_64/Packages/l/libarrow-doc-9.0.0-11.el9.noarch.rpm (IP: 140.211.166.134) 2024-04-26T03:35:08.511 INFO:teuthology.orchestra.run.smithi028.stdout:(25/32): libunwind-1.6.2-1.el9.x86_64.rpm 619 kB/s | 67 kB 00:00 2024-04-26T03:35:08.586 INFO:teuthology.orchestra.run.smithi092.stdout:[MIRROR] libarrow-doc-9.0.0-11.el9.noarch.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/fedora-epel/9/Everything/x86_64/Packages/l/libarrow-doc-9.0.0-11.el9.noarch.rpm (IP: 140.211.166.134) 2024-04-26T03:35:08.682 INFO:teuthology.orchestra.run.smithi066.stdout:(21/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 508 kB/s | 309 kB 00:00 2024-04-26T03:35:08.736 INFO:teuthology.orchestra.run.smithi028.stdout:(26/32): libarrow-9.0.0-11.el9.x86_64.rpm 4.7 MB/s | 4.4 MB 00:00 2024-04-26T03:35:08.799 INFO:teuthology.orchestra.run.smithi092.stdout:(23/32): libarrow-9.0.0-11.el9.x86_64.rpm 2.5 MB/s | 4.4 MB 00:01 2024-04-26T03:35:08.811 INFO:teuthology.orchestra.run.smithi028.stdout:(27/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.7 MB/s | 839 kB 00:00 2024-04-26T03:35:08.833 INFO:teuthology.orchestra.run.smithi092.stdout:(24/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 62 kB/s | 26 kB 00:00 2024-04-26T03:35:08.836 INFO:teuthology.orchestra.run.smithi028.stdout:(28/32): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-04-26T03:35:09.028 INFO:teuthology.orchestra.run.smithi028.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 7.3 MB/s | 1.6 MB 00:00 2024-04-26T03:35:09.124 INFO:teuthology.orchestra.run.smithi092.stdout:(25/32): liboath-2.6.7-2.el9.x86_64.rpm 149 kB/s | 49 kB 00:00 2024-04-26T03:35:09.199 INFO:teuthology.orchestra.run.smithi066.stdout:(22/32): libarrow-9.0.0-11.el9.x86_64.rpm 8.5 MB/s | 4.4 MB 00:00 2024-04-26T03:35:09.242 INFO:teuthology.orchestra.run.smithi092.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 165 kB/s | 67 kB 00:00 2024-04-26T03:35:09.242 INFO:teuthology.orchestra.run.smithi092.stdout:[MIRROR] parquet-libs-9.0.0-11.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/fedora-epel/9/Everything/x86_64/Packages/p/parquet-libs-9.0.0-11.el9.x86_64.rpm (IP: 140.211.166.134) 2024-04-26T03:35:09.300 INFO:teuthology.orchestra.run.smithi066.stdout:(23/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 260 kB/s | 26 kB 00:00 2024-04-26T03:35:09.320 INFO:teuthology.orchestra.run.smithi028.stdout:(30/32): librbd1-19.0.0-2457.gddeaa917.el9.x86_ 11 MB/s | 3.1 MB 00:00 2024-04-26T03:35:09.328 INFO:teuthology.orchestra.run.smithi092.stdout:[MIRROR] parquet-libs-9.0.0-11.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/fedora-epel/9/Everything/x86_64/Packages/p/parquet-libs-9.0.0-11.el9.x86_64.rpm (IP: 140.211.166.134) 2024-04-26T03:35:09.400 INFO:teuthology.orchestra.run.smithi066.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 486 kB/s | 49 kB 00:00 2024-04-26T03:35:09.500 INFO:teuthology.orchestra.run.smithi066.stdout:(25/32): libunwind-1.6.2-1.el9.x86_64.rpm 675 kB/s | 67 kB 00:00 2024-04-26T03:35:09.567 INFO:teuthology.orchestra.run.smithi092.stdout:(27/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 1.9 MB/s | 839 kB 00:00 2024-04-26T03:35:09.592 INFO:teuthology.orchestra.run.smithi092.stdout:(28/32): re2-20211101-3.el9.x86_64.rpm 547 kB/s | 192 kB 00:00 2024-04-26T03:35:09.609 INFO:teuthology.orchestra.run.smithi066.stdout:(26/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.6 MB/s | 839 kB 00:00 2024-04-26T03:35:09.709 INFO:teuthology.orchestra.run.smithi066.stdout:(27/32): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-04-26T03:35:09.751 INFO:teuthology.orchestra.run.smithi092.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 8.6 MB/s | 1.6 MB 00:00 2024-04-26T03:35:09.843 INFO:teuthology.orchestra.run.smithi066.stdout:(28/32): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-04-26T03:35:09.903 INFO:teuthology.orchestra.run.smithi042.stdout:(32/32): librados2-19.0.0-2457.gddeaa917.el9.x8 946 kB/s | 3.4 MB 00:03 2024-04-26T03:35:09.908 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:35:09.908 INFO:teuthology.orchestra.run.smithi042.stdout:Total 6.7 MB/s | 60 MB 00:08 2024-04-26T03:35:10.012 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-04-26T03:35:10.053 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-04-26T03:35:10.053 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-04-26T03:35:10.110 INFO:teuthology.orchestra.run.smithi092.stdout:(30/32): librbd1-19.0.0-2457.gddeaa917.el9.x86_ 8.7 MB/s | 3.1 MB 00:00 2024-04-26T03:35:10.145 INFO:teuthology.orchestra.run.smithi028.stdout:(31/32): librados2-19.0.0-2457.gddeaa917.el9.x8 2.6 MB/s | 3.4 MB 00:01 2024-04-26T03:35:10.485 INFO:teuthology.orchestra.run.smithi066.stdout:(29/32): librados2-19.0.0-2457.gddeaa917.el9.x8 5.3 MB/s | 3.4 MB 00:00 2024-04-26T03:35:10.510 INFO:teuthology.orchestra.run.smithi092.stdout:(31/32): librados2-19.0.0-2457.gddeaa917.el9.x8 3.7 MB/s | 3.4 MB 00:00 2024-04-26T03:35:10.542 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-04-26T03:35:10.542 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-04-26T03:35:10.744 INFO:teuthology.orchestra.run.smithi066.stdout:(30/32): librbd1-19.0.0-2457.gddeaa917.el9.x86_ 12 MB/s | 3.1 MB 00:00 2024-04-26T03:35:12.043 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-04-26T03:35:12.130 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-04-26T03:35:12.166 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-04-26T03:35:12.192 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-04-26T03:35:12.215 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-04-26T03:35:12.239 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-04-26T03:35:12.457 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-04-26T03:35:12.512 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-04-26T03:35:12.870 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-04-26T03:35:12.960 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : librados2-2:19.0.0-2457.gddeaa917.el9.x86_64 7/34 2024-04-26T03:35:13.053 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librados2-2:19.0.0-2457.gddeaa917.el9.x86_64 7/34 2024-04-26T03:35:13.146 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rados-2:19.0.0-2457.gddeaa917.el9.x86_64 8/34 2024-04-26T03:35:13.181 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libcephfs2-2:19.0.0-2457.gddeaa917.el9.x86_64 9/34 2024-04-26T03:35:13.244 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libcephfs2-2:19.0.0-2457.gddeaa917.el9.x86_64 9/34 2024-04-26T03:35:13.273 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-04-26T03:35:13.301 INFO:teuthology.orchestra.run.smithi092.stdout:(32/32): ceph-common-19.0.0-2457.gddeaa917.el9. 1.9 MB/s | 21 MB 00:10 2024-04-26T03:35:13.307 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:35:13.307 INFO:teuthology.orchestra.run.smithi092.stdout:Total 5.3 MB/s | 60 MB 00:11 2024-04-26T03:35:13.317 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-ceph-argparse-2:19.0.0-2457.gddeaa917.el9. 11/34 2024-04-26T03:35:13.396 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cephfs-2:19.0.0-2457.gddeaa917.el9.x86_64 12/34 2024-04-26T03:35:13.409 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-04-26T03:35:13.425 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libradosstriper1-2:19.0.0-2457.gddeaa917.el9.x86_6 13/34 2024-04-26T03:35:13.445 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-04-26T03:35:13.445 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-04-26T03:35:13.481 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2457.gddeaa917.el9.x86_6 13/34 2024-04-26T03:35:13.515 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-04-26T03:35:13.544 INFO:teuthology.orchestra.run.smithi066.stdout:(31/32): ceph-radosgw-19.0.0-2457.gddeaa917.el9 1.1 MB/s | 10 MB 00:09 2024-04-26T03:35:13.626 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-04-26T03:35:13.657 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-04-26T03:35:13.706 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-04-26T03:35:13.739 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-04-26T03:35:13.769 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-04-26T03:35:13.802 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-04-26T03:35:13.909 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-04-26T03:35:13.909 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-04-26T03:35:14.134 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-04-26T03:35:14.173 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : librbd1-2:19.0.0-2457.gddeaa917.el9.x86_64 22/34 2024-04-26T03:35:14.264 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librbd1-2:19.0.0-2457.gddeaa917.el9.x86_64 22/34 2024-04-26T03:35:14.286 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rbd-2:19.0.0-2457.gddeaa917.el9.x86_64 23/34 2024-04-26T03:35:14.322 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-04-26T03:35:14.404 INFO:teuthology.orchestra.run.smithi028.stdout:(32/32): ceph-common-19.0.0-2457.gddeaa917.el9. 1.8 MB/s | 21 MB 00:11 2024-04-26T03:35:14.410 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:35:14.410 INFO:teuthology.orchestra.run.smithi028.stdout:Total 4.9 MB/s | 60 MB 00:12 2024-04-26T03:35:14.517 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-04-26T03:35:14.559 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-04-26T03:35:14.559 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-04-26T03:35:14.824 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-04-26T03:35:14.859 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librgw2-2:19.0.0-2457.gddeaa917.el9.x86_64 26/34 2024-04-26T03:35:14.931 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librgw2-2:19.0.0-2457.gddeaa917.el9.x86_64 26/34 2024-04-26T03:35:15.011 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rgw-2:19.0.0-2457.gddeaa917.el9.x86_64 27/34 2024-04-26T03:35:15.053 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-04-26T03:35:15.053 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-04-26T03:35:15.069 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-ceph-common-2:19.0.0-2457.gddeaa917.el9.x8 28/34 2024-04-26T03:35:15.396 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-04-26T03:35:15.476 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-04-26T03:35:15.531 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-04-26T03:35:15.556 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-04-26T03:35:15.579 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-04-26T03:35:15.602 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-04-26T03:35:15.821 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-04-26T03:35:15.876 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-04-26T03:35:16.280 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-04-26T03:35:16.333 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librados2-2:19.0.0-2457.gddeaa917.el9.x86_64 7/34 2024-04-26T03:35:16.418 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-2:19.0.0-2457.gddeaa917.el9.x86_64 7/34 2024-04-26T03:35:16.515 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rados-2:19.0.0-2457.gddeaa917.el9.x86_64 8/34 2024-04-26T03:35:16.546 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs2-2:19.0.0-2457.gddeaa917.el9.x86_64 9/34 2024-04-26T03:35:16.574 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-04-26T03:35:16.607 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephfs2-2:19.0.0-2457.gddeaa917.el9.x86_64 9/34 2024-04-26T03:35:16.636 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-04-26T03:35:16.664 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-04-26T03:35:16.679 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-argparse-2:19.0.0-2457.gddeaa917.el9. 11/34 2024-04-26T03:35:16.692 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-04-26T03:35:16.717 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-04-26T03:35:16.740 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-04-26T03:35:16.757 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cephfs-2:19.0.0-2457.gddeaa917.el9.x86_64 12/34 2024-04-26T03:35:16.768 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-04-26T03:35:16.780 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libradosstriper1-2:19.0.0-2457.gddeaa917.el9.x86_6 13/34 2024-04-26T03:35:16.844 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2457.gddeaa917.el9.x86_6 13/34 2024-04-26T03:35:16.878 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-04-26T03:35:16.975 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-04-26T03:35:16.987 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-04-26T03:35:17.022 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-04-26T03:35:17.030 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-04-26T03:35:17.060 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-04-26T03:35:17.086 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-04-26T03:35:17.107 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-04-26T03:35:17.131 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-04-26T03:35:17.383 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-04-26T03:35:17.435 INFO:teuthology.orchestra.run.smithi066.stdout:(32/32): ceph-common-19.0.0-2457.gddeaa917.el9. 1.6 MB/s | 21 MB 00:13 2024-04-26T03:35:17.439 INFO:teuthology.orchestra.run.smithi066.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:35:17.439 INFO:teuthology.orchestra.run.smithi066.stdout:Total 4.3 MB/s | 60 MB 00:13 2024-04-26T03:35:17.443 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-04-26T03:35:17.486 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librbd1-2:19.0.0-2457.gddeaa917.el9.x86_64 22/34 2024-04-26T03:35:17.496 INFO:teuthology.orchestra.run.smithi028.stdout: Upgrading : librados2-2:19.0.0-2457.gddeaa917.el9.x86_64 7/34 2024-04-26T03:35:17.541 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction check 2024-04-26T03:35:17.571 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-common-2:19.0.0-2457.gddeaa917.el9.x86_64 29/34 2024-04-26T03:35:17.575 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-2:19.0.0-2457.gddeaa917.el9.x86_64 22/34 2024-04-26T03:35:17.582 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction check succeeded. 2024-04-26T03:35:17.583 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction test 2024-04-26T03:35:17.595 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: librados2-2:19.0.0-2457.gddeaa917.el9.x86_64 7/34 2024-04-26T03:35:17.607 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rbd-2:19.0.0-2457.gddeaa917.el9.x86_64 23/34 2024-04-26T03:35:17.608 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-common-2:19.0.0-2457.gddeaa917.el9.x86_64 29/34 2024-04-26T03:35:17.643 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-04-26T03:35:17.685 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-rados-2:19.0.0-2457.gddeaa917.el9.x86_64 8/34 2024-04-26T03:35:17.712 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libcephfs2-2:19.0.0-2457.gddeaa917.el9.x86_64 9/34 2024-04-26T03:35:17.775 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: libcephfs2-2:19.0.0-2457.gddeaa917.el9.x86_64 9/34 2024-04-26T03:35:17.804 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-04-26T03:35:17.849 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-ceph-argparse-2:19.0.0-2457.gddeaa917.el9. 11/34 2024-04-26T03:35:17.918 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-cephfs-2:19.0.0-2457.gddeaa917.el9.x86_64 12/34 2024-04-26T03:35:17.939 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libradosstriper1-2:19.0.0-2457.gddeaa917.el9.x86_6 13/34 2024-04-26T03:35:17.987 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2457.gddeaa917.el9.x86_6 13/34 2024-04-26T03:35:18.021 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-04-26T03:35:18.071 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction test succeeded. 2024-04-26T03:35:18.071 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction 2024-04-26T03:35:18.113 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-04-26T03:35:18.135 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-04-26T03:35:18.138 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librgw2-2:19.0.0-2457.gddeaa917.el9.x86_64 26/34 2024-04-26T03:35:18.164 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-04-26T03:35:18.186 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-common-2:19.0.0-2457.gddeaa917.el9.x86_64 29/34 2024-04-26T03:35:18.200 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librgw2-2:19.0.0-2457.gddeaa917.el9.x86_64 26/34 2024-04-26T03:35:18.204 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-04-26T03:35:18.228 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-base-2:19.0.0-2457.gddeaa917.el9.x86_64 30/34 2024-04-26T03:35:18.237 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-04-26T03:35:18.267 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-04-26T03:35:18.277 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rgw-2:19.0.0-2457.gddeaa917.el9.x86_64 27/34 2024-04-26T03:35:18.291 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-04-26T03:35:18.316 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-common-2:19.0.0-2457.gddeaa917.el9.x8 28/34 2024-04-26T03:35:18.320 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-base-2:19.0.0-2457.gddeaa917.el9.x86_64 30/34 2024-04-26T03:35:18.320 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T03:35:18.320 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T03:35:18.320 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:35:18.346 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-selinux-2:19.0.0-2457.gddeaa917.el9.x86_64 31/34 2024-04-26T03:35:18.631 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-04-26T03:35:18.670 INFO:teuthology.orchestra.run.smithi028.stdout: Upgrading : librbd1-2:19.0.0-2457.gddeaa917.el9.x86_64 22/34 2024-04-26T03:35:18.761 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: librbd1-2:19.0.0-2457.gddeaa917.el9.x86_64 22/34 2024-04-26T03:35:18.791 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-rbd-2:19.0.0-2457.gddeaa917.el9.x86_64 23/34 2024-04-26T03:35:18.828 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-04-26T03:35:19.326 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-04-26T03:35:19.356 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : librgw2-2:19.0.0-2457.gddeaa917.el9.x86_64 26/34 2024-04-26T03:35:19.411 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: librgw2-2:19.0.0-2457.gddeaa917.el9.x86_64 26/34 2024-04-26T03:35:19.491 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-rgw-2:19.0.0-2457.gddeaa917.el9.x86_64 27/34 2024-04-26T03:35:19.534 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-ceph-common-2:19.0.0-2457.gddeaa917.el9.x8 28/34 2024-04-26T03:35:19.576 INFO:teuthology.orchestra.run.smithi066.stdout: Preparing : 1/1 2024-04-26T03:35:19.662 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-04-26T03:35:19.783 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-04-26T03:35:19.824 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-04-26T03:35:19.847 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-04-26T03:35:19.872 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-04-26T03:35:20.090 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-04-26T03:35:20.152 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-04-26T03:35:20.510 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-04-26T03:35:20.535 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:19.0.0-2457.gddeaa917.el9.x86_64 29/34 2024-04-26T03:35:20.609 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : librados2-2:19.0.0-2457.gddeaa917.el9.x86_64 7/34 2024-04-26T03:35:20.654 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-common-2:19.0.0-2457.gddeaa917.el9.x86_64 29/34 2024-04-26T03:35:20.701 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librados2-2:19.0.0-2457.gddeaa917.el9.x86_64 7/34 2024-04-26T03:35:20.792 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rados-2:19.0.0-2457.gddeaa917.el9.x86_64 8/34 2024-04-26T03:35:20.818 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libcephfs2-2:19.0.0-2457.gddeaa917.el9.x86_64 9/34 2024-04-26T03:35:20.873 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libcephfs2-2:19.0.0-2457.gddeaa917.el9.x86_64 9/34 2024-04-26T03:35:20.902 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-04-26T03:35:20.945 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-ceph-argparse-2:19.0.0-2457.gddeaa917.el9. 11/34 2024-04-26T03:35:21.024 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cephfs-2:19.0.0-2457.gddeaa917.el9.x86_64 12/34 2024-04-26T03:35:21.045 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libradosstriper1-2:19.0.0-2457.gddeaa917.el9.x86_6 13/34 2024-04-26T03:35:21.110 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2457.gddeaa917.el9.x86_6 13/34 2024-04-26T03:35:21.144 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-04-26T03:35:21.197 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:19.0.0-2457.gddeaa917.el9.x86_64 29/34 2024-04-26T03:35:21.240 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-base-2:19.0.0-2457.gddeaa917.el9.x86_64 30/34 2024-04-26T03:35:21.256 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-04-26T03:35:21.286 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-04-26T03:35:21.327 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-04-26T03:35:21.329 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-base-2:19.0.0-2457.gddeaa917.el9.x86_64 30/34 2024-04-26T03:35:21.329 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T03:35:21.329 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T03:35:21.329 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:35:21.358 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-selinux-2:19.0.0-2457.gddeaa917.el9.x86_64 31/34 2024-04-26T03:35:21.360 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-04-26T03:35:21.389 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-04-26T03:35:21.422 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-04-26T03:35:21.750 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-04-26T03:35:21.793 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : librbd1-2:19.0.0-2457.gddeaa917.el9.x86_64 22/34 2024-04-26T03:35:21.883 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librbd1-2:19.0.0-2457.gddeaa917.el9.x86_64 22/34 2024-04-26T03:35:21.914 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rbd-2:19.0.0-2457.gddeaa917.el9.x86_64 23/34 2024-04-26T03:35:21.927 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-common-2:19.0.0-2457.gddeaa917.el9.x86_64 29/34 2024-04-26T03:35:21.950 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-04-26T03:35:22.205 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-common-2:19.0.0-2457.gddeaa917.el9.x86_64 29/34 2024-04-26T03:35:22.447 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-04-26T03:35:22.470 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librgw2-2:19.0.0-2457.gddeaa917.el9.x86_64 26/34 2024-04-26T03:35:22.541 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librgw2-2:19.0.0-2457.gddeaa917.el9.x86_64 26/34 2024-04-26T03:35:22.622 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rgw-2:19.0.0-2457.gddeaa917.el9.x86_64 27/34 2024-04-26T03:35:22.656 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-ceph-common-2:19.0.0-2457.gddeaa917.el9.x8 28/34 2024-04-26T03:35:22.778 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-common-2:19.0.0-2457.gddeaa917.el9.x86_64 29/34 2024-04-26T03:35:22.817 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-base-2:19.0.0-2457.gddeaa917.el9.x86_64 30/34 2024-04-26T03:35:22.909 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-base-2:19.0.0-2457.gddeaa917.el9.x86_64 30/34 2024-04-26T03:35:22.909 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T03:35:22.910 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T03:35:22.910 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:35:22.959 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-selinux-2:19.0.0-2457.gddeaa917.el9.x86_64 31/34 2024-04-26T03:35:25.058 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-common-2:19.0.0-2457.gddeaa917.el9.x86_64 29/34 2024-04-26T03:35:25.252 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-common-2:19.0.0-2457.gddeaa917.el9.x86_64 29/34 2024-04-26T03:35:25.824 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-common-2:19.0.0-2457.gddeaa917.el9.x86_64 29/34 2024-04-26T03:35:25.864 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-base-2:19.0.0-2457.gddeaa917.el9.x86_64 30/34 2024-04-26T03:35:25.958 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-base-2:19.0.0-2457.gddeaa917.el9.x86_64 30/34 2024-04-26T03:35:25.959 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T03:35:25.959 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T03:35:25.959 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:35:25.990 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-selinux-2:19.0.0-2457.gddeaa917.el9.x86_64 31/34 2024-04-26T03:35:28.242 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2457.gddeaa917.el9.x86_64 31/34 2024-04-26T03:35:28.242 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /sys 2024-04-26T03:35:28.242 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /proc 2024-04-26T03:35:28.243 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /mnt 2024-04-26T03:35:28.243 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /var/tmp 2024-04-26T03:35:28.243 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /home 2024-04-26T03:35:28.243 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /root 2024-04-26T03:35:28.243 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /tmp 2024-04-26T03:35:28.243 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:35:29.249 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-radosgw-2:19.0.0-2457.gddeaa917.el9.x86_64 32/34 2024-04-26T03:35:29.275 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2457.gddeaa917.el9.x86_64 32/34 2024-04-26T03:35:29.275 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T03:35:29.275 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T03:35:29.275 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T03:35:29.275 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T03:35:29.275 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:35:29.276 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-04-26T03:35:29.360 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-04-26T03:35:29.360 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-04-26T03:35:30.430 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2457.gddeaa917.el9.x86_64 31/34 2024-04-26T03:35:30.430 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /sys 2024-04-26T03:35:30.430 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /proc 2024-04-26T03:35:30.430 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /mnt 2024-04-26T03:35:30.430 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /var/tmp 2024-04-26T03:35:30.431 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /home 2024-04-26T03:35:30.431 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /root 2024-04-26T03:35:30.431 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /tmp 2024-04-26T03:35:30.431 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:35:30.615 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-04-26T03:35:30.615 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-base-2:19.0.0-2457.gddeaa917.el9.x86_64 1/34 2024-04-26T03:35:30.615 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-common-2:19.0.0-2457.gddeaa917.el9.x86_64 2/34 2024-04-26T03:35:30.615 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-radosgw-2:19.0.0-2457.gddeaa917.el9.x86_64 3/34 2024-04-26T03:35:30.615 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-selinux-2:19.0.0-2457.gddeaa917.el9.x86_64 4/34 2024-04-26T03:35:30.616 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libcephfs2-2:19.0.0-2457.gddeaa917.el9.x86_64 5/34 2024-04-26T03:35:30.616 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libradosstriper1-2:19.0.0-2457.gddeaa917.el9.x86_6 6/34 2024-04-26T03:35:30.616 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librgw2-2:19.0.0-2457.gddeaa917.el9.x86_64 7/34 2024-04-26T03:35:30.616 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2457.gddeaa917.el9. 8/34 2024-04-26T03:35:30.616 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-ceph-common-2:19.0.0-2457.gddeaa917.el9.x8 9/34 2024-04-26T03:35:30.616 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cephfs-2:19.0.0-2457.gddeaa917.el9.x86_64 10/34 2024-04-26T03:35:30.616 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rados-2:19.0.0-2457.gddeaa917.el9.x86_64 11/34 2024-04-26T03:35:30.616 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rbd-2:19.0.0-2457.gddeaa917.el9.x86_64 12/34 2024-04-26T03:35:30.616 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rgw-2:19.0.0-2457.gddeaa917.el9.x86_64 13/34 2024-04-26T03:35:30.616 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-04-26T03:35:30.616 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-04-26T03:35:30.616 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-04-26T03:35:30.616 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-04-26T03:35:30.617 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-04-26T03:35:30.617 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-04-26T03:35:30.617 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-04-26T03:35:30.617 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-04-26T03:35:30.617 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-04-26T03:35:30.617 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-04-26T03:35:30.617 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-04-26T03:35:30.617 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-04-26T03:35:30.617 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-04-26T03:35:30.617 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-04-26T03:35:30.617 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-04-26T03:35:30.617 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-04-26T03:35:30.617 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-04-26T03:35:30.618 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librados2-2:19.0.0-2457.gddeaa917.el9.x86_64 31/34 2024-04-26T03:35:30.618 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-04-26T03:35:30.618 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librbd1-2:19.0.0-2457.gddeaa917.el9.x86_64 33/34 2024-04-26T03:35:31.383 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-radosgw-2:19.0.0-2457.gddeaa917.el9.x86_64 32/34 2024-04-26T03:35:31.407 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2457.gddeaa917.el9.x86_64 32/34 2024-04-26T03:35:31.408 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T03:35:31.408 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T03:35:31.408 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T03:35:31.408 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T03:35:31.408 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:35:31.410 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-04-26T03:35:31.493 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-04-26T03:35:31.493 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-04-26T03:35:31.548 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-04-26T03:35:31.548 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:35:31.548 INFO:teuthology.orchestra.run.smithi042.stdout:Upgraded: 2024-04-26T03:35:31.548 INFO:teuthology.orchestra.run.smithi042.stdout: librados2-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:31.548 INFO:teuthology.orchestra.run.smithi042.stdout: librbd1-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:31.548 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-04-26T03:35:31.548 INFO:teuthology.orchestra.run.smithi042.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T03:35:31.548 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-base-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:31.548 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-common-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:31.549 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-radosgw-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:31.549 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-selinux-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:31.549 INFO:teuthology.orchestra.run.smithi042.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T03:35:31.549 INFO:teuthology.orchestra.run.smithi042.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T03:35:31.549 INFO:teuthology.orchestra.run.smithi042.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T03:35:31.549 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs2-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:31.549 INFO:teuthology.orchestra.run.smithi042.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T03:35:31.549 INFO:teuthology.orchestra.run.smithi042.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T03:35:31.549 INFO:teuthology.orchestra.run.smithi042.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T03:35:31.549 INFO:teuthology.orchestra.run.smithi042.stdout: libradosstriper1-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:31.549 INFO:teuthology.orchestra.run.smithi042.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T03:35:31.549 INFO:teuthology.orchestra.run.smithi042.stdout: librgw2-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:31.549 INFO:teuthology.orchestra.run.smithi042.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T03:35:31.549 INFO:teuthology.orchestra.run.smithi042.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T03:35:31.550 INFO:teuthology.orchestra.run.smithi042.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T03:35:31.550 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-argparse-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:31.550 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-common-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:31.550 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cephfs-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:31.550 INFO:teuthology.orchestra.run.smithi042.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T03:35:31.550 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rados-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:31.550 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rbd-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:31.550 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rgw-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:31.550 INFO:teuthology.orchestra.run.smithi042.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T03:35:31.550 INFO:teuthology.orchestra.run.smithi042.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T03:35:31.550 INFO:teuthology.orchestra.run.smithi042.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T03:35:31.550 INFO:teuthology.orchestra.run.smithi042.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T03:35:31.551 INFO:teuthology.orchestra.run.smithi042.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T03:35:31.551 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:35:31.551 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-04-26T03:35:31.836 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-test 2024-04-26T03:35:32.409 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:00:34 ago on Fri 26 Apr 2024 03:34:58 AM UTC. 2024-04-26T03:35:32.534 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-04-26T03:35:32.534 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:35:32.534 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-04-26T03:35:32.535 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:35:32.535 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-04-26T03:35:32.535 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-test x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 48 M 2024-04-26T03:35:32.535 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-04-26T03:35:32.535 INFO:teuthology.orchestra.run.smithi042.stdout: libcephsqlite x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 164 k 2024-04-26T03:35:32.535 INFO:teuthology.orchestra.run.smithi042.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T03:35:32.535 INFO:teuthology.orchestra.run.smithi042.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T03:35:32.535 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:35:32.535 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-04-26T03:35:32.535 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:35:32.535 INFO:teuthology.orchestra.run.smithi042.stdout:Install 4 Packages 2024-04-26T03:35:32.535 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:35:32.536 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 49 M 2024-04-26T03:35:32.536 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 207 M 2024-04-26T03:35:32.536 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-04-26T03:35:32.761 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-04-26T03:35:32.761 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-base-2:19.0.0-2457.gddeaa917.el9.x86_64 1/34 2024-04-26T03:35:32.761 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-common-2:19.0.0-2457.gddeaa917.el9.x86_64 2/34 2024-04-26T03:35:32.761 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-radosgw-2:19.0.0-2457.gddeaa917.el9.x86_64 3/34 2024-04-26T03:35:32.761 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-selinux-2:19.0.0-2457.gddeaa917.el9.x86_64 4/34 2024-04-26T03:35:32.761 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs2-2:19.0.0-2457.gddeaa917.el9.x86_64 5/34 2024-04-26T03:35:32.761 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libradosstriper1-2:19.0.0-2457.gddeaa917.el9.x86_6 6/34 2024-04-26T03:35:32.762 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librgw2-2:19.0.0-2457.gddeaa917.el9.x86_64 7/34 2024-04-26T03:35:32.762 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2457.gddeaa917.el9. 8/34 2024-04-26T03:35:32.762 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-common-2:19.0.0-2457.gddeaa917.el9.x8 9/34 2024-04-26T03:35:32.762 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cephfs-2:19.0.0-2457.gddeaa917.el9.x86_64 10/34 2024-04-26T03:35:32.762 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rados-2:19.0.0-2457.gddeaa917.el9.x86_64 11/34 2024-04-26T03:35:32.762 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rbd-2:19.0.0-2457.gddeaa917.el9.x86_64 12/34 2024-04-26T03:35:32.762 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rgw-2:19.0.0-2457.gddeaa917.el9.x86_64 13/34 2024-04-26T03:35:32.762 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-04-26T03:35:32.762 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-04-26T03:35:32.762 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-04-26T03:35:32.762 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-04-26T03:35:32.762 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-04-26T03:35:32.762 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-04-26T03:35:32.763 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-04-26T03:35:32.763 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-04-26T03:35:32.763 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-04-26T03:35:32.763 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-04-26T03:35:32.763 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-04-26T03:35:32.763 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-04-26T03:35:32.763 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-04-26T03:35:32.763 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-04-26T03:35:32.763 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-04-26T03:35:32.763 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-04-26T03:35:32.763 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-04-26T03:35:32.763 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-2:19.0.0-2457.gddeaa917.el9.x86_64 31/34 2024-04-26T03:35:32.763 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-04-26T03:35:32.763 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-2:19.0.0-2457.gddeaa917.el9.x86_64 33/34 2024-04-26T03:35:32.774 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2457.gddeaa917.el9.x86_64 31/34 2024-04-26T03:35:32.774 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /sys 2024-04-26T03:35:32.774 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /proc 2024-04-26T03:35:32.774 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /mnt 2024-04-26T03:35:32.774 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /var/tmp 2024-04-26T03:35:32.774 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /home 2024-04-26T03:35:32.774 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /root 2024-04-26T03:35:32.774 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /tmp 2024-04-26T03:35:32.774 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:35:32.810 INFO:teuthology.orchestra.run.smithi042.stdout:(1/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.6 MB/s | 305 kB 00:00 2024-04-26T03:35:32.869 INFO:teuthology.orchestra.run.smithi042.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-26T03:35:33.037 INFO:teuthology.orchestra.run.smithi042.stdout:(3/4): libcephsqlite-19.0.0-2457.gddeaa917.el9. 401 kB/s | 164 kB 00:00 2024-04-26T03:35:33.521 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-04-26T03:35:33.521 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:35:33.521 INFO:teuthology.orchestra.run.smithi092.stdout:Upgraded: 2024-04-26T03:35:33.521 INFO:teuthology.orchestra.run.smithi092.stdout: librados2-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:33.521 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:33.522 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-04-26T03:35:33.522 INFO:teuthology.orchestra.run.smithi092.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T03:35:33.522 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:33.522 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:33.522 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:33.522 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:33.522 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T03:35:33.522 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T03:35:33.522 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T03:35:33.522 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:33.522 INFO:teuthology.orchestra.run.smithi092.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T03:35:33.522 INFO:teuthology.orchestra.run.smithi092.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T03:35:33.522 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T03:35:33.523 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:33.523 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T03:35:33.523 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:33.523 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T03:35:33.523 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T03:35:33.523 INFO:teuthology.orchestra.run.smithi092.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T03:35:33.523 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:33.523 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:33.523 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:33.523 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T03:35:33.523 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:33.523 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:33.523 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:33.523 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T03:35:33.524 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T03:35:33.524 INFO:teuthology.orchestra.run.smithi092.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T03:35:33.524 INFO:teuthology.orchestra.run.smithi092.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T03:35:33.524 INFO:teuthology.orchestra.run.smithi092.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T03:35:33.524 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:35:33.524 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-26T03:35:33.762 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-radosgw-2:19.0.0-2457.gddeaa917.el9.x86_64 32/34 2024-04-26T03:35:33.785 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-test 2024-04-26T03:35:33.788 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2457.gddeaa917.el9.x86_64 32/34 2024-04-26T03:35:33.788 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T03:35:33.789 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T03:35:33.789 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T03:35:33.789 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T03:35:33.789 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:35:33.790 INFO:teuthology.orchestra.run.smithi028.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-04-26T03:35:33.881 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-04-26T03:35:33.881 INFO:teuthology.orchestra.run.smithi028.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-04-26T03:35:34.352 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:35 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:35:34.474 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-26T03:35:34.475 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:35:34.475 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-04-26T03:35:34.475 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:35:34.475 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-04-26T03:35:34.475 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 48 M 2024-04-26T03:35:34.475 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-04-26T03:35:34.475 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 164 k 2024-04-26T03:35:34.475 INFO:teuthology.orchestra.run.smithi092.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T03:35:34.475 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T03:35:34.476 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:35:34.476 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-04-26T03:35:34.476 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:35:34.476 INFO:teuthology.orchestra.run.smithi092.stdout:Install 4 Packages 2024-04-26T03:35:34.476 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:35:34.476 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 49 M 2024-04-26T03:35:34.476 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 207 M 2024-04-26T03:35:34.476 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-04-26T03:35:34.952 INFO:teuthology.orchestra.run.smithi092.stdout:(1/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.5 MB/s | 305 kB 00:00 2024-04-26T03:35:35.002 INFO:teuthology.orchestra.run.smithi092.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-26T03:35:35.014 INFO:teuthology.orchestra.run.smithi042.stdout:(4/4): ceph-test-19.0.0-2457.gddeaa917.el9.x86_ 20 MB/s | 48 MB 00:02 2024-04-26T03:35:35.018 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:35:35.018 INFO:teuthology.orchestra.run.smithi042.stdout:Total 20 MB/s | 49 MB 00:02 2024-04-26T03:35:35.045 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-04-26T03:35:35.068 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-04-26T03:35:35.068 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-04-26T03:35:35.071 INFO:teuthology.orchestra.run.smithi092.stdout:(3/4): libcephsqlite-19.0.0-2457.gddeaa917.el9. 516 kB/s | 164 kB 00:00 2024-04-26T03:35:35.168 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-04-26T03:35:35.169 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-base-2:19.0.0-2457.gddeaa917.el9.x86_64 1/34 2024-04-26T03:35:35.169 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-common-2:19.0.0-2457.gddeaa917.el9.x86_64 2/34 2024-04-26T03:35:35.169 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-radosgw-2:19.0.0-2457.gddeaa917.el9.x86_64 3/34 2024-04-26T03:35:35.169 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-selinux-2:19.0.0-2457.gddeaa917.el9.x86_64 4/34 2024-04-26T03:35:35.169 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libcephfs2-2:19.0.0-2457.gddeaa917.el9.x86_64 5/34 2024-04-26T03:35:35.169 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libradosstriper1-2:19.0.0-2457.gddeaa917.el9.x86_6 6/34 2024-04-26T03:35:35.169 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librgw2-2:19.0.0-2457.gddeaa917.el9.x86_64 7/34 2024-04-26T03:35:35.169 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2457.gddeaa917.el9. 8/34 2024-04-26T03:35:35.169 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-ceph-common-2:19.0.0-2457.gddeaa917.el9.x8 9/34 2024-04-26T03:35:35.169 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-cephfs-2:19.0.0-2457.gddeaa917.el9.x86_64 10/34 2024-04-26T03:35:35.169 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-rados-2:19.0.0-2457.gddeaa917.el9.x86_64 11/34 2024-04-26T03:35:35.169 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-rbd-2:19.0.0-2457.gddeaa917.el9.x86_64 12/34 2024-04-26T03:35:35.170 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-rgw-2:19.0.0-2457.gddeaa917.el9.x86_64 13/34 2024-04-26T03:35:35.170 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-04-26T03:35:35.170 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-04-26T03:35:35.170 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-04-26T03:35:35.170 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-04-26T03:35:35.170 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-04-26T03:35:35.170 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-04-26T03:35:35.170 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-04-26T03:35:35.170 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-04-26T03:35:35.170 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-04-26T03:35:35.170 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-04-26T03:35:35.170 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-04-26T03:35:35.170 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-04-26T03:35:35.170 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-04-26T03:35:35.171 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-04-26T03:35:35.171 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-04-26T03:35:35.171 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-04-26T03:35:35.172 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-04-26T03:35:35.172 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librados2-2:19.0.0-2457.gddeaa917.el9.x86_64 31/34 2024-04-26T03:35:35.172 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-04-26T03:35:35.172 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librbd1-2:19.0.0-2457.gddeaa917.el9.x86_64 33/34 2024-04-26T03:35:35.455 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-04-26T03:35:35.456 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-04-26T03:35:35.619 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2457.gddeaa917.el9.x86_64 31/34 2024-04-26T03:35:35.619 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /sys 2024-04-26T03:35:35.619 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /proc 2024-04-26T03:35:35.619 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /mnt 2024-04-26T03:35:35.619 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /var/tmp 2024-04-26T03:35:35.619 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /home 2024-04-26T03:35:35.619 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /root 2024-04-26T03:35:35.619 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /tmp 2024-04-26T03:35:35.619 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:35:35.964 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-04-26T03:35:35.964 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:35:35.965 INFO:teuthology.orchestra.run.smithi028.stdout:Upgraded: 2024-04-26T03:35:35.965 INFO:teuthology.orchestra.run.smithi028.stdout: librados2-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:35.965 INFO:teuthology.orchestra.run.smithi028.stdout: librbd1-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:35.965 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-04-26T03:35:35.965 INFO:teuthology.orchestra.run.smithi028.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T03:35:35.965 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-base-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:35.965 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-common-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:35.965 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-radosgw-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:35.965 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-selinux-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:35.965 INFO:teuthology.orchestra.run.smithi028.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T03:35:35.965 INFO:teuthology.orchestra.run.smithi028.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T03:35:35.965 INFO:teuthology.orchestra.run.smithi028.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T03:35:35.965 INFO:teuthology.orchestra.run.smithi028.stdout: libcephfs2-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:35.966 INFO:teuthology.orchestra.run.smithi028.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T03:35:35.966 INFO:teuthology.orchestra.run.smithi028.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T03:35:35.966 INFO:teuthology.orchestra.run.smithi028.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T03:35:35.966 INFO:teuthology.orchestra.run.smithi028.stdout: libradosstriper1-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:35.966 INFO:teuthology.orchestra.run.smithi028.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T03:35:35.966 INFO:teuthology.orchestra.run.smithi028.stdout: librgw2-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:35.966 INFO:teuthology.orchestra.run.smithi028.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T03:35:35.966 INFO:teuthology.orchestra.run.smithi028.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T03:35:35.966 INFO:teuthology.orchestra.run.smithi028.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T03:35:35.966 INFO:teuthology.orchestra.run.smithi028.stdout: python3-ceph-argparse-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:35.966 INFO:teuthology.orchestra.run.smithi028.stdout: python3-ceph-common-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:35.966 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cephfs-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:35.966 INFO:teuthology.orchestra.run.smithi028.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T03:35:35.967 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rados-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:35.967 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rbd-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:35.967 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rgw-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:35.967 INFO:teuthology.orchestra.run.smithi028.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T03:35:35.967 INFO:teuthology.orchestra.run.smithi028.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T03:35:35.967 INFO:teuthology.orchestra.run.smithi028.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T03:35:35.967 INFO:teuthology.orchestra.run.smithi028.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T03:35:35.967 INFO:teuthology.orchestra.run.smithi028.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T03:35:35.967 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:35:35.967 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-04-26T03:35:36.003 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-04-26T03:35:36.083 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-04-26T03:35:36.123 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-04-26T03:35:36.150 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libcephsqlite-2:19.0.0-2457.gddeaa917.el9.x86_64 3/4 2024-04-26T03:35:36.241 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install ceph-test 2024-04-26T03:35:36.592 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-radosgw-2:19.0.0-2457.gddeaa917.el9.x86_64 32/34 2024-04-26T03:35:36.617 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2457.gddeaa917.el9.x86_64 32/34 2024-04-26T03:35:36.618 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T03:35:36.618 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T03:35:36.618 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T03:35:36.618 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T03:35:36.618 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:35:36.620 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-04-26T03:35:36.703 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-04-26T03:35:36.703 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-04-26T03:35:36.870 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:00:37 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:35:36.997 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-04-26T03:35:36.998 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:35:36.998 INFO:teuthology.orchestra.run.smithi028.stdout: Package Arch Version Repository Size 2024-04-26T03:35:36.998 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:35:36.998 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-04-26T03:35:36.998 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-test x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 48 M 2024-04-26T03:35:36.998 INFO:teuthology.orchestra.run.smithi028.stdout:Installing dependencies: 2024-04-26T03:35:36.998 INFO:teuthology.orchestra.run.smithi028.stdout: libcephsqlite x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 164 k 2024-04-26T03:35:36.999 INFO:teuthology.orchestra.run.smithi028.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T03:35:36.999 INFO:teuthology.orchestra.run.smithi028.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T03:35:36.999 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:35:36.999 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-04-26T03:35:36.999 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:35:36.999 INFO:teuthology.orchestra.run.smithi028.stdout:Install 4 Packages 2024-04-26T03:35:36.999 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:35:37.000 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 49 M 2024-04-26T03:35:37.000 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 207 M 2024-04-26T03:35:37.000 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-04-26T03:35:37.362 INFO:teuthology.orchestra.run.smithi092.stdout:(4/4): ceph-test-19.0.0-2457.gddeaa917.el9.x86_ 19 MB/s | 48 MB 00:02 2024-04-26T03:35:37.365 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:35:37.365 INFO:teuthology.orchestra.run.smithi092.stdout:Total 17 MB/s | 49 MB 00:02 2024-04-26T03:35:37.391 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-04-26T03:35:37.411 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-04-26T03:35:37.411 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-04-26T03:35:37.424 INFO:teuthology.orchestra.run.smithi028.stdout:(1/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.2 MB/s | 305 kB 00:00 2024-04-26T03:35:37.457 INFO:teuthology.orchestra.run.smithi028.stdout:(2/4): libcephsqlite-19.0.0-2457.gddeaa917.el9. 573 kB/s | 164 kB 00:00 2024-04-26T03:35:37.501 INFO:teuthology.orchestra.run.smithi028.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 848 kB/s | 64 kB 00:00 2024-04-26T03:35:37.780 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-04-26T03:35:37.780 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-04-26T03:35:37.872 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-04-26T03:35:37.872 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-base-2:19.0.0-2457.gddeaa917.el9.x86_64 1/34 2024-04-26T03:35:37.872 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-common-2:19.0.0-2457.gddeaa917.el9.x86_64 2/34 2024-04-26T03:35:37.872 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-radosgw-2:19.0.0-2457.gddeaa917.el9.x86_64 3/34 2024-04-26T03:35:37.872 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-selinux-2:19.0.0-2457.gddeaa917.el9.x86_64 4/34 2024-04-26T03:35:37.872 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libcephfs2-2:19.0.0-2457.gddeaa917.el9.x86_64 5/34 2024-04-26T03:35:37.872 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libradosstriper1-2:19.0.0-2457.gddeaa917.el9.x86_6 6/34 2024-04-26T03:35:37.872 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librgw2-2:19.0.0-2457.gddeaa917.el9.x86_64 7/34 2024-04-26T03:35:37.872 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2457.gddeaa917.el9. 8/34 2024-04-26T03:35:37.872 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-ceph-common-2:19.0.0-2457.gddeaa917.el9.x8 9/34 2024-04-26T03:35:37.872 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cephfs-2:19.0.0-2457.gddeaa917.el9.x86_64 10/34 2024-04-26T03:35:37.873 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rados-2:19.0.0-2457.gddeaa917.el9.x86_64 11/34 2024-04-26T03:35:37.873 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rbd-2:19.0.0-2457.gddeaa917.el9.x86_64 12/34 2024-04-26T03:35:37.873 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rgw-2:19.0.0-2457.gddeaa917.el9.x86_64 13/34 2024-04-26T03:35:37.873 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-04-26T03:35:37.873 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-04-26T03:35:37.873 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-04-26T03:35:37.873 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-04-26T03:35:37.873 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-04-26T03:35:37.873 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-04-26T03:35:37.873 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-04-26T03:35:37.873 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-04-26T03:35:37.873 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-04-26T03:35:37.873 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-04-26T03:35:37.873 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-04-26T03:35:37.874 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-04-26T03:35:37.874 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-04-26T03:35:37.874 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-04-26T03:35:37.874 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-04-26T03:35:37.874 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-04-26T03:35:37.875 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-04-26T03:35:37.875 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librados2-2:19.0.0-2457.gddeaa917.el9.x86_64 31/34 2024-04-26T03:35:37.875 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-04-26T03:35:37.875 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librbd1-2:19.0.0-2457.gddeaa917.el9.x86_64 33/34 2024-04-26T03:35:38.303 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-04-26T03:35:38.388 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-04-26T03:35:38.427 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-04-26T03:35:38.456 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephsqlite-2:19.0.0-2457.gddeaa917.el9.x86_64 3/4 2024-04-26T03:35:38.650 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-04-26T03:35:38.651 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:35:38.651 INFO:teuthology.orchestra.run.smithi066.stdout:Upgraded: 2024-04-26T03:35:38.651 INFO:teuthology.orchestra.run.smithi066.stdout: librados2-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:38.651 INFO:teuthology.orchestra.run.smithi066.stdout: librbd1-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:38.651 INFO:teuthology.orchestra.run.smithi066.stdout:Installed: 2024-04-26T03:35:38.651 INFO:teuthology.orchestra.run.smithi066.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T03:35:38.651 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-base-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:38.651 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-common-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:38.651 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-radosgw-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:38.651 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-selinux-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:38.651 INFO:teuthology.orchestra.run.smithi066.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T03:35:38.652 INFO:teuthology.orchestra.run.smithi066.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T03:35:38.652 INFO:teuthology.orchestra.run.smithi066.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T03:35:38.652 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs2-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:38.652 INFO:teuthology.orchestra.run.smithi066.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T03:35:38.652 INFO:teuthology.orchestra.run.smithi066.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T03:35:38.652 INFO:teuthology.orchestra.run.smithi066.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T03:35:38.652 INFO:teuthology.orchestra.run.smithi066.stdout: libradosstriper1-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:38.652 INFO:teuthology.orchestra.run.smithi066.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T03:35:38.652 INFO:teuthology.orchestra.run.smithi066.stdout: librgw2-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:38.652 INFO:teuthology.orchestra.run.smithi066.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T03:35:38.652 INFO:teuthology.orchestra.run.smithi066.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T03:35:38.652 INFO:teuthology.orchestra.run.smithi066.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T03:35:38.652 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-argparse-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:38.652 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-common-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:38.652 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cephfs-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:38.653 INFO:teuthology.orchestra.run.smithi066.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T03:35:38.653 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rados-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:38.653 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rbd-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:38.653 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rgw-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:38.653 INFO:teuthology.orchestra.run.smithi066.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T03:35:38.653 INFO:teuthology.orchestra.run.smithi066.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T03:35:38.653 INFO:teuthology.orchestra.run.smithi066.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T03:35:38.653 INFO:teuthology.orchestra.run.smithi066.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T03:35:38.653 INFO:teuthology.orchestra.run.smithi066.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T03:35:38.653 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:35:38.653 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-04-26T03:35:38.957 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum -y install ceph-test 2024-04-26T03:35:39.520 INFO:teuthology.orchestra.run.smithi066.stdout:Last metadata expiration check: 0:00:38 ago on Fri 26 Apr 2024 03:35:01 AM UTC. 2024-04-26T03:35:39.648 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-04-26T03:35:39.648 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:35:39.648 INFO:teuthology.orchestra.run.smithi066.stdout: Package Arch Version Repository Size 2024-04-26T03:35:39.649 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:35:39.649 INFO:teuthology.orchestra.run.smithi066.stdout:Installing: 2024-04-26T03:35:39.649 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-test x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 48 M 2024-04-26T03:35:39.649 INFO:teuthology.orchestra.run.smithi066.stdout:Installing dependencies: 2024-04-26T03:35:39.649 INFO:teuthology.orchestra.run.smithi066.stdout: libcephsqlite x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 164 k 2024-04-26T03:35:39.649 INFO:teuthology.orchestra.run.smithi066.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T03:35:39.649 INFO:teuthology.orchestra.run.smithi066.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T03:35:39.649 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:35:39.649 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction Summary 2024-04-26T03:35:39.649 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:35:39.649 INFO:teuthology.orchestra.run.smithi066.stdout:Install 4 Packages 2024-04-26T03:35:39.649 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:35:39.650 INFO:teuthology.orchestra.run.smithi066.stdout:Total download size: 49 M 2024-04-26T03:35:39.650 INFO:teuthology.orchestra.run.smithi066.stdout:Installed size: 207 M 2024-04-26T03:35:39.650 INFO:teuthology.orchestra.run.smithi066.stdout:Downloading Packages: 2024-04-26T03:35:39.926 INFO:teuthology.orchestra.run.smithi028.stdout:(4/4): ceph-test-19.0.0-2457.gddeaa917.el9.x86_ 18 MB/s | 48 MB 00:02 2024-04-26T03:35:39.929 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:35:39.929 INFO:teuthology.orchestra.run.smithi028.stdout:Total 17 MB/s | 49 MB 00:02 2024-04-26T03:35:39.956 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-04-26T03:35:39.979 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-04-26T03:35:39.979 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-04-26T03:35:40.129 INFO:teuthology.orchestra.run.smithi066.stdout:(1/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.5 MB/s | 305 kB 00:00 2024-04-26T03:35:40.179 INFO:teuthology.orchestra.run.smithi066.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2024-04-26T03:35:40.365 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-04-26T03:35:40.365 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-04-26T03:35:40.596 INFO:teuthology.orchestra.run.smithi066.stdout:(3/4): libcephsqlite-19.0.0-2457.gddeaa917.el9. 245 kB/s | 164 kB 00:00 2024-04-26T03:35:40.915 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-04-26T03:35:40.922 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2457.gddeaa917.el9.x86_64 3/4 2024-04-26T03:35:41.000 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-test-2:19.0.0-2457.gddeaa917.el9.x86_64 4/4 2024-04-26T03:35:41.002 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-04-26T03:35:41.042 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-04-26T03:35:41.069 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libcephsqlite-2:19.0.0-2457.gddeaa917.el9.x86_64 3/4 2024-04-26T03:35:43.003 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2457.gddeaa917.el9.x86_64 3/4 2024-04-26T03:35:43.077 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-test-2:19.0.0-2457.gddeaa917.el9.x86_64 4/4 2024-04-26T03:35:43.302 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-test-2:19.0.0-2457.gddeaa917.el9.x86_64 4/4 2024-04-26T03:35:43.302 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-test-2:19.0.0-2457.gddeaa917.el9.x86_64 1/4 2024-04-26T03:35:43.302 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libcephsqlite-2:19.0.0-2457.gddeaa917.el9.x86_64 2/4 2024-04-26T03:35:43.302 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-04-26T03:35:43.587 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-04-26T03:35:43.588 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:35:43.588 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-04-26T03:35:43.588 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-test-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:43.588 INFO:teuthology.orchestra.run.smithi042.stdout: libcephsqlite-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:43.588 INFO:teuthology.orchestra.run.smithi042.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T03:35:43.588 INFO:teuthology.orchestra.run.smithi042.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T03:35:43.588 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:35:43.588 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-04-26T03:35:43.866 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph 2024-04-26T03:35:44.403 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:00:46 ago on Fri 26 Apr 2024 03:34:58 AM UTC. 2024-04-26T03:35:44.538 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-04-26T03:35:44.541 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:35:44.541 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-04-26T03:35:44.541 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:35:44.541 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-04-26T03:35:44.541 INFO:teuthology.orchestra.run.smithi042.stdout: ceph x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 6.6 k 2024-04-26T03:35:44.541 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-04-26T03:35:44.541 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mds x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 2.4 M 2024-04-26T03:35:44.541 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 1.5 M 2024-04-26T03:35:44.541 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2457.gddeaa917.el9 ceph-noarch 249 k 2024-04-26T03:35:44.541 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mon x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 4.7 M 2024-04-26T03:35:44.542 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-osd x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 17 M 2024-04-26T03:35:44.542 INFO:teuthology.orchestra.run.smithi042.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T03:35:44.542 INFO:teuthology.orchestra.run.smithi042.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T03:35:44.542 INFO:teuthology.orchestra.run.smithi042.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T03:35:44.542 INFO:teuthology.orchestra.run.smithi042.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T03:35:44.542 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T03:35:44.542 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T03:35:44.542 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T03:35:44.542 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T03:35:44.542 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T03:35:44.542 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T03:35:44.542 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-collections 2024-04-26T03:35:44.543 INFO:teuthology.orchestra.run.smithi042.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-04-26T03:35:44.543 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T03:35:44.543 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T03:35:44.543 INFO:teuthology.orchestra.run.smithi042.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T03:35:44.543 INFO:teuthology.orchestra.run.smithi042.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T03:35:44.543 INFO:teuthology.orchestra.run.smithi042.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T03:35:44.543 INFO:teuthology.orchestra.run.smithi042.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T03:35:44.543 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T03:35:44.543 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T03:35:44.543 INFO:teuthology.orchestra.run.smithi042.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T03:35:44.543 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T03:35:44.543 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T03:35:44.544 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T03:35:44.544 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T03:35:44.544 INFO:teuthology.orchestra.run.smithi042.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T03:35:44.544 INFO:teuthology.orchestra.run.smithi042.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T03:35:44.544 INFO:teuthology.orchestra.run.smithi042.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T03:35:44.544 INFO:teuthology.orchestra.run.smithi042.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T03:35:44.544 INFO:teuthology.orchestra.run.smithi042.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T03:35:44.544 INFO:teuthology.orchestra.run.smithi042.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T03:35:44.544 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:35:44.544 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-04-26T03:35:44.544 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:35:44.544 INFO:teuthology.orchestra.run.smithi042.stdout:Install 36 Packages 2024-04-26T03:35:44.544 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:35:44.545 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 30 M 2024-04-26T03:35:44.545 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 107 M 2024-04-26T03:35:44.545 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-04-26T03:35:45.200 INFO:teuthology.orchestra.run.smithi042.stdout:(1/36): ceph-19.0.0-2457.gddeaa917.el9.x86_64.r 31 kB/s | 6.6 kB 00:00 2024-04-26T03:35:45.359 INFO:teuthology.orchestra.run.smithi042.stdout:(2/36): ceph-mgr-19.0.0-2457.gddeaa917.el9.x86_ 4.0 MB/s | 1.5 MB 00:00 2024-04-26T03:35:45.559 INFO:teuthology.orchestra.run.smithi042.stdout:(3/36): ceph-mds-19.0.0-2457.gddeaa917.el9.x86_ 4.1 MB/s | 2.4 MB 00:00 2024-04-26T03:35:45.614 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-test-2:19.0.0-2457.gddeaa917.el9.x86_64 4/4 2024-04-26T03:35:45.614 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-test-2:19.0.0-2457.gddeaa917.el9.x86_64 1/4 2024-04-26T03:35:45.614 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephsqlite-2:19.0.0-2457.gddeaa917.el9.x86_64 2/4 2024-04-26T03:35:45.614 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-04-26T03:35:45.668 INFO:teuthology.orchestra.run.smithi042.stdout:(4/36): ceph-mgr-modules-core-19.0.0-2457.gddea 2.2 MB/s | 249 kB 00:00 2024-04-26T03:35:45.810 INFO:teuthology.orchestra.run.smithi042.stdout:(5/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.7 MB/s | 253 kB 00:00 2024-04-26T03:35:45.846 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2457.gddeaa917.el9.x86_64 3/4 2024-04-26T03:35:45.878 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-04-26T03:35:45.878 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:35:45.878 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-04-26T03:35:45.878 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:45.878 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:45.878 INFO:teuthology.orchestra.run.smithi092.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T03:35:45.879 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T03:35:45.879 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:35:45.879 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-26T03:35:45.921 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-test-2:19.0.0-2457.gddeaa917.el9.x86_64 4/4 2024-04-26T03:35:45.943 INFO:teuthology.orchestra.run.smithi042.stdout:(6/36): python3-cryptography-36.0.1-4.el9.x86_6 9.2 MB/s | 1.2 MB 00:00 2024-04-26T03:35:46.010 INFO:teuthology.orchestra.run.smithi042.stdout:(7/36): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-04-26T03:35:46.077 INFO:teuthology.orchestra.run.smithi042.stdout:(8/36): python3-pycparser-2.20-6.el9.noarch.rpm 2.0 MB/s | 135 kB 00:00 2024-04-26T03:35:46.145 INFO:teuthology.orchestra.run.smithi042.stdout:(9/36): python3-requests-2.25.1-8.el9.noarch.rp 1.8 MB/s | 125 kB 00:00 2024-04-26T03:35:46.156 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph 2024-04-26T03:35:46.220 INFO:teuthology.orchestra.run.smithi042.stdout:(10/36): python3-urllib3-1.26.5-5.el9.noarch.rp 2.8 MB/s | 215 kB 00:00 2024-04-26T03:35:46.395 INFO:teuthology.orchestra.run.smithi042.stdout:(11/36): ceph-osd-19.0.0-2457.gddeaa917.el9.x86 16 MB/s | 17 MB 00:01 2024-04-26T03:35:46.412 INFO:teuthology.orchestra.run.smithi042.stdout:(12/36): lua-5.4.4-4.el9.x86_64.rpm 982 kB/s | 188 kB 00:00 2024-04-26T03:35:46.437 INFO:teuthology.orchestra.run.smithi042.stdout:(13/36): python3-mako-1.1.4-6.el9.noarch.rpm 4.1 MB/s | 172 kB 00:00 2024-04-26T03:35:46.454 INFO:teuthology.orchestra.run.smithi042.stdout:(14/36): python3-markupsafe-1.1.1-12.el9.x86_64 821 kB/s | 35 kB 00:00 2024-04-26T03:35:46.471 INFO:teuthology.orchestra.run.smithi042.stdout:(15/36): python3-pytz-2021.1-5.el9.noarch.rpm 1.5 MB/s | 51 kB 00:00 2024-04-26T03:35:46.488 INFO:teuthology.orchestra.run.smithi042.stdout:(16/36): python3-toml-0.10.2-6.el9.noarch.rpm 1.2 MB/s | 42 kB 00:00 2024-04-26T03:35:46.522 INFO:teuthology.orchestra.run.smithi042.stdout:(17/36): lua-devel-5.4.4-4.el9.x86_64.rpm 446 kB/s | 22 kB 00:00 2024-04-26T03:35:46.686 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:47 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:35:46.811 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-26T03:35:46.814 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:35:46.814 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-04-26T03:35:46.814 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:35:46.814 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-04-26T03:35:46.814 INFO:teuthology.orchestra.run.smithi092.stdout: ceph x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 6.6 k 2024-04-26T03:35:46.814 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-04-26T03:35:46.814 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 2.4 M 2024-04-26T03:35:46.814 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 1.5 M 2024-04-26T03:35:46.815 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2457.gddeaa917.el9 ceph-noarch 249 k 2024-04-26T03:35:46.815 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 4.7 M 2024-04-26T03:35:46.815 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 17 M 2024-04-26T03:35:46.815 INFO:teuthology.orchestra.run.smithi092.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T03:35:46.815 INFO:teuthology.orchestra.run.smithi092.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T03:35:46.815 INFO:teuthology.orchestra.run.smithi092.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T03:35:46.815 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T03:35:46.815 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T03:35:46.815 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T03:35:46.815 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T03:35:46.815 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T03:35:46.815 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T03:35:46.815 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T03:35:46.815 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-collections 2024-04-26T03:35:46.816 INFO:teuthology.orchestra.run.smithi092.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-04-26T03:35:46.816 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T03:35:46.816 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T03:35:46.816 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T03:35:46.816 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T03:35:46.816 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T03:35:46.816 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T03:35:46.816 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T03:35:46.816 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T03:35:46.816 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T03:35:46.816 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T03:35:46.816 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T03:35:46.816 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T03:35:46.816 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T03:35:46.817 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T03:35:46.817 INFO:teuthology.orchestra.run.smithi092.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T03:35:46.817 INFO:teuthology.orchestra.run.smithi092.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T03:35:46.817 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T03:35:46.817 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T03:35:46.817 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T03:35:46.817 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:35:46.817 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-04-26T03:35:46.817 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:35:46.817 INFO:teuthology.orchestra.run.smithi092.stdout:Install 36 Packages 2024-04-26T03:35:46.817 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:35:46.818 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 30 M 2024-04-26T03:35:46.818 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 107 M 2024-04-26T03:35:46.818 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-04-26T03:35:47.064 INFO:teuthology.orchestra.run.smithi042.stdout:(18/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 80 kB/s | 43 kB 00:00 2024-04-26T03:35:47.239 INFO:teuthology.orchestra.run.smithi042.stdout:(19/36): luarocks-3.9.2-1.el9.noarch.rpm 201 kB/s | 151 kB 00:00 2024-04-26T03:35:47.398 INFO:teuthology.orchestra.run.smithi042.stdout:(20/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 515 kB/s | 172 kB 00:00 2024-04-26T03:35:47.515 INFO:teuthology.orchestra.run.smithi042.stdout:(21/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 92 kB/s | 11 kB 00:00 2024-04-26T03:35:47.540 INFO:teuthology.orchestra.run.smithi042.stdout:(22/36): python3-cherrypy-18.6.1-2.el9.noarch.r 1.2 MB/s | 358 kB 00:00 2024-04-26T03:35:47.624 INFO:teuthology.orchestra.run.smithi042.stdout:(23/36): python3-jaraco-classes-3.2.1-5.el9.noa 163 kB/s | 18 kB 00:00 2024-04-26T03:35:47.657 INFO:teuthology.orchestra.run.smithi042.stdout:(24/36): python3-jaraco-collections-3.0.0-8.el9 198 kB/s | 23 kB 00:00 2024-04-26T03:35:47.732 INFO:teuthology.orchestra.run.smithi042.stdout:(25/36): python3-jaraco-functools-3.5.0-2.el9.n 179 kB/s | 19 kB 00:00 2024-04-26T03:35:47.766 INFO:teuthology.orchestra.run.smithi042.stdout:(26/36): python3-jaraco-text-3.2.0-6.el9.noarch 181 kB/s | 20 kB 00:00 2024-04-26T03:35:47.841 INFO:teuthology.orchestra.run.smithi042.stdout:(27/36): python3-logutils-0.3.5-21.el9.noarch.r 426 kB/s | 46 kB 00:00 2024-04-26T03:35:47.900 INFO:teuthology.orchestra.run.smithi042.stdout:(28/36): python3-more-itertools-8.12.0-2.el9.no 590 kB/s | 79 kB 00:00 2024-04-26T03:35:47.992 INFO:teuthology.orchestra.run.smithi042.stdout:(29/36): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-04-26T03:35:48.017 INFO:teuthology.orchestra.run.smithi042.stdout:(30/36): python3-portend-3.1.0-2.el9.noarch.rpm 140 kB/s | 16 kB 00:00 2024-04-26T03:35:48.109 INFO:teuthology.orchestra.run.smithi042.stdout:(31/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 770 kB/s | 90 kB 00:00 2024-04-26T03:35:48.134 INFO:teuthology.orchestra.run.smithi042.stdout:(32/36): python3-tempora-5.0.0-2.el9.noarch.rpm 306 kB/s | 36 kB 00:00 2024-04-26T03:35:48.246 INFO:teuthology.orchestra.run.smithi092.stdout:(1/36): ceph-19.0.0-2457.gddeaa917.el9.x86_64.r 23 kB/s | 6.6 kB 00:00 2024-04-26T03:35:48.295 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-test-2:19.0.0-2457.gddeaa917.el9.x86_64 4/4 2024-04-26T03:35:48.295 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-test-2:19.0.0-2457.gddeaa917.el9.x86_64 1/4 2024-04-26T03:35:48.295 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libcephsqlite-2:19.0.0-2457.gddeaa917.el9.x86_64 2/4 2024-04-26T03:35:48.295 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-04-26T03:35:48.318 INFO:teuthology.orchestra.run.smithi042.stdout:(33/36): ceph-mon-19.0.0-2457.gddeaa917.el9.x86 1.5 MB/s | 4.7 MB 00:03 2024-04-26T03:35:48.343 INFO:teuthology.orchestra.run.smithi042.stdout:(34/36): python3-webob-1.8.7-6.el9.noarch.rpm 983 kB/s | 230 kB 00:00 2024-04-26T03:35:48.410 INFO:teuthology.orchestra.run.smithi042.stdout:(35/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 1.5 MB/s | 427 kB 00:00 2024-04-26T03:35:48.555 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-04-26T03:35:48.556 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:35:48.556 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-04-26T03:35:48.556 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-test-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:48.556 INFO:teuthology.orchestra.run.smithi028.stdout: libcephsqlite-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:48.556 INFO:teuthology.orchestra.run.smithi028.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T03:35:48.556 INFO:teuthology.orchestra.run.smithi028.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T03:35:48.556 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:35:48.556 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-04-26T03:35:48.719 INFO:teuthology.orchestra.run.smithi042.stdout:(36/36): python3-zc-lockfile-2.0-10.el9.noarch. 50 kB/s | 20 kB 00:00 2024-04-26T03:35:48.725 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:35:48.726 INFO:teuthology.orchestra.run.smithi042.stdout:Total 7.2 MB/s | 30 MB 00:04 2024-04-26T03:35:48.827 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install ceph 2024-04-26T03:35:48.866 INFO:teuthology.orchestra.run.smithi066.stdout:(4/4): ceph-test-19.0.0-2457.gddeaa917.el9.x86_ 5.4 MB/s | 48 MB 00:08 2024-04-26T03:35:48.866 INFO:teuthology.orchestra.run.smithi066.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:35:48.867 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-04-26T03:35:48.867 INFO:teuthology.orchestra.run.smithi066.stdout:Total 5.3 MB/s | 49 MB 00:09 2024-04-26T03:35:48.893 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction check 2024-04-26T03:35:48.911 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-04-26T03:35:48.911 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-04-26T03:35:48.916 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction check succeeded. 2024-04-26T03:35:48.916 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction test 2024-04-26T03:35:49.004 INFO:teuthology.orchestra.run.smithi092.stdout:(2/36): ceph-mgr-19.0.0-2457.gddeaa917.el9.x86_ 1.4 MB/s | 1.5 MB 00:01 2024-04-26T03:35:49.188 INFO:teuthology.orchestra.run.smithi092.stdout:(3/36): ceph-mds-19.0.0-2457.gddeaa917.el9.x86_ 1.9 MB/s | 2.4 MB 00:01 2024-04-26T03:35:49.225 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-04-26T03:35:49.226 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-04-26T03:35:49.303 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction test succeeded. 2024-04-26T03:35:49.303 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction 2024-04-26T03:35:49.305 INFO:teuthology.orchestra.run.smithi092.stdout:(4/36): ceph-mgr-modules-core-19.0.0-2457.gddea 2.1 MB/s | 249 kB 00:00 2024-04-26T03:35:49.365 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:00:50 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:35:49.499 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-04-26T03:35:49.501 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:35:49.502 INFO:teuthology.orchestra.run.smithi028.stdout: Package Arch Version Repository Size 2024-04-26T03:35:49.502 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:35:49.502 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-04-26T03:35:49.502 INFO:teuthology.orchestra.run.smithi028.stdout: ceph x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 6.6 k 2024-04-26T03:35:49.502 INFO:teuthology.orchestra.run.smithi028.stdout:Installing dependencies: 2024-04-26T03:35:49.502 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mds x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 2.4 M 2024-04-26T03:35:49.502 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 1.5 M 2024-04-26T03:35:49.502 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2457.gddeaa917.el9 ceph-noarch 249 k 2024-04-26T03:35:49.503 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mon x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 4.7 M 2024-04-26T03:35:49.503 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-osd x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 17 M 2024-04-26T03:35:49.503 INFO:teuthology.orchestra.run.smithi028.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T03:35:49.503 INFO:teuthology.orchestra.run.smithi028.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T03:35:49.503 INFO:teuthology.orchestra.run.smithi028.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T03:35:49.503 INFO:teuthology.orchestra.run.smithi028.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T03:35:49.503 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T03:35:49.504 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T03:35:49.504 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T03:35:49.504 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T03:35:49.504 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T03:35:49.504 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T03:35:49.505 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-collections 2024-04-26T03:35:49.505 INFO:teuthology.orchestra.run.smithi028.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-04-26T03:35:49.505 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T03:35:49.505 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T03:35:49.505 INFO:teuthology.orchestra.run.smithi028.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T03:35:49.505 INFO:teuthology.orchestra.run.smithi028.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T03:35:49.506 INFO:teuthology.orchestra.run.smithi028.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T03:35:49.506 INFO:teuthology.orchestra.run.smithi028.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T03:35:49.506 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T03:35:49.506 INFO:teuthology.orchestra.run.smithi028.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T03:35:49.506 INFO:teuthology.orchestra.run.smithi028.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T03:35:49.507 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T03:35:49.507 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T03:35:49.507 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T03:35:49.507 INFO:teuthology.orchestra.run.smithi028.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T03:35:49.507 INFO:teuthology.orchestra.run.smithi028.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T03:35:49.508 INFO:teuthology.orchestra.run.smithi028.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T03:35:49.508 INFO:teuthology.orchestra.run.smithi028.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T03:35:49.508 INFO:teuthology.orchestra.run.smithi028.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T03:35:49.508 INFO:teuthology.orchestra.run.smithi028.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T03:35:49.508 INFO:teuthology.orchestra.run.smithi028.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T03:35:49.508 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:35:49.508 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-04-26T03:35:49.508 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:35:49.508 INFO:teuthology.orchestra.run.smithi028.stdout:Install 36 Packages 2024-04-26T03:35:49.509 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:35:49.510 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 30 M 2024-04-26T03:35:49.510 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 107 M 2024-04-26T03:35:49.510 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-04-26T03:35:49.588 INFO:teuthology.orchestra.run.smithi092.stdout:(5/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 893 kB/s | 253 kB 00:00 2024-04-26T03:35:49.689 INFO:teuthology.orchestra.run.smithi092.stdout:(6/36): ceph-mon-19.0.0-2457.gddeaa917.el9.x86_ 3.3 MB/s | 4.7 MB 00:01 2024-04-26T03:35:49.845 INFO:teuthology.orchestra.run.smithi066.stdout: Preparing : 1/1 2024-04-26T03:35:49.914 INFO:teuthology.orchestra.run.smithi092.stdout:(7/36): python3-cryptography-36.0.1-4.el9.x86_6 3.8 MB/s | 1.2 MB 00:00 2024-04-26T03:35:49.923 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-04-26T03:35:49.947 INFO:teuthology.orchestra.run.smithi092.stdout:(8/36): python3-ply-3.11-14.el9.noarch.rpm 411 kB/s | 106 kB 00:00 2024-04-26T03:35:49.962 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-04-26T03:35:49.981 INFO:teuthology.orchestra.run.smithi092.stdout:(9/36): python3-pycparser-2.20-6.el9.noarch.rpm 2.0 MB/s | 135 kB 00:00 2024-04-26T03:35:49.989 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libcephsqlite-2:19.0.0-2457.gddeaa917.el9.x86_64 3/4 2024-04-26T03:35:50.006 INFO:teuthology.orchestra.run.smithi092.stdout:(10/36): python3-requests-2.25.1-8.el9.noarch.r 2.1 MB/s | 125 kB 00:00 2024-04-26T03:35:50.032 INFO:teuthology.orchestra.run.smithi092.stdout:(11/36): python3-urllib3-1.26.5-5.el9.noarch.rp 4.2 MB/s | 215 kB 00:00 2024-04-26T03:35:50.057 INFO:teuthology.orchestra.run.smithi092.stdout:(12/36): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-26T03:35:50.082 INFO:teuthology.orchestra.run.smithi092.stdout:(13/36): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-04-26T03:35:50.107 INFO:teuthology.orchestra.run.smithi092.stdout:(14/36): python3-markupsafe-1.1.1-12.el9.x86_64 692 kB/s | 35 kB 00:00 2024-04-26T03:35:50.133 INFO:teuthology.orchestra.run.smithi092.stdout:(15/36): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-26T03:35:50.158 INFO:teuthology.orchestra.run.smithi092.stdout:(16/36): python3-toml-0.10.2-6.el9.noarch.rpm 830 kB/s | 42 kB 00:00 2024-04-26T03:35:50.183 INFO:teuthology.orchestra.run.smithi092.stdout:(17/36): lua-devel-5.4.4-4.el9.x86_64.rpm 446 kB/s | 22 kB 00:00 2024-04-26T03:35:50.309 INFO:teuthology.orchestra.run.smithi092.stdout:(18/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 347 kB/s | 43 kB 00:00 2024-04-26T03:35:50.342 INFO:teuthology.orchestra.run.smithi092.stdout:(19/36): luarocks-3.9.2-1.el9.noarch.rpm 822 kB/s | 151 kB 00:00 2024-04-26T03:35:50.417 INFO:teuthology.orchestra.run.smithi092.stdout:(20/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.5 MB/s | 172 kB 00:00 2024-04-26T03:35:50.435 INFO:teuthology.orchestra.run.smithi028.stdout:(1/36): ceph-19.0.0-2457.gddeaa917.el9.x86_64.r 21 kB/s | 6.6 kB 00:00 2024-04-26T03:35:50.442 INFO:teuthology.orchestra.run.smithi092.stdout:(21/36): python3-cherrypy-18.6.1-2.el9.noarch.r 3.5 MB/s | 358 kB 00:00 2024-04-26T03:35:50.468 INFO:teuthology.orchestra.run.smithi092.stdout:(22/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 213 kB/s | 11 kB 00:00 2024-04-26T03:35:50.493 INFO:teuthology.orchestra.run.smithi092.stdout:(23/36): python3-jaraco-classes-3.2.1-5.el9.noa 351 kB/s | 18 kB 00:00 2024-04-26T03:35:50.499 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-04-26T03:35:50.510 INFO:teuthology.orchestra.run.smithi092.stdout:(24/36): python3-jaraco-collections-3.0.0-8.el9 551 kB/s | 23 kB 00:00 2024-04-26T03:35:50.544 INFO:teuthology.orchestra.run.smithi092.stdout:(25/36): python3-jaraco-functools-3.5.0-2.el9.n 388 kB/s | 19 kB 00:00 2024-04-26T03:35:50.563 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-04-26T03:35:50.569 INFO:teuthology.orchestra.run.smithi092.stdout:(26/36): python3-jaraco-text-3.2.0-6.el9.noarch 336 kB/s | 20 kB 00:00 2024-04-26T03:35:50.594 INFO:teuthology.orchestra.run.smithi092.stdout:(27/36): python3-logutils-0.3.5-21.el9.noarch.r 920 kB/s | 46 kB 00:00 2024-04-26T03:35:50.597 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-04-26T03:35:50.619 INFO:teuthology.orchestra.run.smithi092.stdout:(28/36): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-04-26T03:35:50.624 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-04-26T03:35:50.653 INFO:teuthology.orchestra.run.smithi092.stdout:(29/36): python3-pecan-1.4.2-3.el9.noarch.rpm 4.5 MB/s | 272 kB 00:00 2024-04-26T03:35:50.670 INFO:teuthology.orchestra.run.smithi092.stdout:(30/36): python3-portend-3.1.0-2.el9.noarch.rpm 327 kB/s | 16 kB 00:00 2024-04-26T03:35:50.701 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-04-26T03:35:50.703 INFO:teuthology.orchestra.run.smithi092.stdout:(31/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-04-26T03:35:50.719 INFO:teuthology.orchestra.run.smithi028.stdout:(2/36): ceph-mds-19.0.0-2457.gddeaa917.el9.x86_ 3.9 MB/s | 2.4 MB 00:00 2024-04-26T03:35:50.720 INFO:teuthology.orchestra.run.smithi092.stdout:(32/36): python3-tempora-5.0.0-2.el9.noarch.rpm 714 kB/s | 36 kB 00:00 2024-04-26T03:35:50.741 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-04-26T03:35:50.763 INFO:teuthology.orchestra.run.smithi092.stdout:(33/36): python3-webob-1.8.7-6.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-04-26T03:35:50.773 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-04-26T03:35:50.804 INFO:teuthology.orchestra.run.smithi092.stdout:(34/36): python3-zc-lockfile-2.0-10.el9.noarch. 479 kB/s | 20 kB 00:00 2024-04-26T03:35:50.838 INFO:teuthology.orchestra.run.smithi092.stdout:(35/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.6 MB/s | 427 kB 00:00 2024-04-26T03:35:50.871 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-04-26T03:35:50.948 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-04-26T03:35:50.994 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-04-26T03:35:51.028 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-04-26T03:35:51.057 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/36 2024-04-26T03:35:51.092 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/36 2024-04-26T03:35:51.125 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/36 2024-04-26T03:35:51.166 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/36 2024-04-26T03:35:51.210 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/36 2024-04-26T03:35:51.244 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/36 2024-04-26T03:35:51.282 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/36 2024-04-26T03:35:51.359 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 18/36 2024-04-26T03:35:51.513 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 19/36 2024-04-26T03:35:51.595 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 20/36 2024-04-26T03:35:51.653 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 21/36 2024-04-26T03:35:51.690 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 22/36 2024-04-26T03:35:51.730 INFO:teuthology.orchestra.run.smithi092.stdout:(36/36): ceph-osd-19.0.0-2457.gddeaa917.el9.x86 6.2 MB/s | 17 MB 00:02 2024-04-26T03:35:51.734 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:35:51.734 INFO:teuthology.orchestra.run.smithi092.stdout:Total 6.1 MB/s | 30 MB 00:04 2024-04-26T03:35:51.748 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-ply-3.11-14.el9.noarch 23/36 2024-04-26T03:35:51.802 INFO:teuthology.orchestra.run.smithi028.stdout:(3/36): ceph-mgr-19.0.0-2457.gddeaa917.el9.x86_ 909 kB/s | 1.5 MB 00:01 2024-04-26T03:35:51.813 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 24/36 2024-04-26T03:35:51.869 INFO:teuthology.orchestra.run.smithi028.stdout:(4/36): ceph-mon-19.0.0-2457.gddeaa917.el9.x86_ 3.3 MB/s | 4.7 MB 00:01 2024-04-26T03:35:51.870 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-04-26T03:35:51.908 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-04-26T03:35:51.909 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-04-26T03:35:51.986 INFO:teuthology.orchestra.run.smithi028.stdout:(5/36): ceph-mgr-modules-core-19.0.0-2457.gddea 1.3 MB/s | 249 kB 00:00 2024-04-26T03:35:52.010 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 25/36 2024-04-26T03:35:52.011 INFO:teuthology.orchestra.run.smithi028.stdout:(6/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.7 MB/s | 253 kB 00:00 2024-04-26T03:35:52.062 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 26/36 2024-04-26T03:35:52.070 INFO:teuthology.orchestra.run.smithi028.stdout:(7/36): python3-cryptography-36.0.1-4.el9.x86_6 15 MB/s | 1.2 MB 00:00 2024-04-26T03:35:52.095 INFO:teuthology.orchestra.run.smithi028.stdout:(8/36): python3-ply-3.11-14.el9.noarch.rpm 1.3 MB/s | 106 kB 00:00 2024-04-26T03:35:52.120 INFO:teuthology.orchestra.run.smithi028.stdout:(9/36): python3-pycparser-2.20-6.el9.noarch.rpm 2.7 MB/s | 135 kB 00:00 2024-04-26T03:35:52.143 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 27/36 2024-04-26T03:35:52.145 INFO:teuthology.orchestra.run.smithi028.stdout:(10/36): python3-requests-2.25.1-8.el9.noarch.r 2.4 MB/s | 125 kB 00:00 2024-04-26T03:35:52.162 INFO:teuthology.orchestra.run.smithi028.stdout:(11/36): python3-urllib3-1.26.5-5.el9.noarch.rp 5.0 MB/s | 215 kB 00:00 2024-04-26T03:35:52.188 INFO:teuthology.orchestra.run.smithi028.stdout:(12/36): lua-5.4.4-4.el9.x86_64.rpm 4.4 MB/s | 188 kB 00:00 2024-04-26T03:35:52.204 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-04-26T03:35:52.205 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-04-26T03:35:52.213 INFO:teuthology.orchestra.run.smithi028.stdout:(13/36): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-04-26T03:35:52.238 INFO:teuthology.orchestra.run.smithi028.stdout:(14/36): python3-markupsafe-1.1.1-12.el9.x86_64 694 kB/s | 35 kB 00:00 2024-04-26T03:35:52.242 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 28/36 2024-04-26T03:35:52.263 INFO:teuthology.orchestra.run.smithi028.stdout:(15/36): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-26T03:35:52.277 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 29/36 2024-04-26T03:35:52.289 INFO:teuthology.orchestra.run.smithi028.stdout:(16/36): python3-toml-0.10.2-6.el9.noarch.rpm 834 kB/s | 42 kB 00:00 2024-04-26T03:35:52.314 INFO:teuthology.orchestra.run.smithi028.stdout:(17/36): lua-devel-5.4.4-4.el9.x86_64.rpm 443 kB/s | 22 kB 00:00 2024-04-26T03:35:52.410 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 30/36 2024-04-26T03:35:52.481 INFO:teuthology.orchestra.run.smithi028.stdout:(18/36): ceph-osd-19.0.0-2457.gddeaa917.el9.x86 9.5 MB/s | 17 MB 00:01 2024-04-26T03:35:52.583 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2457.gddeaa917.el9. 31/36 2024-04-26T03:35:52.612 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-2:19.0.0-2457.gddeaa917.el9.x86_64 32/36 2024-04-26T03:35:52.639 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2457.gddeaa917.el9.x86_64 32/36 2024-04-26T03:35:52.639 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T03:35:52.639 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T03:35:52.639 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T03:35:52.639 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T03:35:52.639 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:35:52.748 INFO:teuthology.orchestra.run.smithi028.stdout:(19/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 100 kB/s | 43 kB 00:00 2024-04-26T03:35:52.790 INFO:teuthology.orchestra.run.smithi028.stdout:(20/36): luarocks-3.9.2-1.el9.noarch.rpm 302 kB/s | 151 kB 00:00 2024-04-26T03:35:52.857 INFO:teuthology.orchestra.run.smithi028.stdout:(21/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 160 kB/s | 11 kB 00:00 2024-04-26T03:35:52.908 INFO:teuthology.orchestra.run.smithi028.stdout:(22/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 461 kB/s | 172 kB 00:00 2024-04-26T03:35:52.933 INFO:teuthology.orchestra.run.smithi028.stdout:(23/36): python3-jaraco-classes-3.2.1-5.el9.noa 234 kB/s | 18 kB 00:00 2024-04-26T03:35:52.975 INFO:teuthology.orchestra.run.smithi028.stdout:(24/36): python3-cherrypy-18.6.1-2.el9.noarch.r 1.5 MB/s | 358 kB 00:00 2024-04-26T03:35:53.000 INFO:teuthology.orchestra.run.smithi028.stdout:(25/36): python3-jaraco-collections-3.0.0-8.el9 251 kB/s | 23 kB 00:00 2024-04-26T03:35:53.025 INFO:teuthology.orchestra.run.smithi028.stdout:(26/36): python3-jaraco-functools-3.5.0-2.el9.n 211 kB/s | 19 kB 00:00 2024-04-26T03:35:53.051 INFO:teuthology.orchestra.run.smithi028.stdout:(27/36): python3-jaraco-text-3.2.0-6.el9.noarch 260 kB/s | 20 kB 00:00 2024-04-26T03:35:53.076 INFO:teuthology.orchestra.run.smithi028.stdout:(28/36): python3-logutils-0.3.5-21.el9.noarch.r 616 kB/s | 46 kB 00:00 2024-04-26T03:35:53.110 INFO:teuthology.orchestra.run.smithi028.stdout:(29/36): python3-more-itertools-8.12.0-2.el9.no 941 kB/s | 79 kB 00:00 2024-04-26T03:35:53.143 INFO:teuthology.orchestra.run.smithi028.stdout:(30/36): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-04-26T03:35:53.168 INFO:teuthology.orchestra.run.smithi028.stdout:(31/36): python3-portend-3.1.0-2.el9.noarch.rpm 178 kB/s | 16 kB 00:00 2024-04-26T03:35:53.194 INFO:teuthology.orchestra.run.smithi028.stdout:(32/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.0 MB/s | 90 kB 00:00 2024-04-26T03:35:53.219 INFO:teuthology.orchestra.run.smithi028.stdout:(33/36): python3-tempora-5.0.0-2.el9.noarch.rpm 474 kB/s | 36 kB 00:00 2024-04-26T03:35:53.286 INFO:teuthology.orchestra.run.smithi028.stdout:(34/36): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-04-26T03:35:53.311 INFO:teuthology.orchestra.run.smithi028.stdout:(35/36): python3-zc-lockfile-2.0-10.el9.noarch. 217 kB/s | 20 kB 00:00 2024-04-26T03:35:53.361 INFO:teuthology.orchestra.run.smithi028.stdout:(36/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 2.5 MB/s | 427 kB 00:00 2024-04-26T03:35:53.368 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:35:53.368 INFO:teuthology.orchestra.run.smithi028.stdout:Total 7.8 MB/s | 30 MB 00:03 2024-04-26T03:35:53.447 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-04-26T03:35:53.508 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-04-26T03:35:53.512 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-04-26T03:35:53.549 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-04-26T03:35:53.556 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-04-26T03:35:53.556 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-04-26T03:35:53.585 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-04-26T03:35:53.657 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-04-26T03:35:53.717 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-04-26T03:35:53.750 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-04-26T03:35:53.842 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-04-26T03:35:53.870 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-04-26T03:35:53.871 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-04-26T03:35:53.928 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-04-26T03:35:53.970 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-04-26T03:35:54.004 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-04-26T03:35:54.034 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/36 2024-04-26T03:35:54.068 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/36 2024-04-26T03:35:54.102 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/36 2024-04-26T03:35:54.143 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/36 2024-04-26T03:35:54.155 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-osd-2:19.0.0-2457.gddeaa917.el9.x86_64 33/36 2024-04-26T03:35:54.183 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-osd-2:19.0.0-2457.gddeaa917.el9.x86_64 33/36 2024-04-26T03:35:54.183 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T03:35:54.183 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T03:35:54.183 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T03:35:54.183 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T03:35:54.183 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:35:54.194 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/36 2024-04-26T03:35:54.229 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/36 2024-04-26T03:35:54.259 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/36 2024-04-26T03:35:54.332 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 18/36 2024-04-26T03:35:54.470 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 19/36 2024-04-26T03:35:54.549 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 20/36 2024-04-26T03:35:54.603 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 21/36 2024-04-26T03:35:54.641 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 22/36 2024-04-26T03:35:54.648 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mon-2:19.0.0-2457.gddeaa917.el9.x86_64 34/36 2024-04-26T03:35:54.672 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mon-2:19.0.0-2457.gddeaa917.el9.x86_64 34/36 2024-04-26T03:35:54.672 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T03:35:54.673 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T03:35:54.673 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T03:35:54.673 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T03:35:54.673 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:35:54.690 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ply-3.11-14.el9.noarch 23/36 2024-04-26T03:35:54.754 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 24/36 2024-04-26T03:35:54.762 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2457.gddeaa917.el9.x86_64 3/4 2024-04-26T03:35:54.839 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-test-2:19.0.0-2457.gddeaa917.el9.x86_64 4/4 2024-04-26T03:35:54.907 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mds-2:19.0.0-2457.gddeaa917.el9.x86_64 35/36 2024-04-26T03:35:54.931 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mds-2:19.0.0-2457.gddeaa917.el9.x86_64 35/36 2024-04-26T03:35:54.931 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T03:35:54.931 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T03:35:54.931 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T03:35:54.931 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T03:35:54.931 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:35:54.951 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 25/36 2024-04-26T03:35:54.995 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 26/36 2024-04-26T03:35:54.997 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-2:19.0.0-2457.gddeaa917.el9.x86_64 36/36 2024-04-26T03:35:55.074 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 27/36 2024-04-26T03:35:55.146 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-04-26T03:35:55.162 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 28/36 2024-04-26T03:35:55.194 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 29/36 2024-04-26T03:35:55.214 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-04-26T03:35:55.247 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-04-26T03:35:55.275 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-04-26T03:35:55.312 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 30/36 2024-04-26T03:35:55.350 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-04-26T03:35:55.391 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-04-26T03:35:55.424 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-04-26T03:35:55.470 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2457.gddeaa917.el9. 31/36 2024-04-26T03:35:55.496 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-2:19.0.0-2457.gddeaa917.el9.x86_64 32/36 2024-04-26T03:35:55.522 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2457.gddeaa917.el9.x86_64 32/36 2024-04-26T03:35:55.522 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T03:35:55.522 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T03:35:55.522 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T03:35:55.522 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T03:35:55.522 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:35:55.522 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-04-26T03:35:55.598 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-04-26T03:35:55.652 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-04-26T03:35:55.687 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-04-26T03:35:55.716 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/36 2024-04-26T03:35:55.750 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/36 2024-04-26T03:35:55.784 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/36 2024-04-26T03:35:55.825 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/36 2024-04-26T03:35:55.869 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/36 2024-04-26T03:35:55.902 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/36 2024-04-26T03:35:55.941 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/36 2024-04-26T03:35:56.017 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 18/36 2024-04-26T03:35:56.173 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 19/36 2024-04-26T03:35:56.261 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 20/36 2024-04-26T03:35:56.311 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 21/36 2024-04-26T03:35:56.349 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 22/36 2024-04-26T03:35:56.407 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-ply-3.11-14.el9.noarch 23/36 2024-04-26T03:35:56.472 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 24/36 2024-04-26T03:35:56.668 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 25/36 2024-04-26T03:35:56.719 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 26/36 2024-04-26T03:35:56.768 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-2:19.0.0-2457.gddeaa917.el9.x86_64 36/36 2024-04-26T03:35:56.768 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-2:19.0.0-2457.gddeaa917.el9.x86_64 1/36 2024-04-26T03:35:56.768 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mds-2:19.0.0-2457.gddeaa917.el9.x86_64 2/36 2024-04-26T03:35:56.768 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-2:19.0.0-2457.gddeaa917.el9.x86_64 3/36 2024-04-26T03:35:56.769 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mon-2:19.0.0-2457.gddeaa917.el9.x86_64 4/36 2024-04-26T03:35:56.769 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-osd-2:19.0.0-2457.gddeaa917.el9.x86_64 5/36 2024-04-26T03:35:56.769 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2457.gddeaa917.el9. 6/36 2024-04-26T03:35:56.769 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/36 2024-04-26T03:35:56.769 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/36 2024-04-26T03:35:56.769 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/36 2024-04-26T03:35:56.769 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/36 2024-04-26T03:35:56.769 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/36 2024-04-26T03:35:56.769 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/36 2024-04-26T03:35:56.769 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/36 2024-04-26T03:35:56.769 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/36 2024-04-26T03:35:56.769 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/36 2024-04-26T03:35:56.769 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/36 2024-04-26T03:35:56.770 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/36 2024-04-26T03:35:56.770 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-04-26T03:35:56.770 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-04-26T03:35:56.770 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-04-26T03:35:56.770 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-04-26T03:35:56.770 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-04-26T03:35:56.770 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-04-26T03:35:56.770 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-04-26T03:35:56.771 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-04-26T03:35:56.771 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-04-26T03:35:56.771 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-04-26T03:35:56.771 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-04-26T03:35:56.771 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-04-26T03:35:56.771 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-04-26T03:35:56.771 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-04-26T03:35:56.771 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-04-26T03:35:56.771 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-04-26T03:35:56.771 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-04-26T03:35:56.771 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-04-26T03:35:56.801 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 27/36 2024-04-26T03:35:56.891 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 28/36 2024-04-26T03:35:56.927 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 29/36 2024-04-26T03:35:56.964 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-osd-2:19.0.0-2457.gddeaa917.el9.x86_64 33/36 2024-04-26T03:35:56.990 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-osd-2:19.0.0-2457.gddeaa917.el9.x86_64 33/36 2024-04-26T03:35:56.991 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T03:35:56.991 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T03:35:56.991 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T03:35:56.991 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T03:35:56.991 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:35:57.045 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-test-2:19.0.0-2457.gddeaa917.el9.x86_64 4/4 2024-04-26T03:35:57.045 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-test-2:19.0.0-2457.gddeaa917.el9.x86_64 1/4 2024-04-26T03:35:57.045 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libcephsqlite-2:19.0.0-2457.gddeaa917.el9.x86_64 2/4 2024-04-26T03:35:57.045 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-04-26T03:35:57.050 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 30/36 2024-04-26T03:35:57.215 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2457.gddeaa917.el9. 31/36 2024-04-26T03:35:57.308 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-04-26T03:35:57.308 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:35:57.308 INFO:teuthology.orchestra.run.smithi066.stdout:Installed: 2024-04-26T03:35:57.308 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-test-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:57.308 INFO:teuthology.orchestra.run.smithi066.stdout: libcephsqlite-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:57.308 INFO:teuthology.orchestra.run.smithi066.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T03:35:57.308 INFO:teuthology.orchestra.run.smithi066.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T03:35:57.309 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:35:57.309 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-04-26T03:35:57.362 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mgr-2:19.0.0-2457.gddeaa917.el9.x86_64 32/36 2024-04-26T03:35:57.389 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2457.gddeaa917.el9.x86_64 32/36 2024-04-26T03:35:57.390 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T03:35:57.390 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T03:35:57.390 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T03:35:57.390 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T03:35:57.390 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:35:57.436 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mon-2:19.0.0-2457.gddeaa917.el9.x86_64 34/36 2024-04-26T03:35:57.464 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mon-2:19.0.0-2457.gddeaa917.el9.x86_64 34/36 2024-04-26T03:35:57.464 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T03:35:57.464 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T03:35:57.464 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T03:35:57.465 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T03:35:57.465 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:35:57.578 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum -y install ceph 2024-04-26T03:35:57.677 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-04-26T03:35:57.677 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:35:57.677 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-04-26T03:35:57.678 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:57.678 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mds-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:57.678 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:57.678 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-modules-core-2:19.0.0-2457.gddeaa917.el9.noarch 2024-04-26T03:35:57.678 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mon-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:57.678 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-osd-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:35:57.678 INFO:teuthology.orchestra.run.smithi042.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T03:35:57.678 INFO:teuthology.orchestra.run.smithi042.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T03:35:57.678 INFO:teuthology.orchestra.run.smithi042.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T03:35:57.678 INFO:teuthology.orchestra.run.smithi042.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T03:35:57.678 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T03:35:57.678 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T03:35:57.678 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T03:35:57.679 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T03:35:57.679 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T03:35:57.679 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T03:35:57.679 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T03:35:57.679 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T03:35:57.679 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T03:35:57.679 INFO:teuthology.orchestra.run.smithi042.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T03:35:57.679 INFO:teuthology.orchestra.run.smithi042.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T03:35:57.679 INFO:teuthology.orchestra.run.smithi042.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T03:35:57.679 INFO:teuthology.orchestra.run.smithi042.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T03:35:57.679 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T03:35:57.679 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T03:35:57.679 INFO:teuthology.orchestra.run.smithi042.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T03:35:57.679 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T03:35:57.680 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T03:35:57.680 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T03:35:57.680 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T03:35:57.680 INFO:teuthology.orchestra.run.smithi042.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T03:35:57.680 INFO:teuthology.orchestra.run.smithi042.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T03:35:57.680 INFO:teuthology.orchestra.run.smithi042.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T03:35:57.680 INFO:teuthology.orchestra.run.smithi042.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T03:35:57.680 INFO:teuthology.orchestra.run.smithi042.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T03:35:57.680 INFO:teuthology.orchestra.run.smithi042.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T03:35:57.680 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:35:57.680 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-04-26T03:35:57.690 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mds-2:19.0.0-2457.gddeaa917.el9.x86_64 35/36 2024-04-26T03:35:57.713 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mds-2:19.0.0-2457.gddeaa917.el9.x86_64 35/36 2024-04-26T03:35:57.713 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T03:35:57.714 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T03:35:57.714 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T03:35:57.714 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T03:35:57.714 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:35:57.777 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-2:19.0.0-2457.gddeaa917.el9.x86_64 36/36 2024-04-26T03:35:58.022 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-base 2024-04-26T03:35:58.111 INFO:teuthology.orchestra.run.smithi066.stdout:Last metadata expiration check: 0:00:57 ago on Fri 26 Apr 2024 03:35:01 AM UTC. 2024-04-26T03:35:58.239 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-04-26T03:35:58.242 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:35:58.242 INFO:teuthology.orchestra.run.smithi066.stdout: Package Arch Version Repository Size 2024-04-26T03:35:58.242 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:35:58.242 INFO:teuthology.orchestra.run.smithi066.stdout:Installing: 2024-04-26T03:35:58.242 INFO:teuthology.orchestra.run.smithi066.stdout: ceph x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 6.6 k 2024-04-26T03:35:58.242 INFO:teuthology.orchestra.run.smithi066.stdout:Installing dependencies: 2024-04-26T03:35:58.242 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mds x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 2.4 M 2024-04-26T03:35:58.242 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 1.5 M 2024-04-26T03:35:58.243 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2457.gddeaa917.el9 ceph-noarch 249 k 2024-04-26T03:35:58.243 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mon x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 4.7 M 2024-04-26T03:35:58.243 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-osd x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 17 M 2024-04-26T03:35:58.243 INFO:teuthology.orchestra.run.smithi066.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T03:35:58.243 INFO:teuthology.orchestra.run.smithi066.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T03:35:58.243 INFO:teuthology.orchestra.run.smithi066.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T03:35:58.243 INFO:teuthology.orchestra.run.smithi066.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T03:35:58.243 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T03:35:58.243 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T03:35:58.243 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T03:35:58.243 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T03:35:58.243 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T03:35:58.243 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T03:35:58.244 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-collections 2024-04-26T03:35:58.244 INFO:teuthology.orchestra.run.smithi066.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-04-26T03:35:58.244 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T03:35:58.244 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T03:35:58.244 INFO:teuthology.orchestra.run.smithi066.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T03:35:58.244 INFO:teuthology.orchestra.run.smithi066.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T03:35:58.244 INFO:teuthology.orchestra.run.smithi066.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T03:35:58.244 INFO:teuthology.orchestra.run.smithi066.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T03:35:58.244 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T03:35:58.245 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T03:35:58.245 INFO:teuthology.orchestra.run.smithi066.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T03:35:58.245 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T03:35:58.245 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T03:35:58.245 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T03:35:58.245 INFO:teuthology.orchestra.run.smithi066.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T03:35:58.245 INFO:teuthology.orchestra.run.smithi066.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T03:35:58.245 INFO:teuthology.orchestra.run.smithi066.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T03:35:58.245 INFO:teuthology.orchestra.run.smithi066.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T03:35:58.245 INFO:teuthology.orchestra.run.smithi066.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T03:35:58.245 INFO:teuthology.orchestra.run.smithi066.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T03:35:58.245 INFO:teuthology.orchestra.run.smithi066.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T03:35:58.245 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:35:58.245 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction Summary 2024-04-26T03:35:58.246 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:35:58.246 INFO:teuthology.orchestra.run.smithi066.stdout:Install 36 Packages 2024-04-26T03:35:58.246 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:35:58.246 INFO:teuthology.orchestra.run.smithi066.stdout:Total download size: 30 M 2024-04-26T03:35:58.246 INFO:teuthology.orchestra.run.smithi066.stdout:Installed size: 107 M 2024-04-26T03:35:58.247 INFO:teuthology.orchestra.run.smithi066.stdout:Downloading Packages: 2024-04-26T03:35:58.553 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:00 ago on Fri 26 Apr 2024 03:34:58 AM UTC. 2024-04-26T03:35:58.633 INFO:teuthology.orchestra.run.smithi042.stdout:Package ceph-base-2:19.0.0-2457.gddeaa917.el9.x86_64 is already installed. 2024-04-26T03:35:58.677 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-04-26T03:35:58.678 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-04-26T03:35:58.679 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-04-26T03:35:58.742 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install cephadm 2024-04-26T03:35:58.830 INFO:teuthology.orchestra.run.smithi066.stdout:(1/36): ceph-19.0.0-2457.gddeaa917.el9.x86_64.r 34 kB/s | 6.6 kB 00:00 2024-04-26T03:35:58.904 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-osd-2:19.0.0-2457.gddeaa917.el9.x86_64 33/36 2024-04-26T03:35:58.932 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-osd-2:19.0.0-2457.gddeaa917.el9.x86_64 33/36 2024-04-26T03:35:58.932 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T03:35:58.932 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T03:35:58.932 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T03:35:58.932 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T03:35:58.933 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:35:59.064 INFO:teuthology.orchestra.run.smithi066.stdout:(2/36): ceph-mds-19.0.0-2457.gddeaa917.el9.x86_ 5.5 MB/s | 2.4 MB 00:00 2024-04-26T03:35:59.106 INFO:teuthology.orchestra.run.smithi066.stdout:(3/36): ceph-mgr-19.0.0-2457.gddeaa917.el9.x86_ 3.2 MB/s | 1.5 MB 00:00 2024-04-26T03:35:59.198 INFO:teuthology.orchestra.run.smithi066.stdout:(4/36): ceph-mgr-modules-core-19.0.0-2457.gddea 2.6 MB/s | 249 kB 00:00 2024-04-26T03:35:59.269 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:01 ago on Fri 26 Apr 2024 03:34:58 AM UTC. 2024-04-26T03:35:59.392 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-04-26T03:35:59.393 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:35:59.393 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-04-26T03:35:59.393 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:35:59.393 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-04-26T03:35:59.393 INFO:teuthology.orchestra.run.smithi042.stdout: cephadm noarch 2:19.0.0-2457.gddeaa917.el9 ceph-noarch 762 k 2024-04-26T03:35:59.393 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:35:59.393 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-04-26T03:35:59.393 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:35:59.393 INFO:teuthology.orchestra.run.smithi042.stdout:Install 1 Package 2024-04-26T03:35:59.393 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:35:59.393 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 762 k 2024-04-26T03:35:59.393 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 768 k 2024-04-26T03:35:59.394 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-04-26T03:35:59.397 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mon-2:19.0.0-2457.gddeaa917.el9.x86_64 34/36 2024-04-26T03:35:59.422 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mon-2:19.0.0-2457.gddeaa917.el9.x86_64 34/36 2024-04-26T03:35:59.422 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T03:35:59.422 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T03:35:59.422 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T03:35:59.422 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T03:35:59.422 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:35:59.490 INFO:teuthology.orchestra.run.smithi066.stdout:(5/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 869 kB/s | 253 kB 00:00 2024-04-26T03:35:59.557 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-2:19.0.0-2457.gddeaa917.el9.x86_64 36/36 2024-04-26T03:35:59.557 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-2:19.0.0-2457.gddeaa917.el9.x86_64 1/36 2024-04-26T03:35:59.558 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mds-2:19.0.0-2457.gddeaa917.el9.x86_64 2/36 2024-04-26T03:35:59.558 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-2:19.0.0-2457.gddeaa917.el9.x86_64 3/36 2024-04-26T03:35:59.558 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mon-2:19.0.0-2457.gddeaa917.el9.x86_64 4/36 2024-04-26T03:35:59.558 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-osd-2:19.0.0-2457.gddeaa917.el9.x86_64 5/36 2024-04-26T03:35:59.558 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2457.gddeaa917.el9. 6/36 2024-04-26T03:35:59.558 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/36 2024-04-26T03:35:59.558 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/36 2024-04-26T03:35:59.558 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/36 2024-04-26T03:35:59.558 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/36 2024-04-26T03:35:59.558 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/36 2024-04-26T03:35:59.558 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/36 2024-04-26T03:35:59.558 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/36 2024-04-26T03:35:59.559 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/36 2024-04-26T03:35:59.559 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/36 2024-04-26T03:35:59.559 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/36 2024-04-26T03:35:59.559 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/36 2024-04-26T03:35:59.559 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-04-26T03:35:59.559 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-04-26T03:35:59.559 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-04-26T03:35:59.559 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-04-26T03:35:59.559 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-04-26T03:35:59.559 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-04-26T03:35:59.559 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-04-26T03:35:59.559 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-04-26T03:35:59.561 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-04-26T03:35:59.561 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-04-26T03:35:59.561 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-04-26T03:35:59.561 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-04-26T03:35:59.561 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-04-26T03:35:59.561 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-04-26T03:35:59.561 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-04-26T03:35:59.561 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-04-26T03:35:59.561 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-04-26T03:35:59.561 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-04-26T03:35:59.590 INFO:teuthology.orchestra.run.smithi066.stdout:(6/36): python3-cryptography-36.0.1-4.el9.x86_6 12 MB/s | 1.2 MB 00:00 2024-04-26T03:35:59.632 INFO:teuthology.orchestra.run.smithi066.stdout:(7/36): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-04-26T03:35:59.664 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mds-2:19.0.0-2457.gddeaa917.el9.x86_64 35/36 2024-04-26T03:35:59.674 INFO:teuthology.orchestra.run.smithi066.stdout:(8/36): python3-pycparser-2.20-6.el9.noarch.rpm 3.2 MB/s | 135 kB 00:00 2024-04-26T03:35:59.688 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mds-2:19.0.0-2457.gddeaa917.el9.x86_64 35/36 2024-04-26T03:35:59.689 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T03:35:59.689 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T03:35:59.689 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T03:35:59.689 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T03:35:59.689 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:35:59.716 INFO:teuthology.orchestra.run.smithi066.stdout:(9/36): python3-requests-2.25.1-8.el9.noarch.rp 2.9 MB/s | 125 kB 00:00 2024-04-26T03:35:59.754 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-2:19.0.0-2457.gddeaa917.el9.x86_64 36/36 2024-04-26T03:35:59.758 INFO:teuthology.orchestra.run.smithi066.stdout:(10/36): python3-urllib3-1.26.5-5.el9.noarch.rp 5.0 MB/s | 215 kB 00:00 2024-04-26T03:35:59.873 INFO:teuthology.orchestra.run.smithi042.stdout:cephadm-19.0.0-2457.gddeaa917.el9.noarch.rpm 1.6 MB/s | 762 kB 00:00 2024-04-26T03:35:59.874 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:35:59.874 INFO:teuthology.orchestra.run.smithi042.stdout:Total 1.5 MB/s | 762 kB 00:00 2024-04-26T03:35:59.874 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-04-26T03:35:59.880 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-04-26T03:35:59.880 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-04-26T03:35:59.886 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-04-26T03:35:59.886 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-04-26T03:35:59.933 INFO:teuthology.orchestra.run.smithi066.stdout:(11/36): ceph-osd-19.0.0-2457.gddeaa917.el9.x86 19 MB/s | 17 MB 00:00 2024-04-26T03:35:59.958 INFO:teuthology.orchestra.run.smithi066.stdout:(12/36): lua-5.4.4-4.el9.x86_64.rpm 942 kB/s | 188 kB 00:00 2024-04-26T03:35:59.961 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-04-26T03:35:59.992 INFO:teuthology.orchestra.run.smithi066.stdout:(13/36): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-26T03:36:00.017 INFO:teuthology.orchestra.run.smithi066.stdout:(14/36): python3-markupsafe-1.1.1-12.el9.x86_64 593 kB/s | 35 kB 00:00 2024-04-26T03:36:00.043 INFO:teuthology.orchestra.run.smithi066.stdout:(15/36): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-26T03:36:00.101 INFO:teuthology.orchestra.run.smithi066.stdout:(16/36): ceph-mon-19.0.0-2457.gddeaa917.el9.x86 3.7 MB/s | 4.7 MB 00:01 2024-04-26T03:36:00.126 INFO:teuthology.orchestra.run.smithi066.stdout:(17/36): python3-toml-0.10.2-6.el9.noarch.rpm 383 kB/s | 42 kB 00:00 2024-04-26T03:36:00.152 INFO:teuthology.orchestra.run.smithi066.stdout:(18/36): lua-devel-5.4.4-4.el9.x86_64.rpm 205 kB/s | 22 kB 00:00 2024-04-26T03:36:00.431 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: cephadm-2:19.0.0-2457.gddeaa917.el9.noarch 1/1 2024-04-26T03:36:00.442 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-04-26T03:36:00.443 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:00.443 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-04-26T03:36:00.443 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:36:00.443 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:36:00.443 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:36:00.443 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core-2:19.0.0-2457.gddeaa917.el9.noarch 2024-04-26T03:36:00.443 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:36:00.443 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:36:00.443 INFO:teuthology.orchestra.run.smithi092.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T03:36:00.443 INFO:teuthology.orchestra.run.smithi092.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T03:36:00.443 INFO:teuthology.orchestra.run.smithi092.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T03:36:00.443 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T03:36:00.444 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T03:36:00.444 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T03:36:00.444 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T03:36:00.444 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T03:36:00.444 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T03:36:00.444 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T03:36:00.444 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T03:36:00.444 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T03:36:00.444 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T03:36:00.444 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T03:36:00.444 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T03:36:00.444 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T03:36:00.445 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T03:36:00.445 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T03:36:00.445 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T03:36:00.445 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T03:36:00.445 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T03:36:00.445 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T03:36:00.445 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T03:36:00.445 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T03:36:00.445 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T03:36:00.445 INFO:teuthology.orchestra.run.smithi092.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T03:36:00.445 INFO:teuthology.orchestra.run.smithi092.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T03:36:00.445 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T03:36:00.446 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T03:36:00.446 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T03:36:00.446 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:00.446 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-26T03:36:00.512 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : cephadm-2:19.0.0-2457.gddeaa917.el9.noarch 1/1 2024-04-26T03:36:00.552 INFO:teuthology.orchestra.run.smithi066.stdout:(19/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 109 kB/s | 43 kB 00:00 2024-04-26T03:36:00.727 INFO:teuthology.orchestra.run.smithi066.stdout:(20/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 299 kB/s | 172 kB 00:00 2024-04-26T03:36:00.752 INFO:teuthology.orchestra.run.smithi066.stdout:(21/36): luarocks-3.9.2-1.el9.noarch.rpm 232 kB/s | 151 kB 00:00 2024-04-26T03:36:00.813 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-base 2024-04-26T03:36:00.836 INFO:teuthology.orchestra.run.smithi066.stdout:(22/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 99 kB/s | 11 kB 00:00 2024-04-26T03:36:00.861 INFO:teuthology.orchestra.run.smithi066.stdout:(23/36): python3-jaraco-classes-3.2.1-5.el9.noa 163 kB/s | 18 kB 00:00 2024-04-26T03:36:00.882 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: cephadm-2:19.0.0-2457.gddeaa917.el9.noarch 1/1 2024-04-26T03:36:00.903 INFO:teuthology.orchestra.run.smithi066.stdout:(24/36): python3-cherrypy-18.6.1-2.el9.noarch.r 1.0 MB/s | 358 kB 00:00 2024-04-26T03:36:00.937 INFO:teuthology.orchestra.run.smithi066.stdout:(25/36): python3-jaraco-collections-3.0.0-8.el9 230 kB/s | 23 kB 00:00 2024-04-26T03:36:00.962 INFO:teuthology.orchestra.run.smithi066.stdout:(26/36): python3-jaraco-functools-3.5.0-2.el9.n 193 kB/s | 19 kB 00:00 2024-04-26T03:36:00.995 INFO:teuthology.orchestra.run.smithi066.stdout:(27/36): python3-jaraco-text-3.2.0-6.el9.noarch 213 kB/s | 20 kB 00:00 2024-04-26T03:36:01.046 INFO:teuthology.orchestra.run.smithi066.stdout:(28/36): python3-logutils-0.3.5-21.el9.noarch.r 426 kB/s | 46 kB 00:00 2024-04-26T03:36:01.079 INFO:teuthology.orchestra.run.smithi066.stdout:(29/36): python3-more-itertools-8.12.0-2.el9.no 674 kB/s | 79 kB 00:00 2024-04-26T03:36:01.145 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : cephadm-2:19.0.0-2457.gddeaa917.el9.noarch 1/1 2024-04-26T03:36:01.145 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:01.145 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-04-26T03:36:01.145 INFO:teuthology.orchestra.run.smithi042.stdout: cephadm-2:19.0.0-2457.gddeaa917.el9.noarch 2024-04-26T03:36:01.145 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:01.145 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-04-26T03:36:01.145 INFO:teuthology.orchestra.run.smithi066.stdout:(30/36): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-04-26T03:36:01.155 INFO:teuthology.orchestra.run.smithi066.stdout:(31/36): python3-portend-3.1.0-2.el9.noarch.rpm 151 kB/s | 16 kB 00:00 2024-04-26T03:36:01.205 INFO:teuthology.orchestra.run.smithi066.stdout:(32/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 716 kB/s | 90 kB 00:00 2024-04-26T03:36:01.240 INFO:teuthology.orchestra.run.smithi066.stdout:(33/36): python3-tempora-5.0.0-2.el9.noarch.rpm 354 kB/s | 36 kB 00:00 2024-04-26T03:36:01.276 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-2:19.0.0-2457.gddeaa917.el9.x86_64 36/36 2024-04-26T03:36:01.276 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-2:19.0.0-2457.gddeaa917.el9.x86_64 1/36 2024-04-26T03:36:01.276 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mds-2:19.0.0-2457.gddeaa917.el9.x86_64 2/36 2024-04-26T03:36:01.276 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mgr-2:19.0.0-2457.gddeaa917.el9.x86_64 3/36 2024-04-26T03:36:01.276 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mon-2:19.0.0-2457.gddeaa917.el9.x86_64 4/36 2024-04-26T03:36:01.276 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-osd-2:19.0.0-2457.gddeaa917.el9.x86_64 5/36 2024-04-26T03:36:01.276 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2457.gddeaa917.el9. 6/36 2024-04-26T03:36:01.276 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/36 2024-04-26T03:36:01.276 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/36 2024-04-26T03:36:01.277 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/36 2024-04-26T03:36:01.277 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/36 2024-04-26T03:36:01.277 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/36 2024-04-26T03:36:01.277 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/36 2024-04-26T03:36:01.277 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/36 2024-04-26T03:36:01.277 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/36 2024-04-26T03:36:01.277 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/36 2024-04-26T03:36:01.277 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/36 2024-04-26T03:36:01.277 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/36 2024-04-26T03:36:01.277 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-04-26T03:36:01.277 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-04-26T03:36:01.277 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-04-26T03:36:01.277 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-04-26T03:36:01.277 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-04-26T03:36:01.278 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-04-26T03:36:01.278 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-04-26T03:36:01.278 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-04-26T03:36:01.278 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-04-26T03:36:01.278 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-04-26T03:36:01.279 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-04-26T03:36:01.279 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-04-26T03:36:01.279 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-04-26T03:36:01.279 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-04-26T03:36:01.279 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-04-26T03:36:01.279 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-04-26T03:36:01.279 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-04-26T03:36:01.279 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-04-26T03:36:01.318 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-immutable-object-cache 2024-04-26T03:36:01.322 INFO:teuthology.orchestra.run.smithi066.stdout:(34/36): python3-webob-1.8.7-6.el9.noarch.rpm 1.3 MB/s | 230 kB 00:00 2024-04-26T03:36:01.325 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:02 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:36:01.339 INFO:teuthology.orchestra.run.smithi066.stdout:(35/36): python3-zc-lockfile-2.0-10.el9.noarch. 201 kB/s | 20 kB 00:00 2024-04-26T03:36:01.403 INFO:teuthology.orchestra.run.smithi092.stdout:Package ceph-base-2:19.0.0-2457.gddeaa917.el9.x86_64 is already installed. 2024-04-26T03:36:01.444 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-26T03:36:01.445 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-04-26T03:36:01.445 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-26T03:36:01.473 INFO:teuthology.orchestra.run.smithi066.stdout:(36/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 1.6 MB/s | 427 kB 00:00 2024-04-26T03:36:01.480 INFO:teuthology.orchestra.run.smithi066.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:01.480 INFO:teuthology.orchestra.run.smithi066.stdout:Total 9.3 MB/s | 30 MB 00:03 2024-04-26T03:36:01.515 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install cephadm 2024-04-26T03:36:01.624 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction check 2024-04-26T03:36:01.668 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction check succeeded. 2024-04-26T03:36:01.668 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction test 2024-04-26T03:36:01.853 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:03 ago on Fri 26 Apr 2024 03:34:58 AM UTC. 2024-04-26T03:36:01.979 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction test succeeded. 2024-04-26T03:36:01.979 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-04-26T03:36:01.980 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:36:01.980 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repo Size 2024-04-26T03:36:01.980 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:36:01.980 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-04-26T03:36:01.980 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 145 k 2024-04-26T03:36:01.980 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:01.980 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-04-26T03:36:01.980 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:36:01.980 INFO:teuthology.orchestra.run.smithi042.stdout:Install 1 Package 2024-04-26T03:36:01.980 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:01.981 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction 2024-04-26T03:36:01.981 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 145 k 2024-04-26T03:36:01.982 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 435 k 2024-04-26T03:36:01.982 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-04-26T03:36:02.013 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:03 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:36:02.134 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-26T03:36:02.135 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:36:02.135 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-04-26T03:36:02.135 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:36:02.135 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-04-26T03:36:02.135 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm noarch 2:19.0.0-2457.gddeaa917.el9 ceph-noarch 762 k 2024-04-26T03:36:02.135 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:02.135 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-04-26T03:36:02.135 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:36:02.135 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-04-26T03:36:02.135 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:02.136 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 762 k 2024-04-26T03:36:02.136 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 768 k 2024-04-26T03:36:02.136 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-04-26T03:36:02.192 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-04-26T03:36:02.192 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:02.192 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-04-26T03:36:02.192 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:36:02.192 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mds-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:36:02.192 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:36:02.193 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-modules-core-2:19.0.0-2457.gddeaa917.el9.noarch 2024-04-26T03:36:02.193 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mon-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:36:02.193 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-osd-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:36:02.193 INFO:teuthology.orchestra.run.smithi028.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T03:36:02.193 INFO:teuthology.orchestra.run.smithi028.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T03:36:02.193 INFO:teuthology.orchestra.run.smithi028.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T03:36:02.193 INFO:teuthology.orchestra.run.smithi028.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T03:36:02.193 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T03:36:02.193 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T03:36:02.193 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T03:36:02.193 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T03:36:02.193 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T03:36:02.194 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T03:36:02.194 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T03:36:02.194 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T03:36:02.194 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T03:36:02.194 INFO:teuthology.orchestra.run.smithi028.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T03:36:02.194 INFO:teuthology.orchestra.run.smithi028.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T03:36:02.194 INFO:teuthology.orchestra.run.smithi028.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T03:36:02.194 INFO:teuthology.orchestra.run.smithi028.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T03:36:02.194 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T03:36:02.194 INFO:teuthology.orchestra.run.smithi028.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T03:36:02.194 INFO:teuthology.orchestra.run.smithi028.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T03:36:02.194 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T03:36:02.195 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T03:36:02.195 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T03:36:02.195 INFO:teuthology.orchestra.run.smithi028.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T03:36:02.195 INFO:teuthology.orchestra.run.smithi028.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T03:36:02.195 INFO:teuthology.orchestra.run.smithi028.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T03:36:02.195 INFO:teuthology.orchestra.run.smithi028.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T03:36:02.195 INFO:teuthology.orchestra.run.smithi028.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T03:36:02.195 INFO:teuthology.orchestra.run.smithi028.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T03:36:02.195 INFO:teuthology.orchestra.run.smithi028.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T03:36:02.195 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:02.195 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-04-26T03:36:02.205 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-immutable-object-cache-19.0.0-2457.gddeaa9 647 kB/s | 145 kB 00:00 2024-04-26T03:36:02.206 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:02.206 INFO:teuthology.orchestra.run.smithi042.stdout:Total 641 kB/s | 145 kB 00:00 2024-04-26T03:36:02.206 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-04-26T03:36:02.214 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-04-26T03:36:02.215 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-04-26T03:36:02.274 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-04-26T03:36:02.275 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-04-26T03:36:02.415 INFO:teuthology.orchestra.run.smithi092.stdout:cephadm-19.0.0-2457.gddeaa917.el9.noarch.rpm 2.7 MB/s | 762 kB 00:00 2024-04-26T03:36:02.416 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:02.416 INFO:teuthology.orchestra.run.smithi092.stdout:Total 2.6 MB/s | 762 kB 00:00 2024-04-26T03:36:02.416 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-04-26T03:36:02.421 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-04-26T03:36:02.421 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-04-26T03:36:02.427 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-04-26T03:36:02.427 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-04-26T03:36:02.434 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-04-26T03:36:02.500 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-04-26T03:36:02.506 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2457.gddeaa917. 1/1 2024-04-26T03:36:02.522 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install ceph-base 2024-04-26T03:36:02.532 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2457.gddeaa917. 1/1 2024-04-26T03:36:02.532 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T03:36:02.532 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T03:36:02.532 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:02.961 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephadm-2:19.0.0-2457.gddeaa917.el9.noarch 1/1 2024-04-26T03:36:03.025 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephadm-2:19.0.0-2457.gddeaa917.el9.noarch 1/1 2024-04-26T03:36:03.050 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:04 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:36:03.130 INFO:teuthology.orchestra.run.smithi028.stdout:Package ceph-base-2:19.0.0-2457.gddeaa917.el9.x86_64 is already installed. 2024-04-26T03:36:03.174 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-04-26T03:36:03.175 INFO:teuthology.orchestra.run.smithi028.stdout:Nothing to do. 2024-04-26T03:36:03.175 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-04-26T03:36:03.248 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install cephadm 2024-04-26T03:36:03.389 INFO:teuthology.orchestra.run.smithi066.stdout: Preparing : 1/1 2024-04-26T03:36:03.418 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephadm-2:19.0.0-2457.gddeaa917.el9.noarch 1/1 2024-04-26T03:36:03.432 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2457.gddeaa917. 1/1 2024-04-26T03:36:03.433 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:03.433 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-04-26T03:36:03.433 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-immutable-object-cache-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:36:03.433 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:03.433 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-04-26T03:36:03.451 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-04-26T03:36:03.484 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-04-26T03:36:03.512 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-04-26T03:36:03.588 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-04-26T03:36:03.636 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-04-26T03:36:03.637 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-mgr 2024-04-26T03:36:03.667 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephadm-2:19.0.0-2457.gddeaa917.el9.noarch 1/1 2024-04-26T03:36:03.668 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:03.668 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-04-26T03:36:03.668 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm-2:19.0.0-2457.gddeaa917.el9.noarch 2024-04-26T03:36:03.668 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:03.668 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-26T03:36:03.669 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-04-26T03:36:03.765 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-04-26T03:36:03.772 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:04 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:36:03.860 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-immutable-object-cache 2024-04-26T03:36:03.896 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-04-26T03:36:03.897 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:36:03.897 INFO:teuthology.orchestra.run.smithi028.stdout: Package Arch Version Repository Size 2024-04-26T03:36:03.897 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:36:03.897 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-04-26T03:36:03.898 INFO:teuthology.orchestra.run.smithi028.stdout: cephadm noarch 2:19.0.0-2457.gddeaa917.el9 ceph-noarch 762 k 2024-04-26T03:36:03.898 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:03.898 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-04-26T03:36:03.898 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:36:03.898 INFO:teuthology.orchestra.run.smithi028.stdout:Install 1 Package 2024-04-26T03:36:03.898 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:03.898 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 762 k 2024-04-26T03:36:03.898 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 768 k 2024-04-26T03:36:03.898 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-04-26T03:36:03.958 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-04-26T03:36:04.005 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-04-26T03:36:04.039 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-04-26T03:36:04.069 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/36 2024-04-26T03:36:04.103 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/36 2024-04-26T03:36:04.137 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/36 2024-04-26T03:36:04.162 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:06 ago on Fri 26 Apr 2024 03:34:58 AM UTC. 2024-04-26T03:36:04.178 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/36 2024-04-26T03:36:04.188 INFO:teuthology.orchestra.run.smithi028.stdout:cephadm-19.0.0-2457.gddeaa917.el9.noarch.rpm 2.6 MB/s | 762 kB 00:00 2024-04-26T03:36:04.189 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:04.189 INFO:teuthology.orchestra.run.smithi028.stdout:Total 2.6 MB/s | 762 kB 00:00 2024-04-26T03:36:04.189 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-04-26T03:36:04.195 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-04-26T03:36:04.195 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-04-26T03:36:04.201 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-04-26T03:36:04.201 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-04-26T03:36:04.230 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/36 2024-04-26T03:36:04.243 INFO:teuthology.orchestra.run.smithi042.stdout:Package ceph-mgr-2:19.0.0-2457.gddeaa917.el9.x86_64 is already installed. 2024-04-26T03:36:04.263 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/36 2024-04-26T03:36:04.275 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-04-26T03:36:04.286 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-04-26T03:36:04.287 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-04-26T03:36:04.288 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-04-26T03:36:04.294 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/36 2024-04-26T03:36:04.350 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-mgr-dashboard 2024-04-26T03:36:04.379 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:05 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:36:04.379 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 18/36 2024-04-26T03:36:04.503 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-26T03:36:04.503 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:36:04.503 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repo Size 2024-04-26T03:36:04.503 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:36:04.503 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-04-26T03:36:04.503 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 145 k 2024-04-26T03:36:04.503 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:04.503 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-04-26T03:36:04.503 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:36:04.504 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-04-26T03:36:04.504 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:04.504 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 145 k 2024-04-26T03:36:04.504 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 435 k 2024-04-26T03:36:04.504 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-04-26T03:36:04.532 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 19/36 2024-04-26T03:36:04.614 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 20/36 2024-04-26T03:36:04.672 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 21/36 2024-04-26T03:36:04.710 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 22/36 2024-04-26T03:36:04.768 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-ply-3.11-14.el9.noarch 23/36 2024-04-26T03:36:04.790 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: cephadm-2:19.0.0-2457.gddeaa917.el9.noarch 1/1 2024-04-26T03:36:04.805 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-immutable-object-cache-19.0.0-2457.gddeaa9 479 kB/s | 145 kB 00:00 2024-04-26T03:36:04.805 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:04.806 INFO:teuthology.orchestra.run.smithi092.stdout:Total 476 kB/s | 145 kB 00:00 2024-04-26T03:36:04.806 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-04-26T03:36:04.814 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-04-26T03:36:04.814 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-04-26T03:36:04.832 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 24/36 2024-04-26T03:36:04.871 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-04-26T03:36:04.871 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-04-26T03:36:04.871 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:06 ago on Fri 26 Apr 2024 03:34:58 AM UTC. 2024-04-26T03:36:04.876 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : cephadm-2:19.0.0-2457.gddeaa917.el9.noarch 1/1 2024-04-26T03:36:04.998 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-04-26T03:36:04.999 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:36:05.000 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-04-26T03:36:05.000 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:36:05.000 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-04-26T03:36:05.000 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2457.gddeaa917.el9 ceph-noarch 3.5 M 2024-04-26T03:36:05.000 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-04-26T03:36:05.000 INFO:teuthology.orchestra.run.smithi042.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T03:36:05.000 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2457.gddeaa917.el9 ceph-noarch 32 k 2024-04-26T03:36:05.000 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2457.gddeaa917.el9 ceph-noarch 15 k 2024-04-26T03:36:05.000 INFO:teuthology.orchestra.run.smithi042.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T03:36:05.000 INFO:teuthology.orchestra.run.smithi042.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T03:36:05.001 INFO:teuthology.orchestra.run.smithi042.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T03:36:05.001 INFO:teuthology.orchestra.run.smithi042.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T03:36:05.001 INFO:teuthology.orchestra.run.smithi042.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T03:36:05.001 INFO:teuthology.orchestra.run.smithi042.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T03:36:05.001 INFO:teuthology.orchestra.run.smithi042.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T03:36:05.001 INFO:teuthology.orchestra.run.smithi042.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T03:36:05.001 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:05.001 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-04-26T03:36:05.001 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:36:05.001 INFO:teuthology.orchestra.run.smithi042.stdout:Install 12 Packages 2024-04-26T03:36:05.001 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:05.002 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 8.6 M 2024-04-26T03:36:05.002 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 103 M 2024-04-26T03:36:05.002 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-04-26T03:36:05.019 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-04-26T03:36:05.030 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 25/36 2024-04-26T03:36:05.080 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 26/36 2024-04-26T03:36:05.106 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2457.gddeaa917. 1/1 2024-04-26T03:36:05.131 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2457.gddeaa917. 1/1 2024-04-26T03:36:05.131 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T03:36:05.132 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T03:36:05.132 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:05.161 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 27/36 2024-04-26T03:36:05.211 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: cephadm-2:19.0.0-2457.gddeaa917.el9.noarch 1/1 2024-04-26T03:36:05.252 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 28/36 2024-04-26T03:36:05.287 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 29/36 2024-04-26T03:36:05.420 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 30/36 2024-04-26T03:36:05.474 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : cephadm-2:19.0.0-2457.gddeaa917.el9.noarch 1/1 2024-04-26T03:36:05.475 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:05.475 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-04-26T03:36:05.475 INFO:teuthology.orchestra.run.smithi028.stdout: cephadm-2:19.0.0-2457.gddeaa917.el9.noarch 2024-04-26T03:36:05.475 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:05.475 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-04-26T03:36:05.584 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2457.gddeaa917.el9. 31/36 2024-04-26T03:36:05.614 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-2:19.0.0-2457.gddeaa917.el9.x86_64 32/36 2024-04-26T03:36:05.641 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2457.gddeaa917.el9.x86_64 32/36 2024-04-26T03:36:05.641 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T03:36:05.641 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T03:36:05.642 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T03:36:05.642 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T03:36:05.642 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:05.677 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install ceph-immutable-object-cache 2024-04-26T03:36:05.984 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2457.gddeaa917. 1/1 2024-04-26T03:36:05.984 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:05.984 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-04-26T03:36:05.984 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:36:05.984 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:05.985 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-26T03:36:06.118 INFO:teuthology.orchestra.run.smithi042.stdout:(1/12): ceph-prometheus-alerts-19.0.0-2457.gdde 76 kB/s | 15 kB 00:00 2024-04-26T03:36:06.177 INFO:teuthology.orchestra.run.smithi042.stdout:(2/12): ceph-grafana-dashboards-19.0.0-2457.gdd 121 kB/s | 32 kB 00:00 2024-04-26T03:36:06.199 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-mgr 2024-04-26T03:36:06.217 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:07 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:36:06.261 INFO:teuthology.orchestra.run.smithi042.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 3.2 MB/s | 269 kB 00:00 2024-04-26T03:36:06.302 INFO:teuthology.orchestra.run.smithi042.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 5.5 MB/s | 1.0 MB 00:00 2024-04-26T03:36:06.344 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-04-26T03:36:06.344 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:36:06.344 INFO:teuthology.orchestra.run.smithi028.stdout: Package Arch Version Repo Size 2024-04-26T03:36:06.345 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:36:06.345 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-04-26T03:36:06.345 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 145 k 2024-04-26T03:36:06.345 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:06.345 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-04-26T03:36:06.345 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:36:06.345 INFO:teuthology.orchestra.run.smithi028.stdout:Install 1 Package 2024-04-26T03:36:06.345 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:06.345 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 145 k 2024-04-26T03:36:06.345 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 435 k 2024-04-26T03:36:06.345 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-04-26T03:36:06.346 INFO:teuthology.orchestra.run.smithi042.stdout:(5/12): protobuf-compiler-3.14.0-13.el9.x86_64. 10 MB/s | 863 kB 00:00 2024-04-26T03:36:06.611 INFO:teuthology.orchestra.run.smithi042.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 73 kB/s | 19 kB 00:00 2024-04-26T03:36:06.707 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:07 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:36:06.786 INFO:teuthology.orchestra.run.smithi092.stdout:Package ceph-mgr-2:19.0.0-2457.gddeaa917.el9.x86_64 is already installed. 2024-04-26T03:36:06.829 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-26T03:36:06.830 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-04-26T03:36:06.830 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-26T03:36:06.877 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-immutable-object-cache-19.0.0-2457.gddeaa9 272 kB/s | 145 kB 00:00 2024-04-26T03:36:06.878 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:06.878 INFO:teuthology.orchestra.run.smithi028.stdout:Total 271 kB/s | 145 kB 00:00 2024-04-26T03:36:06.878 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-04-26T03:36:06.887 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-04-26T03:36:06.887 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-04-26T03:36:06.896 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-mgr-dashboard 2024-04-26T03:36:06.949 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-04-26T03:36:06.949 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-04-26T03:36:06.953 INFO:teuthology.orchestra.run.smithi042.stdout:(7/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 844 kB/s | 548 kB 00:00 2024-04-26T03:36:07.070 INFO:teuthology.orchestra.run.smithi042.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 1.2 MB/s | 144 kB 00:00 2024-04-26T03:36:07.106 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-04-26T03:36:07.157 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-osd-2:19.0.0-2457.gddeaa917.el9.x86_64 33/36 2024-04-26T03:36:07.170 INFO:teuthology.orchestra.run.smithi042.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 308 kB/s | 31 kB 00:00 2024-04-26T03:36:07.179 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2457.gddeaa917. 1/1 2024-04-26T03:36:07.185 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-osd-2:19.0.0-2457.gddeaa917.el9.x86_64 33/36 2024-04-26T03:36:07.185 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T03:36:07.185 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T03:36:07.185 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T03:36:07.185 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T03:36:07.185 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:07.204 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2457.gddeaa917. 1/1 2024-04-26T03:36:07.204 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T03:36:07.205 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T03:36:07.205 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:07.279 INFO:teuthology.orchestra.run.smithi042.stdout:(10/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 3.1 MB/s | 2.0 MB 00:00 2024-04-26T03:36:07.313 INFO:teuthology.orchestra.run.smithi042.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 1.3 MB/s | 188 kB 00:00 2024-04-26T03:36:07.398 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:08 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:36:07.496 INFO:teuthology.orchestra.run.smithi042.stdout:(12/12): ceph-mgr-dashboard-19.0.0-2457.gddeaa9 2.2 MB/s | 3.5 MB 00:01 2024-04-26T03:36:07.500 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:07.500 INFO:teuthology.orchestra.run.smithi042.stdout:Total 3.4 MB/s | 8.6 MB 00:02 2024-04-26T03:36:07.517 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-26T03:36:07.518 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:36:07.519 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-04-26T03:36:07.519 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:36:07.519 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-04-26T03:36:07.519 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2457.gddeaa917.el9 ceph-noarch 3.5 M 2024-04-26T03:36:07.519 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-04-26T03:36:07.519 INFO:teuthology.orchestra.run.smithi092.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T03:36:07.519 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2457.gddeaa917.el9 ceph-noarch 32 k 2024-04-26T03:36:07.519 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2457.gddeaa917.el9 ceph-noarch 15 k 2024-04-26T03:36:07.519 INFO:teuthology.orchestra.run.smithi092.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T03:36:07.520 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T03:36:07.520 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T03:36:07.520 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T03:36:07.520 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T03:36:07.520 INFO:teuthology.orchestra.run.smithi092.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T03:36:07.520 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T03:36:07.520 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T03:36:07.521 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:07.521 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-04-26T03:36:07.521 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:36:07.521 INFO:teuthology.orchestra.run.smithi092.stdout:Install 12 Packages 2024-04-26T03:36:07.521 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:07.522 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 8.6 M 2024-04-26T03:36:07.523 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 103 M 2024-04-26T03:36:07.523 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-04-26T03:36:07.553 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-04-26T03:36:07.574 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-04-26T03:36:07.574 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-04-26T03:36:07.649 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mon-2:19.0.0-2457.gddeaa917.el9.x86_64 34/36 2024-04-26T03:36:07.673 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mon-2:19.0.0-2457.gddeaa917.el9.x86_64 34/36 2024-04-26T03:36:07.673 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T03:36:07.674 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T03:36:07.674 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T03:36:07.674 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T03:36:07.674 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:07.741 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-04-26T03:36:07.742 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-04-26T03:36:07.908 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mds-2:19.0.0-2457.gddeaa917.el9.x86_64 35/36 2024-04-26T03:36:07.933 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mds-2:19.0.0-2457.gddeaa917.el9.x86_64 35/36 2024-04-26T03:36:07.933 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T03:36:07.933 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T03:36:07.933 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T03:36:07.933 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T03:36:07.933 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:07.998 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-2:19.0.0-2457.gddeaa917.el9.x86_64 36/36 2024-04-26T03:36:08.129 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2457.gddeaa917. 1/1 2024-04-26T03:36:08.129 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:08.129 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-04-26T03:36:08.130 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-immutable-object-cache-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:36:08.130 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:08.130 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-04-26T03:36:08.279 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-04-26T03:36:08.325 INFO:teuthology.orchestra.run.smithi092.stdout:(1/12): ceph-prometheus-alerts-19.0.0-2457.gdde 79 kB/s | 15 kB 00:00 2024-04-26T03:36:08.356 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-04-26T03:36:08.359 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install ceph-mgr 2024-04-26T03:36:08.375 INFO:teuthology.orchestra.run.smithi092.stdout:(2/12): ceph-grafana-dashboards-19.0.0-2457.gdd 130 kB/s | 32 kB 00:00 2024-04-26T03:36:08.400 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-04-26T03:36:08.446 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-04-26T03:36:08.476 INFO:teuthology.orchestra.run.smithi092.stdout:(3/12): protobuf-3.14.0-13.el9.x86_64.rpm 6.7 MB/s | 1.0 MB 00:00 2024-04-26T03:36:08.484 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-04-26T03:36:08.492 INFO:teuthology.orchestra.run.smithi092.stdout:(4/12): python3-protobuf-3.14.0-13.el9.noarch.r 2.3 MB/s | 269 kB 00:00 2024-04-26T03:36:08.526 INFO:teuthology.orchestra.run.smithi092.stdout:(5/12): protobuf-compiler-3.14.0-13.el9.x86_64. 17 MB/s | 863 kB 00:00 2024-04-26T03:36:08.564 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-04-26T03:36:08.699 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-04-26T03:36:08.776 INFO:teuthology.orchestra.run.smithi092.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 78 kB/s | 19 kB 00:00 2024-04-26T03:36:08.791 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-04-26T03:36:08.841 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-04-26T03:36:08.876 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-04-26T03:36:08.889 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:09 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:36:08.909 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2457.gddeaa917.el9 10/12 2024-04-26T03:36:08.972 INFO:teuthology.orchestra.run.smithi028.stdout:Package ceph-mgr-2:19.0.0-2457.gddeaa917.el9.x86_64 is already installed. 2024-04-26T03:36:09.015 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-04-26T03:36:09.016 INFO:teuthology.orchestra.run.smithi028.stdout:Nothing to do. 2024-04-26T03:36:09.017 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-04-26T03:36:09.086 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install ceph-mgr-dashboard 2024-04-26T03:36:09.135 INFO:teuthology.orchestra.run.smithi092.stdout:(7/12): ceph-mgr-dashboard-19.0.0-2457.gddeaa91 3.5 MB/s | 3.5 MB 00:01 2024-04-26T03:36:09.160 INFO:teuthology.orchestra.run.smithi092.stdout:(8/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 823 kB/s | 548 kB 00:00 2024-04-26T03:36:09.260 INFO:teuthology.orchestra.run.smithi092.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 308 kB/s | 31 kB 00:00 2024-04-26T03:36:09.394 INFO:teuthology.orchestra.run.smithi092.stdout:(10/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 3.3 MB/s | 2.0 MB 00:00 2024-04-26T03:36:09.419 INFO:teuthology.orchestra.run.smithi092.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 1.2 MB/s | 188 kB 00:00 2024-04-26T03:36:09.586 INFO:teuthology.orchestra.run.smithi092.stdout:(12/12): python3-grpcio-tools-1.46.7-10.el9.x86 320 kB/s | 144 kB 00:00 2024-04-26T03:36:09.588 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:09.589 INFO:teuthology.orchestra.run.smithi092.stdout:Total 4.2 MB/s | 8.6 MB 00:02 2024-04-26T03:36:09.609 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:10 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:36:09.636 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-04-26T03:36:09.653 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-04-26T03:36:09.653 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-04-26T03:36:09.734 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-04-26T03:36:09.736 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:36:09.736 INFO:teuthology.orchestra.run.smithi028.stdout: Package Arch Version Repository Size 2024-04-26T03:36:09.736 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:36:09.736 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-04-26T03:36:09.736 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2457.gddeaa917.el9 ceph-noarch 3.5 M 2024-04-26T03:36:09.736 INFO:teuthology.orchestra.run.smithi028.stdout:Installing dependencies: 2024-04-26T03:36:09.736 INFO:teuthology.orchestra.run.smithi028.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T03:36:09.736 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2457.gddeaa917.el9 ceph-noarch 32 k 2024-04-26T03:36:09.736 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2457.gddeaa917.el9 ceph-noarch 15 k 2024-04-26T03:36:09.737 INFO:teuthology.orchestra.run.smithi028.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T03:36:09.737 INFO:teuthology.orchestra.run.smithi028.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T03:36:09.737 INFO:teuthology.orchestra.run.smithi028.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T03:36:09.737 INFO:teuthology.orchestra.run.smithi028.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T03:36:09.737 INFO:teuthology.orchestra.run.smithi028.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T03:36:09.737 INFO:teuthology.orchestra.run.smithi028.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T03:36:09.737 INFO:teuthology.orchestra.run.smithi028.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T03:36:09.737 INFO:teuthology.orchestra.run.smithi028.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T03:36:09.737 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:09.737 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-04-26T03:36:09.737 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:36:09.737 INFO:teuthology.orchestra.run.smithi028.stdout:Install 12 Packages 2024-04-26T03:36:09.737 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:09.738 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 8.6 M 2024-04-26T03:36:09.738 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 103 M 2024-04-26T03:36:09.738 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-04-26T03:36:09.807 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-04-26T03:36:09.808 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-04-26T03:36:09.849 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-2:19.0.0-2457.gddeaa917.el9.x86_64 36/36 2024-04-26T03:36:09.849 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-2:19.0.0-2457.gddeaa917.el9.x86_64 1/36 2024-04-26T03:36:09.849 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mds-2:19.0.0-2457.gddeaa917.el9.x86_64 2/36 2024-04-26T03:36:09.849 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-2:19.0.0-2457.gddeaa917.el9.x86_64 3/36 2024-04-26T03:36:09.849 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mon-2:19.0.0-2457.gddeaa917.el9.x86_64 4/36 2024-04-26T03:36:09.849 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-osd-2:19.0.0-2457.gddeaa917.el9.x86_64 5/36 2024-04-26T03:36:09.849 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2457.gddeaa917.el9. 6/36 2024-04-26T03:36:09.849 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/36 2024-04-26T03:36:09.850 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/36 2024-04-26T03:36:09.850 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/36 2024-04-26T03:36:09.850 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/36 2024-04-26T03:36:09.850 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/36 2024-04-26T03:36:09.850 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/36 2024-04-26T03:36:09.850 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/36 2024-04-26T03:36:09.850 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/36 2024-04-26T03:36:09.850 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/36 2024-04-26T03:36:09.850 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/36 2024-04-26T03:36:09.850 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/36 2024-04-26T03:36:09.851 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-04-26T03:36:09.851 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-04-26T03:36:09.851 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-04-26T03:36:09.851 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-04-26T03:36:09.851 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-04-26T03:36:09.851 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-04-26T03:36:09.851 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-04-26T03:36:09.851 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-04-26T03:36:09.851 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-04-26T03:36:09.851 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-04-26T03:36:09.851 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-04-26T03:36:09.852 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-04-26T03:36:09.852 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-04-26T03:36:09.852 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-04-26T03:36:09.852 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-04-26T03:36:09.852 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-04-26T03:36:09.852 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-04-26T03:36:09.852 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-04-26T03:36:10.009 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2457.gddeaa917.el 11/12 2024-04-26T03:36:10.042 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2457.gddeaa917.el9.noa 12/12 2024-04-26T03:36:10.273 INFO:teuthology.orchestra.run.smithi028.stdout:(1/12): ceph-grafana-dashboards-19.0.0-2457.gdd 126 kB/s | 32 kB 00:00 2024-04-26T03:36:10.306 INFO:teuthology.orchestra.run.smithi028.stdout:(2/12): ceph-prometheus-alerts-19.0.0-2457.gdde 53 kB/s | 15 kB 00:00 2024-04-26T03:36:10.339 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-04-26T03:36:10.420 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-04-26T03:36:10.448 INFO:teuthology.orchestra.run.smithi028.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 2.3 MB/s | 269 kB 00:00 2024-04-26T03:36:10.463 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-04-26T03:36:10.491 INFO:teuthology.orchestra.run.smithi028.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 4.6 MB/s | 1.0 MB 00:00 2024-04-26T03:36:10.510 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-04-26T03:36:10.533 INFO:teuthology.orchestra.run.smithi028.stdout:(5/12): protobuf-compiler-3.14.0-13.el9.x86_64. 10 MB/s | 863 kB 00:00 2024-04-26T03:36:10.540 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-04-26T03:36:10.616 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-04-26T03:36:10.728 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-04-26T03:36:10.728 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:10.728 INFO:teuthology.orchestra.run.smithi066.stdout:Installed: 2024-04-26T03:36:10.728 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:36:10.728 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mds-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:36:10.728 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:36:10.728 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-modules-core-2:19.0.0-2457.gddeaa917.el9.noarch 2024-04-26T03:36:10.728 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mon-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:36:10.728 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-osd-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:36:10.728 INFO:teuthology.orchestra.run.smithi066.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T03:36:10.728 INFO:teuthology.orchestra.run.smithi066.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T03:36:10.728 INFO:teuthology.orchestra.run.smithi066.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T03:36:10.729 INFO:teuthology.orchestra.run.smithi066.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T03:36:10.729 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T03:36:10.729 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T03:36:10.729 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T03:36:10.729 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T03:36:10.729 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T03:36:10.729 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T03:36:10.729 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T03:36:10.729 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T03:36:10.729 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T03:36:10.729 INFO:teuthology.orchestra.run.smithi066.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T03:36:10.729 INFO:teuthology.orchestra.run.smithi066.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T03:36:10.729 INFO:teuthology.orchestra.run.smithi066.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T03:36:10.729 INFO:teuthology.orchestra.run.smithi066.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T03:36:10.730 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T03:36:10.730 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T03:36:10.730 INFO:teuthology.orchestra.run.smithi066.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T03:36:10.730 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T03:36:10.730 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T03:36:10.730 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T03:36:10.730 INFO:teuthology.orchestra.run.smithi066.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T03:36:10.730 INFO:teuthology.orchestra.run.smithi066.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T03:36:10.730 INFO:teuthology.orchestra.run.smithi066.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T03:36:10.730 INFO:teuthology.orchestra.run.smithi066.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T03:36:10.730 INFO:teuthology.orchestra.run.smithi066.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T03:36:10.730 INFO:teuthology.orchestra.run.smithi066.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T03:36:10.730 INFO:teuthology.orchestra.run.smithi066.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T03:36:10.730 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:10.731 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-04-26T03:36:10.757 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-04-26T03:36:10.799 INFO:teuthology.orchestra.run.smithi028.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 73 kB/s | 19 kB 00:00 2024-04-26T03:36:10.853 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-04-26T03:36:10.900 INFO:teuthology.orchestra.run.smithi028.stdout:(7/12): ceph-mgr-dashboard-19.0.0-2457.gddeaa91 4.0 MB/s | 3.5 MB 00:00 2024-04-26T03:36:10.903 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-04-26T03:36:10.934 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-04-26T03:36:11.066 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2457.gddeaa917.el9 10/12 2024-04-26T03:36:11.066 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum -y install ceph-base 2024-04-26T03:36:11.126 INFO:teuthology.orchestra.run.smithi028.stdout:(8/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 866 kB/s | 548 kB 00:00 2024-04-26T03:36:11.325 INFO:teuthology.orchestra.run.smithi028.stdout:(9/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 340 kB/s | 144 kB 00:00 2024-04-26T03:36:11.392 INFO:teuthology.orchestra.run.smithi028.stdout:(10/12): python3-repoze-lru-0.7-16.el9.noarch.r 115 kB/s | 31 kB 00:00 2024-04-26T03:36:11.415 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2457.gddeaa917.el9.noa 12/12 2024-04-26T03:36:11.416 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2457.gddeaa917.el 1/12 2024-04-26T03:36:11.416 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2457.gddeaa917.el9.noa 2/12 2024-04-26T03:36:11.416 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2457.gddeaa917.el9 3/12 2024-04-26T03:36:11.416 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-04-26T03:36:11.416 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-04-26T03:36:11.416 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-04-26T03:36:11.416 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-04-26T03:36:11.416 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-04-26T03:36:11.416 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-04-26T03:36:11.416 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-04-26T03:36:11.416 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-04-26T03:36:11.600 INFO:teuthology.orchestra.run.smithi066.stdout:Last metadata expiration check: 0:01:10 ago on Fri 26 Apr 2024 03:35:01 AM UTC. 2024-04-26T03:36:11.680 INFO:teuthology.orchestra.run.smithi066.stdout:Package ceph-base-2:19.0.0-2457.gddeaa917.el9.x86_64 is already installed. 2024-04-26T03:36:11.723 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-04-26T03:36:11.724 INFO:teuthology.orchestra.run.smithi066.stdout:Nothing to do. 2024-04-26T03:36:11.724 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-04-26T03:36:11.725 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-04-26T03:36:11.726 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:11.726 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-04-26T03:36:11.726 INFO:teuthology.orchestra.run.smithi042.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T03:36:11.726 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-grafana-dashboards-2:19.0.0-2457.gddeaa917.el9.noarch 2024-04-26T03:36:11.726 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-dashboard-2:19.0.0-2457.gddeaa917.el9.noarch 2024-04-26T03:36:11.726 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-prometheus-alerts-2:19.0.0-2457.gddeaa917.el9.noarch 2024-04-26T03:36:11.726 INFO:teuthology.orchestra.run.smithi042.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T03:36:11.726 INFO:teuthology.orchestra.run.smithi042.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T03:36:11.726 INFO:teuthology.orchestra.run.smithi042.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T03:36:11.726 INFO:teuthology.orchestra.run.smithi042.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T03:36:11.727 INFO:teuthology.orchestra.run.smithi042.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T03:36:11.727 INFO:teuthology.orchestra.run.smithi042.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T03:36:11.727 INFO:teuthology.orchestra.run.smithi042.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T03:36:11.727 INFO:teuthology.orchestra.run.smithi042.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T03:36:11.727 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:11.727 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-04-26T03:36:11.776 INFO:teuthology.orchestra.run.smithi028.stdout:(11/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 2.1 MB/s | 2.0 MB 00:00 2024-04-26T03:36:11.801 INFO:teuthology.orchestra.run.smithi028.stdout:(12/12): python3-routes-2.5.1-5.el9.noarch.rpm 397 kB/s | 188 kB 00:00 2024-04-26T03:36:11.805 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum -y install cephadm 2024-04-26T03:36:11.806 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:11.807 INFO:teuthology.orchestra.run.smithi028.stdout:Total 4.2 MB/s | 8.6 MB 00:02 2024-04-26T03:36:11.859 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-04-26T03:36:11.879 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-04-26T03:36:11.879 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-04-26T03:36:11.953 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-26T03:36:12.014 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2457.gddeaa917.el 11/12 2024-04-26T03:36:12.046 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-04-26T03:36:12.046 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-04-26T03:36:12.047 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2457.gddeaa917.el9.noa 12/12 2024-04-26T03:36:12.330 INFO:teuthology.orchestra.run.smithi066.stdout:Last metadata expiration check: 0:01:11 ago on Fri 26 Apr 2024 03:35:01 AM UTC. 2024-04-26T03:36:12.454 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-04-26T03:36:12.455 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:36:12.455 INFO:teuthology.orchestra.run.smithi066.stdout: Package Arch Version Repository Size 2024-04-26T03:36:12.455 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:36:12.455 INFO:teuthology.orchestra.run.smithi066.stdout:Installing: 2024-04-26T03:36:12.455 INFO:teuthology.orchestra.run.smithi066.stdout: cephadm noarch 2:19.0.0-2457.gddeaa917.el9 ceph-noarch 762 k 2024-04-26T03:36:12.455 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:12.455 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction Summary 2024-04-26T03:36:12.455 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:36:12.455 INFO:teuthology.orchestra.run.smithi066.stdout:Install 1 Package 2024-04-26T03:36:12.455 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:12.456 INFO:teuthology.orchestra.run.smithi066.stdout:Total download size: 762 k 2024-04-26T03:36:12.456 INFO:teuthology.orchestra.run.smithi066.stdout:Installed size: 768 k 2024-04-26T03:36:12.456 INFO:teuthology.orchestra.run.smithi066.stdout:Downloading Packages: 2024-04-26T03:36:12.483 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:14 ago on Fri 26 Apr 2024 03:34:58 AM UTC. 2024-04-26T03:36:12.584 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-04-26T03:36:12.609 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-04-26T03:36:12.610 INFO:teuthology.orchestra.run.smithi042.stdout:======================================================================================= 2024-04-26T03:36:12.610 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-04-26T03:36:12.610 INFO:teuthology.orchestra.run.smithi042.stdout:======================================================================================= 2024-04-26T03:36:12.611 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-04-26T03:36:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2457.gddeaa917.el9 ceph-noarch 7.4 M 2024-04-26T03:36:12.611 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-04-26T03:36:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T03:36:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T03:36:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T03:36:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T03:36:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T03:36:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T03:36:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T03:36:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T03:36:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T03:36:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T03:36:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T03:36:12.612 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:12.612 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-04-26T03:36:12.612 INFO:teuthology.orchestra.run.smithi042.stdout:======================================================================================= 2024-04-26T03:36:12.612 INFO:teuthology.orchestra.run.smithi042.stdout:Install 12 Packages 2024-04-26T03:36:12.612 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:12.612 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 38 M 2024-04-26T03:36:12.612 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 209 M 2024-04-26T03:36:12.613 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-04-26T03:36:12.660 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-04-26T03:36:12.705 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-04-26T03:36:12.751 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-04-26T03:36:12.780 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-04-26T03:36:12.861 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-04-26T03:36:12.997 INFO:teuthology.orchestra.run.smithi042.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 195 kB 00:00 2024-04-26T03:36:13.006 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-04-26T03:36:13.022 INFO:teuthology.orchestra.run.smithi042.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 803 kB 00:00 2024-04-26T03:36:13.047 INFO:teuthology.orchestra.run.smithi042.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 660 kB/s | 33 kB 00:00 2024-04-26T03:36:13.074 INFO:teuthology.orchestra.run.smithi066.stdout:cephadm-19.0.0-2457.gddeaa917.el9.noarch.rpm 1.2 MB/s | 762 kB 00:00 2024-04-26T03:36:13.075 INFO:teuthology.orchestra.run.smithi066.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:13.075 INFO:teuthology.orchestra.run.smithi066.stdout:Total 1.2 MB/s | 762 kB 00:00 2024-04-26T03:36:13.075 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction check 2024-04-26T03:36:13.081 INFO:teuthology.orchestra.run.smithi042.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 551 kB/s | 18 kB 00:00 2024-04-26T03:36:13.082 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction check succeeded. 2024-04-26T03:36:13.082 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction test 2024-04-26T03:36:13.087 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction test succeeded. 2024-04-26T03:36:13.087 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction 2024-04-26T03:36:13.124 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-04-26T03:36:13.156 INFO:teuthology.orchestra.run.smithi042.stdout:(5/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 22 MB/s | 3.0 MB 00:00 2024-04-26T03:36:13.162 INFO:teuthology.orchestra.run.smithi066.stdout: Preparing : 1/1 2024-04-26T03:36:13.170 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-04-26T03:36:13.181 INFO:teuthology.orchestra.run.smithi042.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 357 kB/s | 35 kB 00:00 2024-04-26T03:36:13.197 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-04-26T03:36:13.229 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2457.gddeaa917.el9 10/12 2024-04-26T03:36:13.232 INFO:teuthology.orchestra.run.smithi042.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-04-26T03:36:13.341 INFO:teuthology.orchestra.run.smithi042.stdout:(8/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 26 MB/s | 4.7 MB 00:00 2024-04-26T03:36:13.366 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2457.gddeaa917.el9.noa 12/12 2024-04-26T03:36:13.366 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2457.gddeaa917.el 1/12 2024-04-26T03:36:13.366 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2457.gddeaa917.el9.noa 2/12 2024-04-26T03:36:13.366 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2457.gddeaa917.el9 3/12 2024-04-26T03:36:13.366 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-04-26T03:36:13.366 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-04-26T03:36:13.366 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-04-26T03:36:13.366 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-04-26T03:36:13.366 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-04-26T03:36:13.366 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-04-26T03:36:13.367 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-04-26T03:36:13.367 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-04-26T03:36:13.433 INFO:teuthology.orchestra.run.smithi042.stdout:(9/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 26 MB/s | 5.1 MB 00:00 2024-04-26T03:36:13.458 INFO:teuthology.orchestra.run.smithi042.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.1 MB/s | 250 kB 00:00 2024-04-26T03:36:13.640 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: cephadm-2:19.0.0-2457.gddeaa917.el9.noarch 1/1 2024-04-26T03:36:13.658 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-04-26T03:36:13.658 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:13.658 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-04-26T03:36:13.658 INFO:teuthology.orchestra.run.smithi092.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T03:36:13.659 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards-2:19.0.0-2457.gddeaa917.el9.noarch 2024-04-26T03:36:13.659 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard-2:19.0.0-2457.gddeaa917.el9.noarch 2024-04-26T03:36:13.659 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts-2:19.0.0-2457.gddeaa917.el9.noarch 2024-04-26T03:36:13.659 INFO:teuthology.orchestra.run.smithi092.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T03:36:13.659 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T03:36:13.660 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T03:36:13.660 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T03:36:13.660 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T03:36:13.660 INFO:teuthology.orchestra.run.smithi092.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T03:36:13.660 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T03:36:13.661 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T03:36:13.661 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:13.661 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-26T03:36:13.729 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : cephadm-2:19.0.0-2457.gddeaa917.el9.noarch 1/1 2024-04-26T03:36:13.875 INFO:teuthology.orchestra.run.smithi042.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 37 MB/s | 16 MB 00:00 2024-04-26T03:36:13.914 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-26T03:36:14.014 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: cephadm-2:19.0.0-2457.gddeaa917.el9.noarch 1/1 2024-04-26T03:36:14.232 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : cephadm-2:19.0.0-2457.gddeaa917.el9.noarch 1/1 2024-04-26T03:36:14.233 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:14.233 INFO:teuthology.orchestra.run.smithi066.stdout:Installed: 2024-04-26T03:36:14.233 INFO:teuthology.orchestra.run.smithi066.stdout: cephadm-2:19.0.0-2457.gddeaa917.el9.noarch 2024-04-26T03:36:14.233 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:14.233 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-04-26T03:36:14.326 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2457.gddeaa917.el 11/12 2024-04-26T03:36:14.363 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2457.gddeaa917.el9.noa 12/12 2024-04-26T03:36:14.427 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:15 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:36:14.455 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum -y install ceph-immutable-object-cache 2024-04-26T03:36:14.549 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-26T03:36:14.550 INFO:teuthology.orchestra.run.smithi092.stdout:======================================================================================= 2024-04-26T03:36:14.550 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-04-26T03:36:14.550 INFO:teuthology.orchestra.run.smithi092.stdout:======================================================================================= 2024-04-26T03:36:14.551 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-04-26T03:36:14.551 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2457.gddeaa917.el9 ceph-noarch 7.4 M 2024-04-26T03:36:14.551 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-04-26T03:36:14.551 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T03:36:14.551 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T03:36:14.551 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T03:36:14.551 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T03:36:14.551 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T03:36:14.551 INFO:teuthology.orchestra.run.smithi092.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T03:36:14.551 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T03:36:14.551 INFO:teuthology.orchestra.run.smithi092.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T03:36:14.552 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T03:36:14.552 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T03:36:14.552 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T03:36:14.552 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:14.552 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-04-26T03:36:14.552 INFO:teuthology.orchestra.run.smithi092.stdout:======================================================================================= 2024-04-26T03:36:14.552 INFO:teuthology.orchestra.run.smithi092.stdout:Install 12 Packages 2024-04-26T03:36:14.552 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:14.553 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 38 M 2024-04-26T03:36:14.553 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 209 M 2024-04-26T03:36:14.553 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-04-26T03:36:14.990 INFO:teuthology.orchestra.run.smithi066.stdout:Last metadata expiration check: 0:01:13 ago on Fri 26 Apr 2024 03:35:01 AM UTC. 2024-04-26T03:36:15.116 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-04-26T03:36:15.117 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:36:15.117 INFO:teuthology.orchestra.run.smithi066.stdout: Package Arch Version Repo Size 2024-04-26T03:36:15.117 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:36:15.117 INFO:teuthology.orchestra.run.smithi066.stdout:Installing: 2024-04-26T03:36:15.117 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 145 k 2024-04-26T03:36:15.118 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:15.118 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction Summary 2024-04-26T03:36:15.118 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:36:15.118 INFO:teuthology.orchestra.run.smithi066.stdout:Install 1 Package 2024-04-26T03:36:15.118 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:15.118 INFO:teuthology.orchestra.run.smithi066.stdout:Total download size: 145 k 2024-04-26T03:36:15.118 INFO:teuthology.orchestra.run.smithi066.stdout:Installed size: 435 k 2024-04-26T03:36:15.118 INFO:teuthology.orchestra.run.smithi066.stdout:Downloading Packages: 2024-04-26T03:36:15.126 INFO:teuthology.orchestra.run.smithi092.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 195 kB 00:00 2024-04-26T03:36:15.160 INFO:teuthology.orchestra.run.smithi092.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 803 kB 00:00 2024-04-26T03:36:15.185 INFO:teuthology.orchestra.run.smithi092.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 566 kB/s | 33 kB 00:00 2024-04-26T03:36:15.227 INFO:teuthology.orchestra.run.smithi092.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 442 kB/s | 18 kB 00:00 2024-04-26T03:36:15.277 INFO:teuthology.orchestra.run.smithi092.stdout:(5/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 26 MB/s | 3.0 MB 00:00 2024-04-26T03:36:15.302 INFO:teuthology.orchestra.run.smithi092.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 482 kB/s | 35 kB 00:00 2024-04-26T03:36:15.338 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-immutable-object-cache-19.0.0-2457.gddeaa9 658 kB/s | 145 kB 00:00 2024-04-26T03:36:15.339 INFO:teuthology.orchestra.run.smithi066.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:15.339 INFO:teuthology.orchestra.run.smithi066.stdout:Total 653 kB/s | 145 kB 00:00 2024-04-26T03:36:15.339 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction check 2024-04-26T03:36:15.344 INFO:teuthology.orchestra.run.smithi092.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 5.7 MB/s | 244 kB 00:00 2024-04-26T03:36:15.348 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction check succeeded. 2024-04-26T03:36:15.348 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction test 2024-04-26T03:36:15.407 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction test succeeded. 2024-04-26T03:36:15.408 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction 2024-04-26T03:36:15.445 INFO:teuthology.orchestra.run.smithi092.stdout:(8/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 28 MB/s | 4.7 MB 00:00 2024-04-26T03:36:15.529 INFO:teuthology.orchestra.run.smithi092.stdout:(9/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 28 MB/s | 5.1 MB 00:00 2024-04-26T03:36:15.552 INFO:teuthology.orchestra.run.smithi066.stdout: Preparing : 1/1 2024-04-26T03:36:15.554 INFO:teuthology.orchestra.run.smithi092.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.3 MB/s | 250 kB 00:00 2024-04-26T03:36:15.570 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2457.gddeaa917.el9.noa 12/12 2024-04-26T03:36:15.570 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2457.gddeaa917.el 1/12 2024-04-26T03:36:15.570 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2457.gddeaa917.el9.noa 2/12 2024-04-26T03:36:15.570 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2457.gddeaa917.el9 3/12 2024-04-26T03:36:15.570 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-04-26T03:36:15.570 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-04-26T03:36:15.570 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-04-26T03:36:15.570 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-04-26T03:36:15.570 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-04-26T03:36:15.570 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-04-26T03:36:15.570 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-04-26T03:36:15.571 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-04-26T03:36:15.583 INFO:teuthology.orchestra.run.smithi042.stdout:(12/12): ceph-mgr-diskprediction-local-19.0.0-2 2.7 MB/s | 7.4 MB 00:02 2024-04-26T03:36:15.586 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:15.587 INFO:teuthology.orchestra.run.smithi042.stdout:Total 13 MB/s | 38 MB 00:02 2024-04-26T03:36:15.639 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2457.gddeaa917. 1/1 2024-04-26T03:36:15.665 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2457.gddeaa917. 1/1 2024-04-26T03:36:15.665 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T03:36:15.665 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T03:36:15.665 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:15.872 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-04-26T03:36:15.872 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:15.872 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-04-26T03:36:15.872 INFO:teuthology.orchestra.run.smithi028.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T03:36:15.872 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-grafana-dashboards-2:19.0.0-2457.gddeaa917.el9.noarch 2024-04-26T03:36:15.872 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-dashboard-2:19.0.0-2457.gddeaa917.el9.noarch 2024-04-26T03:36:15.872 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-prometheus-alerts-2:19.0.0-2457.gddeaa917.el9.noarch 2024-04-26T03:36:15.872 INFO:teuthology.orchestra.run.smithi028.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T03:36:15.872 INFO:teuthology.orchestra.run.smithi028.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T03:36:15.872 INFO:teuthology.orchestra.run.smithi028.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T03:36:15.873 INFO:teuthology.orchestra.run.smithi028.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T03:36:15.873 INFO:teuthology.orchestra.run.smithi028.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T03:36:15.873 INFO:teuthology.orchestra.run.smithi028.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T03:36:15.873 INFO:teuthology.orchestra.run.smithi028.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T03:36:15.873 INFO:teuthology.orchestra.run.smithi028.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T03:36:15.873 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:15.873 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-04-26T03:36:15.953 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-04-26T03:36:15.981 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-04-26T03:36:15.981 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-04-26T03:36:15.988 INFO:teuthology.orchestra.run.smithi092.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-04-26T03:36:16.121 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-26T03:36:16.122 INFO:teuthology.orchestra.run.smithi092.stdout:(12/12): ceph-mgr-diskprediction-local-19.0.0-2 6.3 MB/s | 7.4 MB 00:01 2024-04-26T03:36:16.124 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:16.124 INFO:teuthology.orchestra.run.smithi092.stdout:Total 24 MB/s | 38 MB 00:01 2024-04-26T03:36:16.415 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-04-26T03:36:16.415 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-04-26T03:36:16.470 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-04-26T03:36:16.496 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-04-26T03:36:16.496 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-04-26T03:36:16.517 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2457.gddeaa917. 1/1 2024-04-26T03:36:16.517 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:16.517 INFO:teuthology.orchestra.run.smithi066.stdout:Installed: 2024-04-26T03:36:16.517 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-immutable-object-cache-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:36:16.517 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:16.517 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-04-26T03:36:16.660 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:17 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:36:16.726 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum -y install ceph-mgr 2024-04-26T03:36:16.787 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-04-26T03:36:16.788 INFO:teuthology.orchestra.run.smithi028.stdout:======================================================================================= 2024-04-26T03:36:16.788 INFO:teuthology.orchestra.run.smithi028.stdout: Package Arch Version Repository Size 2024-04-26T03:36:16.788 INFO:teuthology.orchestra.run.smithi028.stdout:======================================================================================= 2024-04-26T03:36:16.788 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-04-26T03:36:16.788 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2457.gddeaa917.el9 ceph-noarch 7.4 M 2024-04-26T03:36:16.788 INFO:teuthology.orchestra.run.smithi028.stdout:Installing dependencies: 2024-04-26T03:36:16.788 INFO:teuthology.orchestra.run.smithi028.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T03:36:16.788 INFO:teuthology.orchestra.run.smithi028.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T03:36:16.788 INFO:teuthology.orchestra.run.smithi028.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T03:36:16.788 INFO:teuthology.orchestra.run.smithi028.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T03:36:16.788 INFO:teuthology.orchestra.run.smithi028.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T03:36:16.788 INFO:teuthology.orchestra.run.smithi028.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T03:36:16.789 INFO:teuthology.orchestra.run.smithi028.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T03:36:16.789 INFO:teuthology.orchestra.run.smithi028.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T03:36:16.789 INFO:teuthology.orchestra.run.smithi028.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T03:36:16.789 INFO:teuthology.orchestra.run.smithi028.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T03:36:16.789 INFO:teuthology.orchestra.run.smithi028.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T03:36:16.789 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:16.789 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-04-26T03:36:16.789 INFO:teuthology.orchestra.run.smithi028.stdout:======================================================================================= 2024-04-26T03:36:16.789 INFO:teuthology.orchestra.run.smithi028.stdout:Install 12 Packages 2024-04-26T03:36:16.789 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:16.790 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 38 M 2024-04-26T03:36:16.790 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 209 M 2024-04-26T03:36:16.790 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-04-26T03:36:16.900 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-04-26T03:36:16.900 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-04-26T03:36:17.254 INFO:teuthology.orchestra.run.smithi066.stdout:Last metadata expiration check: 0:01:16 ago on Fri 26 Apr 2024 03:35:01 AM UTC. 2024-04-26T03:36:17.270 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-04-26T03:36:17.336 INFO:teuthology.orchestra.run.smithi066.stdout:Package ceph-mgr-2:19.0.0-2457.gddeaa917.el9.x86_64 is already installed. 2024-04-26T03:36:17.347 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-26T03:36:17.379 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-04-26T03:36:17.381 INFO:teuthology.orchestra.run.smithi066.stdout:Nothing to do. 2024-04-26T03:36:17.381 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-04-26T03:36:17.391 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-26T03:36:17.426 INFO:teuthology.orchestra.run.smithi028.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-04-26T03:36:17.460 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum -y install ceph-mgr-dashboard 2024-04-26T03:36:17.475 INFO:teuthology.orchestra.run.smithi028.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 803 kB 00:00 2024-04-26T03:36:17.499 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-26T03:36:17.501 INFO:teuthology.orchestra.run.smithi028.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 438 kB/s | 33 kB 00:00 2024-04-26T03:36:17.542 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-04-26T03:36:17.568 INFO:teuthology.orchestra.run.smithi028.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 33 MB/s | 3.0 MB 00:00 2024-04-26T03:36:17.593 INFO:teuthology.orchestra.run.smithi028.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 202 kB/s | 18 kB 00:00 2024-04-26T03:36:17.618 INFO:teuthology.orchestra.run.smithi028.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 703 kB/s | 35 kB 00:00 2024-04-26T03:36:17.727 INFO:teuthology.orchestra.run.smithi028.stdout:(7/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 35 MB/s | 4.7 MB 00:00 2024-04-26T03:36:17.728 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-26T03:36:17.744 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-04-26T03:36:17.752 INFO:teuthology.orchestra.run.smithi028.stdout:(8/12): python3-devel-3.9.18-3.el9.x86_64.rpm 1.8 MB/s | 244 kB 00:00 2024-04-26T03:36:17.766 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-26T03:36:17.809 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-26T03:36:17.846 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-26T03:36:17.852 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-26T03:36:17.869 INFO:teuthology.orchestra.run.smithi028.stdout:(9/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 5.1 MB 00:00 2024-04-26T03:36:17.894 INFO:teuthology.orchestra.run.smithi028.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 1.8 MB/s | 250 kB 00:00 2024-04-26T03:36:17.951 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-26T03:36:17.987 INFO:teuthology.orchestra.run.smithi066.stdout:Last metadata expiration check: 0:01:16 ago on Fri 26 Apr 2024 03:35:01 AM UTC. 2024-04-26T03:36:17.987 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-04-26T03:36:18.115 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-04-26T03:36:18.116 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:36:18.117 INFO:teuthology.orchestra.run.smithi066.stdout: Package Arch Version Repository Size 2024-04-26T03:36:18.117 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:36:18.117 INFO:teuthology.orchestra.run.smithi066.stdout:Installing: 2024-04-26T03:36:18.117 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2457.gddeaa917.el9 ceph-noarch 3.5 M 2024-04-26T03:36:18.117 INFO:teuthology.orchestra.run.smithi066.stdout:Installing dependencies: 2024-04-26T03:36:18.117 INFO:teuthology.orchestra.run.smithi066.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T03:36:18.117 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2457.gddeaa917.el9 ceph-noarch 32 k 2024-04-26T03:36:18.117 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2457.gddeaa917.el9 ceph-noarch 15 k 2024-04-26T03:36:18.117 INFO:teuthology.orchestra.run.smithi066.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T03:36:18.117 INFO:teuthology.orchestra.run.smithi066.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T03:36:18.118 INFO:teuthology.orchestra.run.smithi066.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T03:36:18.118 INFO:teuthology.orchestra.run.smithi066.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T03:36:18.118 INFO:teuthology.orchestra.run.smithi066.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T03:36:18.118 INFO:teuthology.orchestra.run.smithi066.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T03:36:18.118 INFO:teuthology.orchestra.run.smithi066.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T03:36:18.118 INFO:teuthology.orchestra.run.smithi066.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T03:36:18.118 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:18.118 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction Summary 2024-04-26T03:36:18.118 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:36:18.118 INFO:teuthology.orchestra.run.smithi066.stdout:Install 12 Packages 2024-04-26T03:36:18.118 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:18.119 INFO:teuthology.orchestra.run.smithi066.stdout:Total download size: 8.6 M 2024-04-26T03:36:18.119 INFO:teuthology.orchestra.run.smithi066.stdout:Installed size: 103 M 2024-04-26T03:36:18.119 INFO:teuthology.orchestra.run.smithi066.stdout:Downloading Packages: 2024-04-26T03:36:18.164 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-26T03:36:18.194 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-26T03:36:18.236 INFO:teuthology.orchestra.run.smithi028.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 44 MB/s | 16 MB 00:00 2024-04-26T03:36:18.270 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-26T03:36:18.528 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-26T03:36:18.628 INFO:teuthology.orchestra.run.smithi028.stdout:(12/12): ceph-mgr-diskprediction-local-19.0.0-2 5.3 MB/s | 7.4 MB 00:01 2024-04-26T03:36:18.631 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:18.631 INFO:teuthology.orchestra.run.smithi028.stdout:Total 21 MB/s | 38 MB 00:01 2024-04-26T03:36:18.652 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-26T03:36:18.659 INFO:teuthology.orchestra.run.smithi066.stdout:(1/12): ceph-grafana-dashboards-19.0.0-2457.gdd 168 kB/s | 32 kB 00:00 2024-04-26T03:36:18.759 INFO:teuthology.orchestra.run.smithi066.stdout:(2/12): ceph-prometheus-alerts-19.0.0-2457.gdde 53 kB/s | 15 kB 00:00 2024-04-26T03:36:18.826 INFO:teuthology.orchestra.run.smithi066.stdout:(3/12): protobuf-3.14.0-13.el9.x86_64.rpm 6.1 MB/s | 1.0 MB 00:00 2024-04-26T03:36:18.851 INFO:teuthology.orchestra.run.smithi066.stdout:(4/12): python3-protobuf-3.14.0-13.el9.noarch.r 2.9 MB/s | 269 kB 00:00 2024-04-26T03:36:18.885 INFO:teuthology.orchestra.run.smithi066.stdout:(5/12): protobuf-compiler-3.14.0-13.el9.x86_64. 14 MB/s | 863 kB 00:00 2024-04-26T03:36:18.917 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-26T03:36:18.991 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-04-26T03:36:19.019 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-04-26T03:36:19.020 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-04-26T03:36:19.032 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-26T03:36:19.127 INFO:teuthology.orchestra.run.smithi066.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 80 kB/s | 19 kB 00:00 2024-04-26T03:36:19.227 INFO:teuthology.orchestra.run.smithi066.stdout:(7/12): ceph-mgr-dashboard-19.0.0-2457.gddeaa91 4.6 MB/s | 3.5 MB 00:00 2024-04-26T03:36:19.260 INFO:teuthology.orchestra.run.smithi066.stdout:(8/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 1.3 MB/s | 548 kB 00:00 2024-04-26T03:36:19.386 INFO:teuthology.orchestra.run.smithi066.stdout:(9/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 7.9 MB/s | 2.0 MB 00:00 2024-04-26T03:36:19.411 INFO:teuthology.orchestra.run.smithi066.stdout:(10/12): python3-grpcio-tools-1.46.7-10.el9.x86 787 kB/s | 144 kB 00:00 2024-04-26T03:36:19.436 INFO:teuthology.orchestra.run.smithi066.stdout:(11/12): python3-repoze-lru-0.7-16.el9.noarch.r 176 kB/s | 31 kB 00:00 2024-04-26T03:36:19.454 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-04-26T03:36:19.454 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-04-26T03:36:19.461 INFO:teuthology.orchestra.run.smithi066.stdout:(12/12): python3-routes-2.5.1-5.el9.noarch.rpm 2.5 MB/s | 188 kB 00:00 2024-04-26T03:36:19.466 INFO:teuthology.orchestra.run.smithi066.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:19.466 INFO:teuthology.orchestra.run.smithi066.stdout:Total 6.4 MB/s | 8.6 MB 00:01 2024-04-26T03:36:19.515 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction check 2024-04-26T03:36:19.533 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction check succeeded. 2024-04-26T03:36:19.533 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction test 2024-04-26T03:36:19.701 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction test succeeded. 2024-04-26T03:36:19.701 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction 2024-04-26T03:36:19.971 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-26T03:36:20.252 INFO:teuthology.orchestra.run.smithi066.stdout: Preparing : 1/1 2024-04-26T03:36:20.260 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-26T03:36:20.303 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-04-26T03:36:20.329 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-04-26T03:36:20.358 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-26T03:36:20.364 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-04-26T03:36:20.394 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-26T03:36:20.412 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-04-26T03:36:20.449 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-04-26T03:36:20.497 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-26T03:36:20.529 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-04-26T03:36:20.563 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-04-26T03:36:20.665 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-04-26T03:36:20.746 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-26T03:36:20.757 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-04-26T03:36:20.776 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-26T03:36:20.805 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-04-26T03:36:20.832 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-04-26T03:36:20.857 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-26T03:36:20.864 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2457.gddeaa917.el9 10/12 2024-04-26T03:36:21.062 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-26T03:36:21.091 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2457.gddeaa 12/12 2024-04-26T03:36:21.301 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-26T03:36:21.335 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2457.gddeaa 12/12 2024-04-26T03:36:21.535 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-26T03:36:21.662 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-26T03:36:21.976 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2457.gddeaa917.el 11/12 2024-04-26T03:36:22.015 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2457.gddeaa917.el9.noa 12/12 2024-04-26T03:36:22.962 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-26T03:36:23.340 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2457.gddeaa917.el9.noa 12/12 2024-04-26T03:36:23.340 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2457.gddeaa917.el 1/12 2024-04-26T03:36:23.340 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2457.gddeaa917.el9.noa 2/12 2024-04-26T03:36:23.340 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2457.gddeaa917.el9 3/12 2024-04-26T03:36:23.340 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-04-26T03:36:23.341 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-04-26T03:36:23.341 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-04-26T03:36:23.341 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-04-26T03:36:23.341 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-04-26T03:36:23.341 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-04-26T03:36:23.341 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-04-26T03:36:23.341 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-04-26T03:36:23.646 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-04-26T03:36:23.646 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:23.646 INFO:teuthology.orchestra.run.smithi066.stdout:Installed: 2024-04-26T03:36:23.647 INFO:teuthology.orchestra.run.smithi066.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T03:36:23.647 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-grafana-dashboards-2:19.0.0-2457.gddeaa917.el9.noarch 2024-04-26T03:36:23.647 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-dashboard-2:19.0.0-2457.gddeaa917.el9.noarch 2024-04-26T03:36:23.647 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-prometheus-alerts-2:19.0.0-2457.gddeaa917.el9.noarch 2024-04-26T03:36:23.647 INFO:teuthology.orchestra.run.smithi066.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T03:36:23.647 INFO:teuthology.orchestra.run.smithi066.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T03:36:23.647 INFO:teuthology.orchestra.run.smithi066.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T03:36:23.647 INFO:teuthology.orchestra.run.smithi066.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T03:36:23.647 INFO:teuthology.orchestra.run.smithi066.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T03:36:23.647 INFO:teuthology.orchestra.run.smithi066.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T03:36:23.647 INFO:teuthology.orchestra.run.smithi066.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T03:36:23.647 INFO:teuthology.orchestra.run.smithi066.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T03:36:23.647 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:23.647 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-04-26T03:36:23.847 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2457.gddeaa 12/12 2024-04-26T03:36:23.847 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2457.gddeaa 1/12 2024-04-26T03:36:23.848 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-26T03:36:23.848 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-26T03:36:23.848 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-26T03:36:23.848 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-26T03:36:23.848 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-26T03:36:23.848 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-26T03:36:23.848 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-26T03:36:23.848 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-04-26T03:36:23.848 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-26T03:36:23.848 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-26T03:36:23.880 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-26T03:36:23.949 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2457.gddeaa 12/12 2024-04-26T03:36:23.949 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2457.gddeaa 1/12 2024-04-26T03:36:23.949 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-26T03:36:23.949 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-26T03:36:23.949 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-26T03:36:23.949 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-26T03:36:23.949 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-26T03:36:23.949 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-26T03:36:23.949 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-26T03:36:23.949 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-04-26T03:36:23.949 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-26T03:36:23.949 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-26T03:36:24.061 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-26T03:36:24.093 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2457.gddeaa 12/12 2024-04-26T03:36:24.235 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-26T03:36:24.236 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:24.236 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-04-26T03:36:24.236 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2457.gddeaa917.el9.noarch 2024-04-26T03:36:24.236 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T03:36:24.236 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T03:36:24.236 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T03:36:24.236 INFO:teuthology.orchestra.run.smithi042.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T03:36:24.236 INFO:teuthology.orchestra.run.smithi042.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T03:36:24.236 INFO:teuthology.orchestra.run.smithi042.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T03:36:24.236 INFO:teuthology.orchestra.run.smithi042.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T03:36:24.236 INFO:teuthology.orchestra.run.smithi042.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T03:36:24.236 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T03:36:24.237 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T03:36:24.237 INFO:teuthology.orchestra.run.smithi042.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T03:36:24.237 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:24.237 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-04-26T03:36:24.331 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-26T03:36:24.331 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:24.332 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-04-26T03:36:24.332 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2457.gddeaa917.el9.noarch 2024-04-26T03:36:24.332 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T03:36:24.332 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T03:36:24.332 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T03:36:24.332 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T03:36:24.332 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T03:36:24.332 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T03:36:24.332 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T03:36:24.332 INFO:teuthology.orchestra.run.smithi092.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T03:36:24.332 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T03:36:24.332 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T03:36:24.332 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T03:36:24.332 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:24.333 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-26T03:36:24.414 INFO:teuthology.orchestra.run.smithi066.stdout:Last metadata expiration check: 0:01:23 ago on Fri 26 Apr 2024 03:35:01 AM UTC. 2024-04-26T03:36:24.540 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-04-26T03:36:24.540 INFO:teuthology.orchestra.run.smithi066.stdout:======================================================================================= 2024-04-26T03:36:24.541 INFO:teuthology.orchestra.run.smithi066.stdout: Package Arch Version Repository Size 2024-04-26T03:36:24.541 INFO:teuthology.orchestra.run.smithi066.stdout:======================================================================================= 2024-04-26T03:36:24.541 INFO:teuthology.orchestra.run.smithi066.stdout:Installing: 2024-04-26T03:36:24.541 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2457.gddeaa917.el9 ceph-noarch 7.4 M 2024-04-26T03:36:24.541 INFO:teuthology.orchestra.run.smithi066.stdout:Installing dependencies: 2024-04-26T03:36:24.541 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T03:36:24.541 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T03:36:24.541 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T03:36:24.541 INFO:teuthology.orchestra.run.smithi066.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T03:36:24.541 INFO:teuthology.orchestra.run.smithi066.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T03:36:24.541 INFO:teuthology.orchestra.run.smithi066.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T03:36:24.541 INFO:teuthology.orchestra.run.smithi066.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T03:36:24.541 INFO:teuthology.orchestra.run.smithi066.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T03:36:24.542 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T03:36:24.542 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T03:36:24.542 INFO:teuthology.orchestra.run.smithi066.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T03:36:24.542 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:24.542 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction Summary 2024-04-26T03:36:24.542 INFO:teuthology.orchestra.run.smithi066.stdout:======================================================================================= 2024-04-26T03:36:24.542 INFO:teuthology.orchestra.run.smithi066.stdout:Install 12 Packages 2024-04-26T03:36:24.542 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:24.543 INFO:teuthology.orchestra.run.smithi066.stdout:Total download size: 38 M 2024-04-26T03:36:24.543 INFO:teuthology.orchestra.run.smithi066.stdout:Installed size: 209 M 2024-04-26T03:36:24.543 INFO:teuthology.orchestra.run.smithi066.stdout:Downloading Packages: 2024-04-26T03:36:24.607 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-mgr-rook 2024-04-26T03:36:24.730 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-mgr-rook 2024-04-26T03:36:24.969 INFO:teuthology.orchestra.run.smithi066.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-04-26T03:36:25.044 INFO:teuthology.orchestra.run.smithi066.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 3.0 MB/s | 803 kB 00:00 2024-04-26T03:36:25.069 INFO:teuthology.orchestra.run.smithi066.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 328 kB/s | 33 kB 00:00 2024-04-26T03:36:25.143 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:27 ago on Fri 26 Apr 2024 03:34:58 AM UTC. 2024-04-26T03:36:25.242 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:26 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:36:25.261 INFO:teuthology.orchestra.run.smithi066.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 95 kB/s | 18 kB 00:00 2024-04-26T03:36:25.268 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-04-26T03:36:25.270 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:36:25.270 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-04-26T03:36:25.270 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:36:25.270 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-04-26T03:36:25.270 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-rook noarch 2:19.0.0-2457.gddeaa917.el9 ceph-noarch 49 k 2024-04-26T03:36:25.270 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-04-26T03:36:25.270 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T03:36:25.270 INFO:teuthology.orchestra.run.smithi042.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T03:36:25.271 INFO:teuthology.orchestra.run.smithi042.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T03:36:25.271 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T03:36:25.271 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T03:36:25.271 INFO:teuthology.orchestra.run.smithi042.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T03:36:25.271 INFO:teuthology.orchestra.run.smithi042.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T03:36:25.271 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T03:36:25.271 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T03:36:25.271 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T03:36:25.271 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T03:36:25.271 INFO:teuthology.orchestra.run.smithi042.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T03:36:25.271 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:25.271 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-04-26T03:36:25.271 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:36:25.271 INFO:teuthology.orchestra.run.smithi042.stdout:Install 13 Packages 2024-04-26T03:36:25.272 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:25.272 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 2.2 M 2024-04-26T03:36:25.272 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 26 M 2024-04-26T03:36:25.272 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-04-26T03:36:25.361 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-26T03:36:25.362 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:36:25.362 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-04-26T03:36:25.363 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:36:25.363 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-04-26T03:36:25.363 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook noarch 2:19.0.0-2457.gddeaa917.el9 ceph-noarch 49 k 2024-04-26T03:36:25.363 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-04-26T03:36:25.363 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T03:36:25.363 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T03:36:25.363 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T03:36:25.363 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T03:36:25.363 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T03:36:25.363 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T03:36:25.363 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T03:36:25.363 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T03:36:25.363 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T03:36:25.363 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T03:36:25.364 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T03:36:25.364 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T03:36:25.364 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:25.364 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-04-26T03:36:25.364 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:36:25.364 INFO:teuthology.orchestra.run.smithi092.stdout:Install 13 Packages 2024-04-26T03:36:25.364 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:25.364 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 2.2 M 2024-04-26T03:36:25.365 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 26 M 2024-04-26T03:36:25.365 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-04-26T03:36:25.411 INFO:teuthology.orchestra.run.smithi066.stdout:(5/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 8.1 MB/s | 3.0 MB 00:00 2024-04-26T03:36:25.437 INFO:teuthology.orchestra.run.smithi066.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 201 kB/s | 35 kB 00:00 2024-04-26T03:36:25.604 INFO:teuthology.orchestra.run.smithi066.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 1.4 MB/s | 244 kB 00:00 2024-04-26T03:36:25.653 INFO:teuthology.orchestra.run.smithi092.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 193 kB/s | 26 kB 00:00 2024-04-26T03:36:25.677 INFO:teuthology.orchestra.run.smithi042.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 224 kB/s | 26 kB 00:00 2024-04-26T03:36:25.678 INFO:teuthology.orchestra.run.smithi092.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 121 kB/s | 19 kB 00:00 2024-04-26T03:36:25.687 INFO:teuthology.orchestra.run.smithi066.stdout:(8/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 17 MB/s | 4.7 MB 00:00 2024-04-26T03:36:25.727 INFO:teuthology.orchestra.run.smithi042.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 116 kB/s | 19 kB 00:00 2024-04-26T03:36:25.736 INFO:teuthology.orchestra.run.smithi092.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.6 MB/s | 222 kB 00:00 2024-04-26T03:36:25.762 INFO:teuthology.orchestra.run.smithi092.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2024-04-26T03:36:25.778 INFO:teuthology.orchestra.run.smithi042.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.9 MB/s | 222 kB 00:00 2024-04-26T03:36:25.787 INFO:teuthology.orchestra.run.smithi092.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 5.4 MB/s | 279 kB 00:00 2024-04-26T03:36:25.803 INFO:teuthology.orchestra.run.smithi042.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-04-26T03:36:25.813 INFO:teuthology.orchestra.run.smithi092.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.1 MB/s | 54 kB 00:00 2024-04-26T03:36:25.828 INFO:teuthology.orchestra.run.smithi042.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 5.5 MB/s | 279 kB 00:00 2024-04-26T03:36:25.829 INFO:teuthology.orchestra.run.smithi066.stdout:(9/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 23 MB/s | 5.1 MB 00:00 2024-04-26T03:36:25.838 INFO:teuthology.orchestra.run.smithi092.stdout:(7/13): ceph-mgr-rook-19.0.0-2457.gddeaa917.el9 155 kB/s | 49 kB 00:00 2024-04-26T03:36:25.853 INFO:teuthology.orchestra.run.smithi042.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-04-26T03:36:25.854 INFO:teuthology.orchestra.run.smithi066.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 1.5 MB/s | 250 kB 00:00 2024-04-26T03:36:26.019 INFO:teuthology.orchestra.run.smithi042.stdout:(7/13): ceph-mgr-rook-19.0.0-2457.gddeaa917.el9 108 kB/s | 49 kB 00:00 2024-04-26T03:36:26.046 INFO:teuthology.orchestra.run.smithi042.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 78 kB/s | 15 kB 00:00 2024-04-26T03:36:26.071 INFO:teuthology.orchestra.run.smithi042.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 133 kB/s | 32 kB 00:00 2024-04-26T03:36:26.146 INFO:teuthology.orchestra.run.smithi092.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 45 kB/s | 15 kB 00:00 2024-04-26T03:36:26.222 INFO:teuthology.orchestra.run.smithi042.stdout:(10/13): python3-google-auth-2.29.0-1.el9.noarc 1.0 MB/s | 206 kB 00:00 2024-04-26T03:36:26.230 INFO:teuthology.orchestra.run.smithi092.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 73 kB/s | 32 kB 00:00 2024-04-26T03:36:26.247 INFO:teuthology.orchestra.run.smithi042.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 337 kB/s | 59 kB 00:00 2024-04-26T03:36:26.289 INFO:teuthology.orchestra.run.smithi042.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.3 MB/s | 90 kB 00:00 2024-04-26T03:36:26.331 INFO:teuthology.orchestra.run.smithi042.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 3.6 MB/s | 1.0 MB 00:00 2024-04-26T03:36:26.333 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:26.333 INFO:teuthology.orchestra.run.smithi042.stdout:Total 2.1 MB/s | 2.2 MB 00:01 2024-04-26T03:36:26.363 INFO:teuthology.orchestra.run.smithi066.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 30 MB/s | 16 MB 00:00 2024-04-26T03:36:26.403 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-04-26T03:36:26.420 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-04-26T03:36:26.420 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-04-26T03:36:26.438 INFO:teuthology.orchestra.run.smithi092.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 284 kB/s | 59 kB 00:00 2024-04-26T03:36:26.513 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-04-26T03:36:26.513 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-04-26T03:36:26.574 INFO:teuthology.orchestra.run.smithi092.stdout:(11/13): python3-google-auth-2.29.0-1.el9.noarc 280 kB/s | 206 kB 00:00 2024-04-26T03:36:26.591 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2457.gddeaa 12/12 2024-04-26T03:36:26.592 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2457.gddeaa 1/12 2024-04-26T03:36:26.592 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-26T03:36:26.592 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-26T03:36:26.592 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-26T03:36:26.592 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-26T03:36:26.592 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-26T03:36:26.592 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-26T03:36:26.592 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-26T03:36:26.592 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-04-26T03:36:26.592 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-26T03:36:26.592 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-26T03:36:26.672 INFO:teuthology.orchestra.run.smithi092.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 384 kB/s | 90 kB 00:00 2024-04-26T03:36:26.997 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-26T03:36:26.997 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:26.998 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-04-26T03:36:26.998 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2457.gddeaa917.el9.noarch 2024-04-26T03:36:26.998 INFO:teuthology.orchestra.run.smithi028.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T03:36:26.998 INFO:teuthology.orchestra.run.smithi028.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T03:36:26.999 INFO:teuthology.orchestra.run.smithi028.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T03:36:26.999 INFO:teuthology.orchestra.run.smithi028.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T03:36:26.999 INFO:teuthology.orchestra.run.smithi028.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T03:36:26.999 INFO:teuthology.orchestra.run.smithi028.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T03:36:26.999 INFO:teuthology.orchestra.run.smithi028.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T03:36:26.999 INFO:teuthology.orchestra.run.smithi028.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T03:36:27.000 INFO:teuthology.orchestra.run.smithi028.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T03:36:27.000 INFO:teuthology.orchestra.run.smithi028.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T03:36:27.000 INFO:teuthology.orchestra.run.smithi028.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T03:36:27.000 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:27.000 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-04-26T03:36:27.027 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-04-26T03:36:27.202 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-26T03:36:27.259 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-26T03:36:27.315 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-26T03:36:27.351 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-26T03:36:27.356 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install ceph-mgr-rook 2024-04-26T03:36:27.392 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-26T03:36:27.492 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-26T03:36:27.588 INFO:teuthology.orchestra.run.smithi066.stdout:(12/12): ceph-mgr-diskprediction-local-19.0.0-2 2.6 MB/s | 7.4 MB 00:02 2024-04-26T03:36:27.592 INFO:teuthology.orchestra.run.smithi066.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:27.592 INFO:teuthology.orchestra.run.smithi066.stdout:Total 12 MB/s | 38 MB 00:03 2024-04-26T03:36:27.610 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-04-26T03:36:27.662 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-26T03:36:27.864 INFO:teuthology.orchestra.run.smithi092.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 609 kB/s | 1.0 MB 00:01 2024-04-26T03:36:27.866 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:27.866 INFO:teuthology.orchestra.run.smithi092.stdout:Total 901 kB/s | 2.2 MB 00:02 2024-04-26T03:36:27.890 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:28 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:36:27.934 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-04-26T03:36:27.949 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-04-26T03:36:27.949 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-04-26T03:36:27.959 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction check 2024-04-26T03:36:27.986 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction check succeeded. 2024-04-26T03:36:27.987 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction test 2024-04-26T03:36:28.016 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-04-26T03:36:28.017 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:36:28.017 INFO:teuthology.orchestra.run.smithi028.stdout: Package Arch Version Repository Size 2024-04-26T03:36:28.017 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:36:28.017 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-04-26T03:36:28.017 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-rook noarch 2:19.0.0-2457.gddeaa917.el9 ceph-noarch 49 k 2024-04-26T03:36:28.017 INFO:teuthology.orchestra.run.smithi028.stdout:Installing dependencies: 2024-04-26T03:36:28.017 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T03:36:28.017 INFO:teuthology.orchestra.run.smithi028.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T03:36:28.017 INFO:teuthology.orchestra.run.smithi028.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T03:36:28.018 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T03:36:28.018 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T03:36:28.018 INFO:teuthology.orchestra.run.smithi028.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T03:36:28.018 INFO:teuthology.orchestra.run.smithi028.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T03:36:28.018 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T03:36:28.018 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T03:36:28.018 INFO:teuthology.orchestra.run.smithi028.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T03:36:28.018 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T03:36:28.018 INFO:teuthology.orchestra.run.smithi028.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T03:36:28.018 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:28.018 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-04-26T03:36:28.018 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:36:28.018 INFO:teuthology.orchestra.run.smithi028.stdout:Install 13 Packages 2024-04-26T03:36:28.018 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:28.019 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 2.2 M 2024-04-26T03:36:28.019 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 26 M 2024-04-26T03:36:28.019 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-04-26T03:36:28.032 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-04-26T03:36:28.033 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-04-26T03:36:28.421 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction test succeeded. 2024-04-26T03:36:28.421 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction 2024-04-26T03:36:28.425 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-26T03:36:28.475 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-26T03:36:28.508 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-26T03:36:28.545 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-04-26T03:36:28.558 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-26T03:36:28.594 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-rook-2:19.0.0-2457.gddeaa917.el9.noarch 13/13 2024-04-26T03:36:28.703 INFO:teuthology.orchestra.run.smithi028.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 167 kB/s | 26 kB 00:00 2024-04-26T03:36:28.705 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-26T03:36:28.727 INFO:teuthology.orchestra.run.smithi028.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 107 kB/s | 19 kB 00:00 2024-04-26T03:36:28.765 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-26T03:36:28.820 INFO:teuthology.orchestra.run.smithi028.stdout:(3/13): ceph-mgr-rook-19.0.0-2457.gddeaa917.el9 182 kB/s | 49 kB 00:00 2024-04-26T03:36:28.821 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-26T03:36:28.845 INFO:teuthology.orchestra.run.smithi028.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.9 MB/s | 222 kB 00:00 2024-04-26T03:36:28.858 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-26T03:36:28.870 INFO:teuthology.orchestra.run.smithi028.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.1 MB/s | 159 kB 00:00 2024-04-26T03:36:28.895 INFO:teuthology.orchestra.run.smithi028.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 3.6 MB/s | 279 kB 00:00 2024-04-26T03:36:28.899 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-26T03:36:28.921 INFO:teuthology.orchestra.run.smithi028.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 716 kB/s | 54 kB 00:00 2024-04-26T03:36:28.994 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-26T03:36:29.104 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-04-26T03:36:29.160 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-26T03:36:29.288 INFO:teuthology.orchestra.run.smithi028.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 77 kB/s | 32 kB 00:00 2024-04-26T03:36:29.296 INFO:teuthology.orchestra.run.smithi066.stdout: Preparing : 1/1 2024-04-26T03:36:29.313 INFO:teuthology.orchestra.run.smithi028.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 38 kB/s | 15 kB 00:00 2024-04-26T03:36:29.364 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2457.gddeaa917.el9.noarch 13/13 2024-04-26T03:36:29.365 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2457.gddeaa917.el9.noarch 1/13 2024-04-26T03:36:29.365 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-26T03:36:29.365 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-26T03:36:29.365 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-26T03:36:29.365 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-26T03:36:29.365 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-26T03:36:29.365 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-26T03:36:29.365 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-26T03:36:29.365 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-26T03:36:29.365 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-04-26T03:36:29.365 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-26T03:36:29.365 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-26T03:36:29.368 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-26T03:36:29.429 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-26T03:36:29.521 INFO:teuthology.orchestra.run.smithi028.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 284 kB/s | 59 kB 00:00 2024-04-26T03:36:29.537 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-26T03:36:29.585 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-04-26T03:36:29.680 INFO:teuthology.orchestra.run.smithi028.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 567 kB/s | 90 kB 00:00 2024-04-26T03:36:29.682 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-26T03:36:29.682 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:29.682 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-04-26T03:36:29.682 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-rook-2:19.0.0-2457.gddeaa917.el9.noarch 2024-04-26T03:36:29.682 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T03:36:29.683 INFO:teuthology.orchestra.run.smithi042.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T03:36:29.683 INFO:teuthology.orchestra.run.smithi042.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T03:36:29.683 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T03:36:29.683 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T03:36:29.683 INFO:teuthology.orchestra.run.smithi042.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T03:36:29.683 INFO:teuthology.orchestra.run.smithi042.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T03:36:29.683 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T03:36:29.683 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T03:36:29.683 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T03:36:29.683 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T03:36:29.683 INFO:teuthology.orchestra.run.smithi042.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T03:36:29.683 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:29.683 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-04-26T03:36:29.768 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-26T03:36:29.812 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-26T03:36:29.886 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-26T03:36:29.893 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-26T03:36:29.934 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-mgr-cephadm 2024-04-26T03:36:29.939 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-26T03:36:29.947 INFO:teuthology.orchestra.run.smithi028.stdout:(12/13): python3-google-auth-2.29.0-1.el9.noarc 200 kB/s | 206 kB 00:01 2024-04-26T03:36:29.974 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-26T03:36:30.013 INFO:teuthology.orchestra.run.smithi028.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 1.4 MB/s | 1.0 MB 00:00 2024-04-26T03:36:30.019 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:30.019 INFO:teuthology.orchestra.run.smithi028.stdout:Total 1.1 MB/s | 2.2 MB 00:02 2024-04-26T03:36:30.021 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-26T03:36:30.043 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-rook-2:19.0.0-2457.gddeaa917.el9.noarch 13/13 2024-04-26T03:36:30.092 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-04-26T03:36:30.109 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-04-26T03:36:30.109 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-04-26T03:36:30.203 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-04-26T03:36:30.204 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-04-26T03:36:30.462 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:32 ago on Fri 26 Apr 2024 03:34:58 AM UTC. 2024-04-26T03:36:30.587 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-04-26T03:36:30.588 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:36:30.588 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-04-26T03:36:30.588 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:36:30.588 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-04-26T03:36:30.588 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2457.gddeaa917.el9 ceph-noarch 141 k 2024-04-26T03:36:30.588 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-04-26T03:36:30.588 INFO:teuthology.orchestra.run.smithi042.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T03:36:30.588 INFO:teuthology.orchestra.run.smithi042.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T03:36:30.588 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T03:36:30.589 INFO:teuthology.orchestra.run.smithi042.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T03:36:30.589 INFO:teuthology.orchestra.run.smithi042.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T03:36:30.589 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:30.589 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-04-26T03:36:30.589 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:36:30.589 INFO:teuthology.orchestra.run.smithi042.stdout:Install 6 Packages 2024-04-26T03:36:30.589 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:30.590 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-26T03:36:30.590 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 7.0 M 2024-04-26T03:36:30.590 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 33 M 2024-04-26T03:36:30.590 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-04-26T03:36:30.715 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-26T03:36:30.776 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2457.gddeaa917.el9.noarch 13/13 2024-04-26T03:36:30.777 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2457.gddeaa917.el9.noarch 1/13 2024-04-26T03:36:30.777 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-26T03:36:30.777 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-26T03:36:30.777 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-26T03:36:30.777 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-26T03:36:30.777 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-26T03:36:30.777 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-26T03:36:30.777 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-26T03:36:30.777 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-26T03:36:30.777 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-04-26T03:36:30.777 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-26T03:36:30.777 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-26T03:36:30.911 INFO:teuthology.orchestra.run.smithi042.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.4 MB/s | 248 kB 00:00 2024-04-26T03:36:30.944 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-04-26T03:36:31.104 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-26T03:36:31.104 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:31.104 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-04-26T03:36:31.104 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook-2:19.0.0-2457.gddeaa917.el9.noarch 2024-04-26T03:36:31.104 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T03:36:31.104 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T03:36:31.105 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T03:36:31.105 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T03:36:31.105 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T03:36:31.105 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T03:36:31.105 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T03:36:31.105 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T03:36:31.105 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T03:36:31.105 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T03:36:31.105 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T03:36:31.105 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T03:36:31.105 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:31.105 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-26T03:36:31.119 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-26T03:36:31.128 INFO:teuthology.orchestra.run.smithi042.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-04-26T03:36:31.176 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-26T03:36:31.195 INFO:teuthology.orchestra.run.smithi042.stdout:(3/6): ceph-mgr-cephadm-19.0.0-2457.gddeaa917.e 308 kB/s | 141 kB 00:00 2024-04-26T03:36:31.232 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-26T03:36:31.267 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-26T03:36:31.309 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-26T03:36:31.359 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-mgr-cephadm 2024-04-26T03:36:31.407 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-26T03:36:31.527 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-04-26T03:36:31.578 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-26T03:36:31.595 INFO:teuthology.orchestra.run.smithi042.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 124 kB/s | 58 kB 00:00 2024-04-26T03:36:31.612 INFO:teuthology.orchestra.run.smithi042.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 130 kB/s | 51 kB 00:00 2024-04-26T03:36:31.771 INFO:teuthology.orchestra.run.smithi042.stdout:(6/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 639 kB/s | 548 kB 00:00 2024-04-26T03:36:31.775 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:31.775 INFO:teuthology.orchestra.run.smithi042.stdout:Total 5.9 MB/s | 7.0 MB 00:01 2024-04-26T03:36:31.850 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-04-26T03:36:31.863 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-04-26T03:36:31.863 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-04-26T03:36:31.866 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:32 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:36:31.978 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-04-26T03:36:31.979 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-04-26T03:36:31.985 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-26T03:36:31.986 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:36:31.986 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-04-26T03:36:31.986 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:36:31.987 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-04-26T03:36:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2457.gddeaa917.el9 ceph-noarch 141 k 2024-04-26T03:36:31.987 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-04-26T03:36:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T03:36:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T03:36:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T03:36:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T03:36:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T03:36:31.988 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:31.988 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-04-26T03:36:31.988 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:36:31.988 INFO:teuthology.orchestra.run.smithi092.stdout:Install 6 Packages 2024-04-26T03:36:31.988 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:31.988 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 7.0 M 2024-04-26T03:36:31.988 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 33 M 2024-04-26T03:36:31.989 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-04-26T03:36:32.028 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-26T03:36:32.286 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-04-26T03:36:32.344 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-26T03:36:32.390 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-26T03:36:32.392 INFO:teuthology.orchestra.run.smithi092.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.3 MB/s | 248 kB 00:00 2024-04-26T03:36:32.424 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-26T03:36:32.462 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-26T03:36:32.473 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-26T03:36:32.493 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mgr-rook-2:19.0.0-2457.gddeaa917.el9.noarch 13/13 2024-04-26T03:36:32.507 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-26T03:36:32.601 INFO:teuthology.orchestra.run.smithi092.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 2.6 MB/s | 548 kB 00:00 2024-04-26T03:36:32.651 INFO:teuthology.orchestra.run.smithi092.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.2 MB/s | 58 kB 00:00 2024-04-26T03:36:32.743 INFO:teuthology.orchestra.run.smithi092.stdout:(4/6): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-04-26T03:36:32.768 INFO:teuthology.orchestra.run.smithi092.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 438 kB/s | 51 kB 00:00 2024-04-26T03:36:32.877 INFO:teuthology.orchestra.run.smithi092.stdout:(6/6): ceph-mgr-cephadm-19.0.0-2457.gddeaa917.e 213 kB/s | 141 kB 00:00 2024-04-26T03:36:32.879 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:32.879 INFO:teuthology.orchestra.run.smithi092.stdout:Total 7.8 MB/s | 7.0 MB 00:00 2024-04-26T03:36:32.948 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-04-26T03:36:32.959 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-04-26T03:36:32.959 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-04-26T03:36:33.054 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-04-26T03:36:33.054 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-04-26T03:36:33.081 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-26T03:36:33.120 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-26T03:36:33.153 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2457.gddeaa 12/12 2024-04-26T03:36:33.179 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-26T03:36:33.183 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2457.gddeaa917.el9.noarch 13/13 2024-04-26T03:36:33.183 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2457.gddeaa917.el9.noarch 1/13 2024-04-26T03:36:33.183 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-26T03:36:33.183 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-26T03:36:33.183 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-26T03:36:33.183 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-26T03:36:33.183 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-26T03:36:33.183 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-26T03:36:33.184 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-26T03:36:33.184 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-26T03:36:33.184 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-04-26T03:36:33.184 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-26T03:36:33.184 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-26T03:36:33.249 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-26T03:36:33.277 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2457.gddeaa917.el9.noarch 6/6 2024-04-26T03:36:33.355 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-04-26T03:36:33.503 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-26T03:36:33.504 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:33.504 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-04-26T03:36:33.504 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-rook-2:19.0.0-2457.gddeaa917.el9.noarch 2024-04-26T03:36:33.504 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T03:36:33.504 INFO:teuthology.orchestra.run.smithi028.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T03:36:33.504 INFO:teuthology.orchestra.run.smithi028.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T03:36:33.504 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T03:36:33.504 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T03:36:33.504 INFO:teuthology.orchestra.run.smithi028.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T03:36:33.504 INFO:teuthology.orchestra.run.smithi028.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T03:36:33.504 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T03:36:33.504 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T03:36:33.504 INFO:teuthology.orchestra.run.smithi028.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T03:36:33.504 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T03:36:33.505 INFO:teuthology.orchestra.run.smithi028.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T03:36:33.505 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:33.505 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-04-26T03:36:33.526 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-26T03:36:33.578 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-26T03:36:33.753 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install ceph-mgr-cephadm 2024-04-26T03:36:34.031 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2457.gddeaa917.el9.noarch 6/6 2024-04-26T03:36:34.031 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2457.gddeaa917.el9.noarch 1/6 2024-04-26T03:36:34.031 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-26T03:36:34.031 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-26T03:36:34.031 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-26T03:36:34.031 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-26T03:36:34.107 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-26T03:36:34.181 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-26T03:36:34.244 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-26T03:36:34.274 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2457.gddeaa917.el9.noarch 6/6 2024-04-26T03:36:34.279 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:35 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:36:34.372 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-26T03:36:34.372 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:34.372 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-04-26T03:36:34.373 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-cephadm-2:19.0.0-2457.gddeaa917.el9.noarch 2024-04-26T03:36:34.373 INFO:teuthology.orchestra.run.smithi042.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T03:36:34.373 INFO:teuthology.orchestra.run.smithi042.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T03:36:34.373 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T03:36:34.373 INFO:teuthology.orchestra.run.smithi042.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T03:36:34.373 INFO:teuthology.orchestra.run.smithi042.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T03:36:34.373 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:34.373 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-04-26T03:36:34.404 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-04-26T03:36:34.405 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:36:34.405 INFO:teuthology.orchestra.run.smithi028.stdout: Package Arch Version Repository Size 2024-04-26T03:36:34.405 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:36:34.405 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-04-26T03:36:34.405 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2457.gddeaa917.el9 ceph-noarch 141 k 2024-04-26T03:36:34.405 INFO:teuthology.orchestra.run.smithi028.stdout:Installing dependencies: 2024-04-26T03:36:34.405 INFO:teuthology.orchestra.run.smithi028.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T03:36:34.405 INFO:teuthology.orchestra.run.smithi028.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T03:36:34.405 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T03:36:34.405 INFO:teuthology.orchestra.run.smithi028.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T03:36:34.405 INFO:teuthology.orchestra.run.smithi028.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T03:36:34.406 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:34.406 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-04-26T03:36:34.406 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:36:34.406 INFO:teuthology.orchestra.run.smithi028.stdout:Install 6 Packages 2024-04-26T03:36:34.406 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:34.406 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 7.0 M 2024-04-26T03:36:34.406 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 33 M 2024-04-26T03:36:34.407 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-04-26T03:36:34.577 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-fuse 2024-04-26T03:36:34.949 INFO:teuthology.orchestra.run.smithi028.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.3 MB/s | 248 kB 00:00 2024-04-26T03:36:35.100 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2457.gddeaa917.el9.noarch 6/6 2024-04-26T03:36:35.100 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2457.gddeaa917.el9.noarch 1/6 2024-04-26T03:36:35.100 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-26T03:36:35.100 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-26T03:36:35.100 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-26T03:36:35.100 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-26T03:36:35.105 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:37 ago on Fri 26 Apr 2024 03:34:58 AM UTC. 2024-04-26T03:36:35.116 INFO:teuthology.orchestra.run.smithi028.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-04-26T03:36:35.216 INFO:teuthology.orchestra.run.smithi028.stdout:(3/6): ceph-mgr-cephadm-19.0.0-2457.gddeaa917.e 314 kB/s | 141 kB 00:00 2024-04-26T03:36:35.230 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-04-26T03:36:35.231 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:36:35.231 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-04-26T03:36:35.231 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:36:35.231 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-04-26T03:36:35.231 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-fuse x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 890 k 2024-04-26T03:36:35.231 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-04-26T03:36:35.231 INFO:teuthology.orchestra.run.smithi042.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T03:36:35.231 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:35.231 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-04-26T03:36:35.232 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:36:35.232 INFO:teuthology.orchestra.run.smithi042.stdout:Install 2 Packages 2024-04-26T03:36:35.232 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:35.232 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 970 k 2024-04-26T03:36:35.232 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 2.8 M 2024-04-26T03:36:35.232 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-04-26T03:36:35.359 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-26T03:36:35.359 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:35.359 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-04-26T03:36:35.360 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm-2:19.0.0-2457.gddeaa917.el9.noarch 2024-04-26T03:36:35.360 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T03:36:35.360 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T03:36:35.360 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T03:36:35.360 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T03:36:35.360 INFO:teuthology.orchestra.run.smithi092.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T03:36:35.360 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:35.360 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-26T03:36:35.498 INFO:teuthology.orchestra.run.smithi042.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 621 kB/s | 80 kB 00:00 2024-04-26T03:36:35.550 INFO:teuthology.orchestra.run.smithi028.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 133 kB/s | 58 kB 00:00 2024-04-26T03:36:35.600 INFO:teuthology.orchestra.run.smithi028.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 133 kB/s | 51 kB 00:00 2024-04-26T03:36:35.615 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-fuse 2024-04-26T03:36:35.688 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2457.gddeaa 12/12 2024-04-26T03:36:35.688 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2457.gddeaa 1/12 2024-04-26T03:36:35.688 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-26T03:36:35.688 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-26T03:36:35.688 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-26T03:36:35.689 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-26T03:36:35.689 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-26T03:36:35.689 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-26T03:36:35.689 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-26T03:36:35.689 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-04-26T03:36:35.690 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-26T03:36:35.690 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-26T03:36:35.690 INFO:teuthology.orchestra.run.smithi042.stdout:(2/2): ceph-fuse-19.0.0-2457.gddeaa917.el9.x86_ 2.7 MB/s | 890 kB 00:00 2024-04-26T03:36:35.691 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:35.692 INFO:teuthology.orchestra.run.smithi042.stdout:Total 2.1 MB/s | 970 kB 00:00 2024-04-26T03:36:35.703 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-04-26T03:36:35.726 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-04-26T03:36:35.726 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-04-26T03:36:35.809 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-04-26T03:36:35.809 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-04-26T03:36:36.003 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-04-26T03:36:36.098 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-26T03:36:36.098 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:36.098 INFO:teuthology.orchestra.run.smithi066.stdout:Installed: 2024-04-26T03:36:36.098 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2457.gddeaa917.el9.noarch 2024-04-26T03:36:36.099 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T03:36:36.099 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T03:36:36.099 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T03:36:36.099 INFO:teuthology.orchestra.run.smithi066.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T03:36:36.099 INFO:teuthology.orchestra.run.smithi066.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T03:36:36.099 INFO:teuthology.orchestra.run.smithi066.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T03:36:36.099 INFO:teuthology.orchestra.run.smithi066.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T03:36:36.099 INFO:teuthology.orchestra.run.smithi066.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T03:36:36.099 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T03:36:36.099 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T03:36:36.099 INFO:teuthology.orchestra.run.smithi066.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T03:36:36.099 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:36.099 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-04-26T03:36:36.119 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:37 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:36:36.146 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-26T03:36:36.214 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-fuse-2:19.0.0-2457.gddeaa917.el9.x86_64 2/2 2024-04-26T03:36:36.237 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-26T03:36:36.238 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:36:36.238 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-04-26T03:36:36.238 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:36:36.238 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-04-26T03:36:36.238 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 890 k 2024-04-26T03:36:36.238 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-04-26T03:36:36.238 INFO:teuthology.orchestra.run.smithi092.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T03:36:36.238 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:36.238 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-04-26T03:36:36.238 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:36:36.239 INFO:teuthology.orchestra.run.smithi092.stdout:Install 2 Packages 2024-04-26T03:36:36.239 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:36.239 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 970 k 2024-04-26T03:36:36.239 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 2.8 M 2024-04-26T03:36:36.239 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-04-26T03:36:36.242 INFO:teuthology.orchestra.run.smithi028.stdout:(6/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 424 kB/s | 548 kB 00:01 2024-04-26T03:36:36.244 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:36.244 INFO:teuthology.orchestra.run.smithi028.stdout:Total 3.8 MB/s | 7.0 MB 00:01 2024-04-26T03:36:36.320 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-04-26T03:36:36.333 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-04-26T03:36:36.333 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-04-26T03:36:36.437 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-04-26T03:36:36.437 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-04-26T03:36:36.446 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum -y install ceph-mgr-rook 2024-04-26T03:36:36.504 INFO:teuthology.orchestra.run.smithi092.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 636 kB/s | 80 kB 00:00 2024-04-26T03:36:36.809 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-fuse-2:19.0.0-2457.gddeaa917.el9.x86_64 2/2 2024-04-26T03:36:36.810 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-fuse-2:19.0.0-2457.gddeaa917.el9.x86_64 1/2 2024-04-26T03:36:36.862 INFO:teuthology.orchestra.run.smithi092.stdout:(2/2): ceph-fuse-19.0.0-2457.gddeaa917.el9.x86_ 1.8 MB/s | 890 kB 00:00 2024-04-26T03:36:36.863 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:36.864 INFO:teuthology.orchestra.run.smithi092.stdout:Total 1.5 MB/s | 970 kB 00:00 2024-04-26T03:36:36.866 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-04-26T03:36:36.875 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-04-26T03:36:36.897 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-04-26T03:36:36.897 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-04-26T03:36:36.973 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-04-26T03:36:36.974 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-04-26T03:36:36.985 INFO:teuthology.orchestra.run.smithi066.stdout:Last metadata expiration check: 0:01:35 ago on Fri 26 Apr 2024 03:35:01 AM UTC. 2024-04-26T03:36:37.052 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-26T03:36:37.072 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-26T03:36:37.072 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:37.072 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-04-26T03:36:37.072 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-fuse-2:19.0.0-2457.gddeaa917.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-26T03:36:37.072 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:37.072 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-04-26T03:36:37.106 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-26T03:36:37.111 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-04-26T03:36:37.113 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:36:37.113 INFO:teuthology.orchestra.run.smithi066.stdout: Package Arch Version Repository Size 2024-04-26T03:36:37.113 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:36:37.113 INFO:teuthology.orchestra.run.smithi066.stdout:Installing: 2024-04-26T03:36:37.113 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-rook noarch 2:19.0.0-2457.gddeaa917.el9 ceph-noarch 49 k 2024-04-26T03:36:37.113 INFO:teuthology.orchestra.run.smithi066.stdout:Installing dependencies: 2024-04-26T03:36:37.113 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T03:36:37.113 INFO:teuthology.orchestra.run.smithi066.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T03:36:37.113 INFO:teuthology.orchestra.run.smithi066.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T03:36:37.113 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T03:36:37.113 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T03:36:37.113 INFO:teuthology.orchestra.run.smithi066.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T03:36:37.114 INFO:teuthology.orchestra.run.smithi066.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T03:36:37.114 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T03:36:37.114 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T03:36:37.114 INFO:teuthology.orchestra.run.smithi066.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T03:36:37.114 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T03:36:37.114 INFO:teuthology.orchestra.run.smithi066.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T03:36:37.114 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:37.114 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction Summary 2024-04-26T03:36:37.114 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:36:37.114 INFO:teuthology.orchestra.run.smithi066.stdout:Install 13 Packages 2024-04-26T03:36:37.114 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:37.115 INFO:teuthology.orchestra.run.smithi066.stdout:Total download size: 2.2 M 2024-04-26T03:36:37.115 INFO:teuthology.orchestra.run.smithi066.stdout:Installed size: 26 M 2024-04-26T03:36:37.115 INFO:teuthology.orchestra.run.smithi066.stdout:Downloading Packages: 2024-04-26T03:36:37.165 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-04-26T03:36:37.297 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-26T03:36:37.357 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-volume 2024-04-26T03:36:37.359 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-fuse-2:19.0.0-2457.gddeaa917.el9.x86_64 2/2 2024-04-26T03:36:37.666 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-26T03:36:37.750 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-26T03:36:37.804 INFO:teuthology.orchestra.run.smithi066.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 188 kB/s | 26 kB 00:00 2024-04-26T03:36:37.820 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-26T03:36:37.829 INFO:teuthology.orchestra.run.smithi066.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 119 kB/s | 19 kB 00:00 2024-04-26T03:36:37.847 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2457.gddeaa917.el9.noarch 6/6 2024-04-26T03:36:37.901 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:39 ago on Fri 26 Apr 2024 03:34:58 AM UTC. 2024-04-26T03:36:37.904 INFO:teuthology.orchestra.run.smithi066.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.2 MB/s | 222 kB 00:00 2024-04-26T03:36:37.930 INFO:teuthology.orchestra.run.smithi066.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 159 kB 00:00 2024-04-26T03:36:37.955 INFO:teuthology.orchestra.run.smithi066.stdout:(5/13): ceph-mgr-rook-19.0.0-2457.gddeaa917.el9 171 kB/s | 49 kB 00:00 2024-04-26T03:36:37.963 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-fuse-2:19.0.0-2457.gddeaa917.el9.x86_64 2/2 2024-04-26T03:36:37.964 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-fuse-2:19.0.0-2457.gddeaa917.el9.x86_64 1/2 2024-04-26T03:36:37.980 INFO:teuthology.orchestra.run.smithi066.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 3.6 MB/s | 279 kB 00:00 2024-04-26T03:36:38.005 INFO:teuthology.orchestra.run.smithi066.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 715 kB/s | 54 kB 00:00 2024-04-26T03:36:38.029 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-04-26T03:36:38.030 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:36:38.030 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-04-26T03:36:38.030 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:36:38.030 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-04-26T03:36:38.030 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-volume noarch 2:19.0.0-2457.gddeaa917.el9 ceph-noarch 264 k 2024-04-26T03:36:38.030 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-04-26T03:36:38.030 INFO:teuthology.orchestra.run.smithi042.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T03:36:38.030 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:38.030 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-04-26T03:36:38.030 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:36:38.030 INFO:teuthology.orchestra.run.smithi042.stdout:Install 2 Packages 2024-04-26T03:36:38.031 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:38.031 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 341 k 2024-04-26T03:36:38.031 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 1.5 M 2024-04-26T03:36:38.031 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-04-26T03:36:38.131 INFO:teuthology.orchestra.run.smithi066.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 120 kB/s | 15 kB 00:00 2024-04-26T03:36:38.156 INFO:teuthology.orchestra.run.smithi066.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 160 kB/s | 32 kB 00:00 2024-04-26T03:36:38.202 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-26T03:36:38.202 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:38.202 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-04-26T03:36:38.202 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse-2:19.0.0-2457.gddeaa917.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-26T03:36:38.202 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:38.203 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-26T03:36:38.240 INFO:teuthology.orchestra.run.smithi066.stdout:(10/13): python3-google-auth-2.29.0-1.el9.noarc 880 kB/s | 206 kB 00:00 2024-04-26T03:36:38.273 INFO:teuthology.orchestra.run.smithi066.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 505 kB/s | 59 kB 00:00 2024-04-26T03:36:38.307 INFO:teuthology.orchestra.run.smithi066.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.3 MB/s | 90 kB 00:00 2024-04-26T03:36:38.311 INFO:teuthology.orchestra.run.smithi042.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 586 kB/s | 77 kB 00:00 2024-04-26T03:36:38.424 INFO:teuthology.orchestra.run.smithi066.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 3.5 MB/s | 1.0 MB 00:00 2024-04-26T03:36:38.426 INFO:teuthology.orchestra.run.smithi066.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:38.426 INFO:teuthology.orchestra.run.smithi066.stdout:Total 1.7 MB/s | 2.2 MB 00:01 2024-04-26T03:36:38.444 INFO:teuthology.orchestra.run.smithi042.stdout:(2/2): ceph-volume-19.0.0-2457.gddeaa917.el9.no 993 kB/s | 264 kB 00:00 2024-04-26T03:36:38.445 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:38.446 INFO:teuthology.orchestra.run.smithi042.stdout:Total 820 kB/s | 341 kB 00:00 2024-04-26T03:36:38.457 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-04-26T03:36:38.468 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-04-26T03:36:38.468 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-04-26T03:36:38.486 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-volume 2024-04-26T03:36:38.495 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-04-26T03:36:38.496 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction check 2024-04-26T03:36:38.496 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-04-26T03:36:38.512 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction check succeeded. 2024-04-26T03:36:38.512 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction test 2024-04-26T03:36:38.605 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction test succeeded. 2024-04-26T03:36:38.605 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction 2024-04-26T03:36:38.654 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-04-26T03:36:38.726 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2457.gddeaa917.el9.noarch 6/6 2024-04-26T03:36:38.727 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2457.gddeaa917.el9.noarch 1/6 2024-04-26T03:36:38.727 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-26T03:36:38.727 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-26T03:36:38.727 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-26T03:36:38.727 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-26T03:36:38.847 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-04-26T03:36:38.880 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-volume-2:19.0.0-2457.gddeaa917.el9.noarch 2/2 2024-04-26T03:36:38.896 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-volume-2:19.0.0-2457.gddeaa917.el9.noarch 2/2 2024-04-26T03:36:38.896 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T03:36:38.896 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T03:36:38.897 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:38.994 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-26T03:36:38.994 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:38.994 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-04-26T03:36:38.994 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-cephadm-2:19.0.0-2457.gddeaa917.el9.noarch 2024-04-26T03:36:38.994 INFO:teuthology.orchestra.run.smithi028.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T03:36:38.994 INFO:teuthology.orchestra.run.smithi028.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T03:36:38.994 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T03:36:38.994 INFO:teuthology.orchestra.run.smithi028.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T03:36:38.994 INFO:teuthology.orchestra.run.smithi028.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T03:36:38.994 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:38.995 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-04-26T03:36:39.017 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:40 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:36:39.135 INFO:teuthology.orchestra.run.smithi066.stdout: Preparing : 1/1 2024-04-26T03:36:39.140 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-26T03:36:39.141 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:36:39.141 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-04-26T03:36:39.141 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:36:39.141 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-04-26T03:36:39.142 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume noarch 2:19.0.0-2457.gddeaa917.el9 ceph-noarch 264 k 2024-04-26T03:36:39.142 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-04-26T03:36:39.142 INFO:teuthology.orchestra.run.smithi092.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T03:36:39.142 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:39.142 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-04-26T03:36:39.142 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:36:39.142 INFO:teuthology.orchestra.run.smithi092.stdout:Install 2 Packages 2024-04-26T03:36:39.142 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:39.143 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 341 k 2024-04-26T03:36:39.143 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 1.5 M 2024-04-26T03:36:39.143 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-04-26T03:36:39.265 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install ceph-fuse 2024-04-26T03:36:39.314 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-26T03:36:39.351 INFO:teuthology.orchestra.run.smithi092.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 549 kB/s | 77 kB 00:00 2024-04-26T03:36:39.368 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-26T03:36:39.424 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-26T03:36:39.451 INFO:teuthology.orchestra.run.smithi092.stdout:(2/2): ceph-volume-19.0.0-2457.gddeaa917.el9.no 1.1 MB/s | 264 kB 00:00 2024-04-26T03:36:39.453 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:39.453 INFO:teuthology.orchestra.run.smithi092.stdout:Total 1.1 MB/s | 341 kB 00:00 2024-04-26T03:36:39.460 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-26T03:36:39.465 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-04-26T03:36:39.474 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-04-26T03:36:39.475 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-04-26T03:36:39.500 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-04-26T03:36:39.500 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-04-26T03:36:39.502 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-26T03:36:39.576 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-volume-2:19.0.0-2457.gddeaa917.el9.noarch 1/2 2024-04-26T03:36:39.600 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-26T03:36:39.644 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-04-26T03:36:39.711 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-04-26T03:36:39.762 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-26T03:36:39.803 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:40 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:36:39.818 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-04-26T03:36:39.854 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-volume-2:19.0.0-2457.gddeaa917.el9.noarch 2/2 2024-04-26T03:36:39.870 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-volume-2:19.0.0-2457.gddeaa917.el9.noarch 2/2 2024-04-26T03:36:39.870 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T03:36:39.870 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T03:36:39.870 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:39.873 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-04-26T03:36:39.873 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:39.873 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-04-26T03:36:39.873 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-volume-2:19.0.0-2457.gddeaa917.el9.noarch 2024-04-26T03:36:39.873 INFO:teuthology.orchestra.run.smithi042.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T03:36:39.874 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:39.874 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-04-26T03:36:39.928 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-04-26T03:36:39.928 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:36:39.929 INFO:teuthology.orchestra.run.smithi028.stdout: Package Arch Version Repository Size 2024-04-26T03:36:39.929 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:36:39.929 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-04-26T03:36:39.929 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-fuse x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 890 k 2024-04-26T03:36:39.929 INFO:teuthology.orchestra.run.smithi028.stdout:Installing dependencies: 2024-04-26T03:36:39.929 INFO:teuthology.orchestra.run.smithi028.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T03:36:39.929 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:39.929 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-04-26T03:36:39.929 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:36:39.929 INFO:teuthology.orchestra.run.smithi028.stdout:Install 2 Packages 2024-04-26T03:36:39.929 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:39.930 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 970 k 2024-04-26T03:36:39.930 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 2.8 M 2024-04-26T03:36:39.930 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-04-26T03:36:40.203 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install librados-devel 2024-04-26T03:36:40.369 INFO:teuthology.orchestra.run.smithi028.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 574 kB/s | 80 kB 00:00 2024-04-26T03:36:40.506 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-volume-2:19.0.0-2457.gddeaa917.el9.noarch 1/2 2024-04-26T03:36:40.529 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-26T03:36:40.552 INFO:teuthology.orchestra.run.smithi028.stdout:(2/2): ceph-fuse-19.0.0-2457.gddeaa917.el9.x86_ 2.7 MB/s | 890 kB 00:00 2024-04-26T03:36:40.553 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:40.554 INFO:teuthology.orchestra.run.smithi028.stdout:Total 1.5 MB/s | 970 kB 00:00 2024-04-26T03:36:40.566 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-04-26T03:36:40.575 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-26T03:36:40.589 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-04-26T03:36:40.589 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-04-26T03:36:40.608 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-26T03:36:40.657 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-26T03:36:40.671 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-04-26T03:36:40.671 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-04-26T03:36:40.686 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-rook-2:19.0.0-2457.gddeaa917.el9.noarch 13/13 2024-04-26T03:36:40.744 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:42 ago on Fri 26 Apr 2024 03:34:58 AM UTC. 2024-04-26T03:36:40.777 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-04-26T03:36:40.777 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:40.777 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-04-26T03:36:40.777 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume-2:19.0.0-2457.gddeaa917.el9.noarch 2024-04-26T03:36:40.777 INFO:teuthology.orchestra.run.smithi092.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T03:36:40.777 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:40.777 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-26T03:36:40.867 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-04-26T03:36:40.869 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-04-26T03:36:40.869 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:36:40.869 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repo Size 2024-04-26T03:36:40.870 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:36:40.870 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-04-26T03:36:40.870 INFO:teuthology.orchestra.run.smithi042.stdout: librados-devel x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 127 k 2024-04-26T03:36:40.870 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:40.870 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-04-26T03:36:40.870 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:36:40.870 INFO:teuthology.orchestra.run.smithi042.stdout:Install 1 Package 2024-04-26T03:36:40.870 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:40.871 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 127 k 2024-04-26T03:36:40.871 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 456 k 2024-04-26T03:36:40.871 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-04-26T03:36:41.017 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-26T03:36:41.074 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install librados-devel 2024-04-26T03:36:41.086 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-fuse-2:19.0.0-2457.gddeaa917.el9.x86_64 2/2 2024-04-26T03:36:41.242 INFO:teuthology.orchestra.run.smithi042.stdout:librados-devel-19.0.0-2457.gddeaa917.el9.x86_64 341 kB/s | 127 kB 00:00 2024-04-26T03:36:41.243 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:41.243 INFO:teuthology.orchestra.run.smithi042.stdout:Total 340 kB/s | 127 kB 00:00 2024-04-26T03:36:41.243 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-04-26T03:36:41.249 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-04-26T03:36:41.249 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-04-26T03:36:41.317 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-04-26T03:36:41.318 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-04-26T03:36:41.391 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2457.gddeaa917.el9.noarch 13/13 2024-04-26T03:36:41.391 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2457.gddeaa917.el9.noarch 1/13 2024-04-26T03:36:41.391 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-26T03:36:41.391 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-26T03:36:41.391 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-26T03:36:41.391 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-26T03:36:41.391 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-26T03:36:41.391 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-26T03:36:41.392 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-26T03:36:41.392 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-26T03:36:41.392 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-04-26T03:36:41.392 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-26T03:36:41.392 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-26T03:36:41.471 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-04-26T03:36:41.575 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librados-devel-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:41.597 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:42 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:36:41.721 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-26T03:36:41.722 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-26T03:36:41.722 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:41.722 INFO:teuthology.orchestra.run.smithi066.stdout:Installed: 2024-04-26T03:36:41.722 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-rook-2:19.0.0-2457.gddeaa917.el9.noarch 2024-04-26T03:36:41.722 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T03:36:41.722 INFO:teuthology.orchestra.run.smithi066.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T03:36:41.722 INFO:teuthology.orchestra.run.smithi066.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T03:36:41.723 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T03:36:41.723 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T03:36:41.723 INFO:teuthology.orchestra.run.smithi066.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T03:36:41.723 INFO:teuthology.orchestra.run.smithi066.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T03:36:41.723 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T03:36:41.723 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T03:36:41.723 INFO:teuthology.orchestra.run.smithi066.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T03:36:41.723 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T03:36:41.723 INFO:teuthology.orchestra.run.smithi066.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T03:36:41.723 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:41.723 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-04-26T03:36:41.724 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:36:41.724 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repo Size 2024-04-26T03:36:41.724 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:36:41.724 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-04-26T03:36:41.724 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 127 k 2024-04-26T03:36:41.724 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:41.724 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-04-26T03:36:41.724 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:36:41.724 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-04-26T03:36:41.724 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:41.724 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 127 k 2024-04-26T03:36:41.725 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 456 k 2024-04-26T03:36:41.725 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-04-26T03:36:41.741 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-fuse-2:19.0.0-2457.gddeaa917.el9.x86_64 2/2 2024-04-26T03:36:41.741 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-fuse-2:19.0.0-2457.gddeaa917.el9.x86_64 1/2 2024-04-26T03:36:41.931 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum -y install ceph-mgr-cephadm 2024-04-26T03:36:41.995 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-26T03:36:41.995 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:41.995 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-04-26T03:36:41.995 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-fuse-2:19.0.0-2457.gddeaa917.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-26T03:36:41.995 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:41.995 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-04-26T03:36:42.020 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librados-devel-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:42.066 INFO:teuthology.orchestra.run.smithi092.stdout:librados-devel-19.0.0-2457.gddeaa917.el9.x86_64 369 kB/s | 127 kB 00:00 2024-04-26T03:36:42.067 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:42.067 INFO:teuthology.orchestra.run.smithi092.stdout:Total 367 kB/s | 127 kB 00:00 2024-04-26T03:36:42.067 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-04-26T03:36:42.073 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-04-26T03:36:42.073 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-04-26T03:36:42.136 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-04-26T03:36:42.136 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-04-26T03:36:42.240 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install ceph-volume 2024-04-26T03:36:42.243 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librados-devel-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:42.243 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:42.243 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-04-26T03:36:42.243 INFO:teuthology.orchestra.run.smithi042.stdout: librados-devel-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:36:42.243 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:42.243 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-04-26T03:36:42.283 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-04-26T03:36:42.387 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librados-devel-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:42.444 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install libcephfs2 2024-04-26T03:36:42.463 INFO:teuthology.orchestra.run.smithi066.stdout:Last metadata expiration check: 0:01:41 ago on Fri 26 Apr 2024 03:35:01 AM UTC. 2024-04-26T03:36:42.590 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-04-26T03:36:42.591 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:36:42.591 INFO:teuthology.orchestra.run.smithi066.stdout: Package Arch Version Repository Size 2024-04-26T03:36:42.591 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:36:42.591 INFO:teuthology.orchestra.run.smithi066.stdout:Installing: 2024-04-26T03:36:42.592 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2457.gddeaa917.el9 ceph-noarch 141 k 2024-04-26T03:36:42.592 INFO:teuthology.orchestra.run.smithi066.stdout:Installing dependencies: 2024-04-26T03:36:42.592 INFO:teuthology.orchestra.run.smithi066.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T03:36:42.592 INFO:teuthology.orchestra.run.smithi066.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T03:36:42.592 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T03:36:42.592 INFO:teuthology.orchestra.run.smithi066.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T03:36:42.592 INFO:teuthology.orchestra.run.smithi066.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T03:36:42.592 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:42.592 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction Summary 2024-04-26T03:36:42.592 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:36:42.592 INFO:teuthology.orchestra.run.smithi066.stdout:Install 6 Packages 2024-04-26T03:36:42.592 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:42.593 INFO:teuthology.orchestra.run.smithi066.stdout:Total download size: 7.0 M 2024-04-26T03:36:42.593 INFO:teuthology.orchestra.run.smithi066.stdout:Installed size: 33 M 2024-04-26T03:36:42.593 INFO:teuthology.orchestra.run.smithi066.stdout:Downloading Packages: 2024-04-26T03:36:42.766 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados-devel-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:42.773 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:43 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:36:42.898 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-04-26T03:36:42.899 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:36:42.899 INFO:teuthology.orchestra.run.smithi028.stdout: Package Arch Version Repository Size 2024-04-26T03:36:42.899 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:36:42.899 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-04-26T03:36:42.899 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-volume noarch 2:19.0.0-2457.gddeaa917.el9 ceph-noarch 264 k 2024-04-26T03:36:42.899 INFO:teuthology.orchestra.run.smithi028.stdout:Installing dependencies: 2024-04-26T03:36:42.899 INFO:teuthology.orchestra.run.smithi028.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T03:36:42.899 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:42.899 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-04-26T03:36:42.900 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:36:42.900 INFO:teuthology.orchestra.run.smithi028.stdout:Install 2 Packages 2024-04-26T03:36:42.900 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:42.900 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 341 k 2024-04-26T03:36:42.900 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 1.5 M 2024-04-26T03:36:42.900 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-04-26T03:36:42.979 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:44 ago on Fri 26 Apr 2024 03:34:58 AM UTC. 2024-04-26T03:36:43.001 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados-devel-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:43.001 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:43.001 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-04-26T03:36:43.001 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:36:43.001 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:43.001 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-26T03:36:43.060 INFO:teuthology.orchestra.run.smithi042.stdout:Package libcephfs2-2:19.0.0-2457.gddeaa917.el9.x86_64 is already installed. 2024-04-26T03:36:43.103 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-04-26T03:36:43.105 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-04-26T03:36:43.105 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-04-26T03:36:43.142 INFO:teuthology.orchestra.run.smithi066.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.2 MB/s | 248 kB 00:00 2024-04-26T03:36:43.167 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install libcephfs-devel 2024-04-26T03:36:43.173 INFO:teuthology.orchestra.run.smithi028.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 575 kB/s | 77 kB 00:00 2024-04-26T03:36:43.234 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install libcephfs2 2024-04-26T03:36:43.301 INFO:teuthology.orchestra.run.smithi066.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-04-26T03:36:43.410 INFO:teuthology.orchestra.run.smithi066.stdout:(3/6): ceph-mgr-cephadm-19.0.0-2457.gddeaa917.e 297 kB/s | 141 kB 00:00 2024-04-26T03:36:43.507 INFO:teuthology.orchestra.run.smithi028.stdout:(2/2): ceph-volume-19.0.0-2457.gddeaa917.el9.no 563 kB/s | 264 kB 00:00 2024-04-26T03:36:43.508 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:43.509 INFO:teuthology.orchestra.run.smithi028.stdout:Total 560 kB/s | 341 kB 00:00 2024-04-26T03:36:43.520 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-04-26T03:36:43.532 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-04-26T03:36:43.532 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-04-26T03:36:43.560 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-04-26T03:36:43.561 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-04-26T03:36:43.627 INFO:teuthology.orchestra.run.smithi066.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 177 kB/s | 58 kB 00:00 2024-04-26T03:36:43.707 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:45 ago on Fri 26 Apr 2024 03:34:58 AM UTC. 2024-04-26T03:36:43.717 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-04-26T03:36:43.743 INFO:teuthology.orchestra.run.smithi066.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 153 kB/s | 51 kB 00:00 2024-04-26T03:36:43.760 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:44 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:36:43.785 INFO:teuthology.orchestra.run.smithi066.stdout:(6/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 855 kB/s | 548 kB 00:00 2024-04-26T03:36:43.787 INFO:teuthology.orchestra.run.smithi066.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:43.787 INFO:teuthology.orchestra.run.smithi066.stdout:Total 5.8 MB/s | 7.0 MB 00:01 2024-04-26T03:36:43.832 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-04-26T03:36:43.832 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:36:43.833 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repo Size 2024-04-26T03:36:43.833 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:36:43.833 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-04-26T03:36:43.833 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs-devel x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 32 k 2024-04-26T03:36:43.833 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:43.833 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-04-26T03:36:43.833 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:36:43.833 INFO:teuthology.orchestra.run.smithi042.stdout:Install 1 Package 2024-04-26T03:36:43.833 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:43.833 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 32 k 2024-04-26T03:36:43.833 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 142 k 2024-04-26T03:36:43.833 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-04-26T03:36:43.841 INFO:teuthology.orchestra.run.smithi092.stdout:Package libcephfs2-2:19.0.0-2457.gddeaa917.el9.x86_64 is already installed. 2024-04-26T03:36:43.861 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction check 2024-04-26T03:36:43.874 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction check succeeded. 2024-04-26T03:36:43.874 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction test 2024-04-26T03:36:43.883 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-26T03:36:43.885 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-04-26T03:36:43.885 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-26T03:36:43.909 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-04-26T03:36:43.951 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-volume-2:19.0.0-2457.gddeaa917.el9.noarch 2/2 2024-04-26T03:36:43.967 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-volume-2:19.0.0-2457.gddeaa917.el9.noarch 2/2 2024-04-26T03:36:43.967 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T03:36:43.967 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T03:36:43.967 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:43.973 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install libcephfs-devel 2024-04-26T03:36:43.978 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction test succeeded. 2024-04-26T03:36:43.978 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction 2024-04-26T03:36:44.004 INFO:teuthology.orchestra.run.smithi042.stdout:libcephfs-devel-19.0.0-2457.gddeaa917.el9.x86_6 186 kB/s | 32 kB 00:00 2024-04-26T03:36:44.004 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:44.005 INFO:teuthology.orchestra.run.smithi042.stdout:Total 185 kB/s | 32 kB 00:00 2024-04-26T03:36:44.005 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-04-26T03:36:44.008 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-04-26T03:36:44.009 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-04-26T03:36:44.011 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-04-26T03:36:44.011 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-04-26T03:36:44.096 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-04-26T03:36:44.202 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libcephfs-devel-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:44.286 INFO:teuthology.orchestra.run.smithi066.stdout: Preparing : 1/1 2024-04-26T03:36:44.466 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-26T03:36:44.483 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:45 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:36:44.513 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-26T03:36:44.581 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:44.601 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-26T03:36:44.602 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:36:44.602 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repo Size 2024-04-26T03:36:44.602 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:36:44.603 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-04-26T03:36:44.603 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 32 k 2024-04-26T03:36:44.603 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:44.603 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-04-26T03:36:44.603 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:36:44.603 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-04-26T03:36:44.603 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:44.603 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 32 k 2024-04-26T03:36:44.603 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 142 k 2024-04-26T03:36:44.603 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-04-26T03:36:44.713 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-volume-2:19.0.0-2457.gddeaa917.el9.noarch 1/2 2024-04-26T03:36:44.796 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libcephfs-devel-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:44.796 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:44.796 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-04-26T03:36:44.796 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs-devel-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:36:44.796 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:44.797 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-04-26T03:36:44.845 INFO:teuthology.orchestra.run.smithi092.stdout:libcephfs-devel-19.0.0-2457.gddeaa917.el9.x86_6 131 kB/s | 32 kB 00:00 2024-04-26T03:36:44.846 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:44.846 INFO:teuthology.orchestra.run.smithi092.stdout:Total 130 kB/s | 32 kB 00:00 2024-04-26T03:36:44.846 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-04-26T03:36:44.850 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-04-26T03:36:44.850 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-04-26T03:36:44.852 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-04-26T03:36:44.852 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-04-26T03:36:44.931 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-04-26T03:36:44.970 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install librados2 2024-04-26T03:36:45.010 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-04-26T03:36:45.010 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:45.011 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-04-26T03:36:45.011 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-volume-2:19.0.0-2457.gddeaa917.el9.noarch 2024-04-26T03:36:45.011 INFO:teuthology.orchestra.run.smithi028.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T03:36:45.011 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:45.011 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-04-26T03:36:45.040 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs-devel-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:45.079 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-26T03:36:45.168 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-26T03:36:45.239 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-26T03:36:45.266 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2457.gddeaa917.el9.noarch 6/6 2024-04-26T03:36:45.326 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install librados-devel 2024-04-26T03:36:45.367 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:45.500 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:47 ago on Fri 26 Apr 2024 03:34:58 AM UTC. 2024-04-26T03:36:45.573 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs-devel-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:45.573 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:45.573 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-04-26T03:36:45.573 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:36:45.574 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:45.574 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-26T03:36:45.581 INFO:teuthology.orchestra.run.smithi042.stdout:Package librados2-2:19.0.0-2457.gddeaa917.el9.x86_64 is already installed. 2024-04-26T03:36:45.623 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-04-26T03:36:45.624 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-04-26T03:36:45.625 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-04-26T03:36:45.694 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install librbd1 2024-04-26T03:36:45.758 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install librados2 2024-04-26T03:36:45.867 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:46 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:36:45.993 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-04-26T03:36:45.993 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:36:45.993 INFO:teuthology.orchestra.run.smithi028.stdout: Package Arch Version Repo Size 2024-04-26T03:36:45.994 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:36:45.994 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-04-26T03:36:45.994 INFO:teuthology.orchestra.run.smithi028.stdout: librados-devel x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 127 k 2024-04-26T03:36:45.994 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:45.994 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-04-26T03:36:45.994 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:36:45.994 INFO:teuthology.orchestra.run.smithi028.stdout:Install 1 Package 2024-04-26T03:36:45.994 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:45.994 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 127 k 2024-04-26T03:36:45.994 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 456 k 2024-04-26T03:36:45.994 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-04-26T03:36:46.053 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2457.gddeaa917.el9.noarch 6/6 2024-04-26T03:36:46.053 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2457.gddeaa917.el9.noarch 1/6 2024-04-26T03:36:46.054 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-26T03:36:46.054 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-26T03:36:46.054 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-26T03:36:46.054 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-26T03:36:46.226 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:48 ago on Fri 26 Apr 2024 03:34:58 AM UTC. 2024-04-26T03:36:46.240 INFO:teuthology.orchestra.run.smithi028.stdout:librados-devel-19.0.0-2457.gddeaa917.el9.x86_64 515 kB/s | 127 kB 00:00 2024-04-26T03:36:46.241 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:46.241 INFO:teuthology.orchestra.run.smithi028.stdout:Total 512 kB/s | 127 kB 00:00 2024-04-26T03:36:46.242 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-04-26T03:36:46.248 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-04-26T03:36:46.248 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-04-26T03:36:46.268 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:47 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:36:46.308 INFO:teuthology.orchestra.run.smithi042.stdout:Package librbd1-2:19.0.0-2457.gddeaa917.el9.x86_64 is already installed. 2024-04-26T03:36:46.314 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-26T03:36:46.314 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:46.314 INFO:teuthology.orchestra.run.smithi066.stdout:Installed: 2024-04-26T03:36:46.314 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-cephadm-2:19.0.0-2457.gddeaa917.el9.noarch 2024-04-26T03:36:46.314 INFO:teuthology.orchestra.run.smithi066.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T03:36:46.314 INFO:teuthology.orchestra.run.smithi066.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T03:36:46.314 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T03:36:46.314 INFO:teuthology.orchestra.run.smithi066.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T03:36:46.314 INFO:teuthology.orchestra.run.smithi066.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T03:36:46.314 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:46.315 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-04-26T03:36:46.315 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-04-26T03:36:46.315 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-04-26T03:36:46.346 INFO:teuthology.orchestra.run.smithi092.stdout:Package librados2-2:19.0.0-2457.gddeaa917.el9.x86_64 is already installed. 2024-04-26T03:36:46.351 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-04-26T03:36:46.352 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-04-26T03:36:46.352 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-04-26T03:36:46.387 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-26T03:36:46.389 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-04-26T03:36:46.389 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-26T03:36:46.428 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install python3-rados 2024-04-26T03:36:46.455 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install librbd1 2024-04-26T03:36:46.467 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-04-26T03:36:46.524 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum -y install ceph-fuse 2024-04-26T03:36:46.555 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : librados-devel-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:46.942 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: librados-devel-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:46.958 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:47 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:36:46.970 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:48 ago on Fri 26 Apr 2024 03:34:58 AM UTC. 2024-04-26T03:36:47.037 INFO:teuthology.orchestra.run.smithi092.stdout:Package librbd1-2:19.0.0-2457.gddeaa917.el9.x86_64 is already installed. 2024-04-26T03:36:47.052 INFO:teuthology.orchestra.run.smithi042.stdout:Package python3-rados-2:19.0.0-2457.gddeaa917.el9.x86_64 is already installed. 2024-04-26T03:36:47.055 INFO:teuthology.orchestra.run.smithi066.stdout:Last metadata expiration check: 0:01:46 ago on Fri 26 Apr 2024 03:35:01 AM UTC. 2024-04-26T03:36:47.078 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-26T03:36:47.079 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-04-26T03:36:47.079 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-26T03:36:47.095 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-04-26T03:36:47.096 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-04-26T03:36:47.096 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-04-26T03:36:47.143 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install python3-rados 2024-04-26T03:36:47.160 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install python3-rgw 2024-04-26T03:36:47.179 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-04-26T03:36:47.180 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:36:47.180 INFO:teuthology.orchestra.run.smithi066.stdout: Package Arch Version Repository Size 2024-04-26T03:36:47.180 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:36:47.180 INFO:teuthology.orchestra.run.smithi066.stdout:Installing: 2024-04-26T03:36:47.180 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-fuse x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 890 k 2024-04-26T03:36:47.180 INFO:teuthology.orchestra.run.smithi066.stdout:Installing dependencies: 2024-04-26T03:36:47.180 INFO:teuthology.orchestra.run.smithi066.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T03:36:47.180 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:47.180 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction Summary 2024-04-26T03:36:47.180 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:36:47.180 INFO:teuthology.orchestra.run.smithi066.stdout:Install 2 Packages 2024-04-26T03:36:47.180 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:47.181 INFO:teuthology.orchestra.run.smithi066.stdout:Total download size: 970 k 2024-04-26T03:36:47.181 INFO:teuthology.orchestra.run.smithi066.stdout:Installed size: 2.8 M 2024-04-26T03:36:47.181 INFO:teuthology.orchestra.run.smithi066.stdout:Downloading Packages: 2024-04-26T03:36:47.208 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librados-devel-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:47.209 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:47.209 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-04-26T03:36:47.209 INFO:teuthology.orchestra.run.smithi028.stdout: librados-devel-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:36:47.209 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:47.209 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-04-26T03:36:47.378 INFO:teuthology.orchestra.run.smithi066.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 647 kB/s | 80 kB 00:00 2024-04-26T03:36:47.422 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install libcephfs2 2024-04-26T03:36:47.658 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:48 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:36:47.690 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:49 ago on Fri 26 Apr 2024 03:34:58 AM UTC. 2024-04-26T03:36:47.738 INFO:teuthology.orchestra.run.smithi092.stdout:Package python3-rados-2:19.0.0-2457.gddeaa917.el9.x86_64 is already installed. 2024-04-26T03:36:47.762 INFO:teuthology.orchestra.run.smithi066.stdout:(2/2): ceph-fuse-19.0.0-2457.gddeaa917.el9.x86_ 1.7 MB/s | 890 kB 00:00 2024-04-26T03:36:47.763 INFO:teuthology.orchestra.run.smithi066.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:47.763 INFO:teuthology.orchestra.run.smithi066.stdout:Total 1.6 MB/s | 970 kB 00:00 2024-04-26T03:36:47.771 INFO:teuthology.orchestra.run.smithi042.stdout:Package python3-rgw-2:19.0.0-2457.gddeaa917.el9.x86_64 is already installed. 2024-04-26T03:36:47.775 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction check 2024-04-26T03:36:47.779 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-26T03:36:47.780 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-04-26T03:36:47.780 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-26T03:36:47.798 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction check succeeded. 2024-04-26T03:36:47.798 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction test 2024-04-26T03:36:47.814 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-04-26T03:36:47.815 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-04-26T03:36:47.815 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-04-26T03:36:47.845 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install python3-rgw 2024-04-26T03:36:47.875 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install python3-cephfs 2024-04-26T03:36:47.880 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction test succeeded. 2024-04-26T03:36:47.881 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction 2024-04-26T03:36:47.954 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:48 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:36:48.035 INFO:teuthology.orchestra.run.smithi028.stdout:Package libcephfs2-2:19.0.0-2457.gddeaa917.el9.x86_64 is already installed. 2024-04-26T03:36:48.070 INFO:teuthology.orchestra.run.smithi066.stdout: Preparing : 1/1 2024-04-26T03:36:48.077 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-04-26T03:36:48.079 INFO:teuthology.orchestra.run.smithi028.stdout:Nothing to do. 2024-04-26T03:36:48.079 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-04-26T03:36:48.168 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install libcephfs-devel 2024-04-26T03:36:48.210 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-26T03:36:48.279 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-fuse-2:19.0.0-2457.gddeaa917.el9.x86_64 2/2 2024-04-26T03:36:48.356 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:49 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:36:48.410 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:50 ago on Fri 26 Apr 2024 03:34:58 AM UTC. 2024-04-26T03:36:48.435 INFO:teuthology.orchestra.run.smithi092.stdout:Package python3-rgw-2:19.0.0-2457.gddeaa917.el9.x86_64 is already installed. 2024-04-26T03:36:48.475 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-26T03:36:48.476 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-04-26T03:36:48.477 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-26T03:36:48.492 INFO:teuthology.orchestra.run.smithi042.stdout:Package python3-cephfs-2:19.0.0-2457.gddeaa917.el9.x86_64 is already installed. 2024-04-26T03:36:48.536 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-04-26T03:36:48.537 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-04-26T03:36:48.537 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-04-26T03:36:48.539 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install python3-cephfs 2024-04-26T03:36:48.599 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install python3-rbd 2024-04-26T03:36:48.708 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:49 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:36:48.834 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-04-26T03:36:48.835 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:36:48.835 INFO:teuthology.orchestra.run.smithi028.stdout: Package Arch Version Repo Size 2024-04-26T03:36:48.835 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:36:48.835 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-04-26T03:36:48.835 INFO:teuthology.orchestra.run.smithi028.stdout: libcephfs-devel x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 32 k 2024-04-26T03:36:48.836 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:48.836 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-04-26T03:36:48.836 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:36:48.836 INFO:teuthology.orchestra.run.smithi028.stdout:Install 1 Package 2024-04-26T03:36:48.836 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:48.836 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 32 k 2024-04-26T03:36:48.837 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 142 k 2024-04-26T03:36:48.837 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-04-26T03:36:48.875 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-fuse-2:19.0.0-2457.gddeaa917.el9.x86_64 2/2 2024-04-26T03:36:48.876 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-fuse-2:19.0.0-2457.gddeaa917.el9.x86_64 1/2 2024-04-26T03:36:49.051 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:50 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:36:49.084 INFO:teuthology.orchestra.run.smithi028.stdout:libcephfs-devel-19.0.0-2457.gddeaa917.el9.x86_6 128 kB/s | 32 kB 00:00 2024-04-26T03:36:49.085 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:49.085 INFO:teuthology.orchestra.run.smithi028.stdout:Total 127 kB/s | 32 kB 00:00 2024-04-26T03:36:49.085 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-04-26T03:36:49.089 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-04-26T03:36:49.089 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-04-26T03:36:49.091 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-04-26T03:36:49.092 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-04-26T03:36:49.130 INFO:teuthology.orchestra.run.smithi092.stdout:Package python3-cephfs-2:19.0.0-2457.gddeaa917.el9.x86_64 is already installed. 2024-04-26T03:36:49.146 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:51 ago on Fri 26 Apr 2024 03:34:58 AM UTC. 2024-04-26T03:36:49.171 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-26T03:36:49.171 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:49.171 INFO:teuthology.orchestra.run.smithi066.stdout:Installed: 2024-04-26T03:36:49.171 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-fuse-2:19.0.0-2457.gddeaa917.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-26T03:36:49.171 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:49.172 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-04-26T03:36:49.172 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-26T03:36:49.173 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-04-26T03:36:49.173 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-26T03:36:49.177 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-04-26T03:36:49.228 INFO:teuthology.orchestra.run.smithi042.stdout:Package python3-rbd-2:19.0.0-2457.gddeaa917.el9.x86_64 is already installed. 2024-04-26T03:36:49.234 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install python3-rbd 2024-04-26T03:36:49.272 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-04-26T03:36:49.273 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-04-26T03:36:49.274 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-04-26T03:36:49.291 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libcephfs-devel-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:49.332 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install rbd-fuse 2024-04-26T03:36:49.408 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum -y install ceph-volume 2024-04-26T03:36:49.637 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:49.745 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:50 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:36:49.822 INFO:teuthology.orchestra.run.smithi092.stdout:Package python3-rbd-2:19.0.0-2457.gddeaa917.el9.x86_64 is already installed. 2024-04-26T03:36:49.851 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libcephfs-devel-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:49.851 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:49.851 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-04-26T03:36:49.851 INFO:teuthology.orchestra.run.smithi028.stdout: libcephfs-devel-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:36:49.851 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:49.851 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-04-26T03:36:49.866 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-26T03:36:49.866 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-04-26T03:36:49.866 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-26T03:36:49.867 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:51 ago on Fri 26 Apr 2024 03:34:58 AM UTC. 2024-04-26T03:36:49.925 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install rbd-fuse 2024-04-26T03:36:49.952 INFO:teuthology.orchestra.run.smithi066.stdout:Last metadata expiration check: 0:01:48 ago on Fri 26 Apr 2024 03:35:01 AM UTC. 2024-04-26T03:36:49.990 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-04-26T03:36:49.991 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:36:49.991 INFO:teuthology.orchestra.run.smithi042.stdout: Package Architecture Version Repository Size 2024-04-26T03:36:49.991 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:36:49.991 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-04-26T03:36:49.991 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-fuse x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 86 k 2024-04-26T03:36:49.991 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:49.992 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-04-26T03:36:49.992 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:36:49.992 INFO:teuthology.orchestra.run.smithi042.stdout:Install 1 Package 2024-04-26T03:36:49.992 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:49.992 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 86 k 2024-04-26T03:36:49.992 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 231 k 2024-04-26T03:36:49.992 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-04-26T03:36:50.074 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install librados2 2024-04-26T03:36:50.078 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-04-26T03:36:50.078 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:36:50.078 INFO:teuthology.orchestra.run.smithi066.stdout: Package Arch Version Repository Size 2024-04-26T03:36:50.078 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:36:50.079 INFO:teuthology.orchestra.run.smithi066.stdout:Installing: 2024-04-26T03:36:50.079 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-volume noarch 2:19.0.0-2457.gddeaa917.el9 ceph-noarch 264 k 2024-04-26T03:36:50.079 INFO:teuthology.orchestra.run.smithi066.stdout:Installing dependencies: 2024-04-26T03:36:50.079 INFO:teuthology.orchestra.run.smithi066.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T03:36:50.079 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:50.079 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction Summary 2024-04-26T03:36:50.079 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:36:50.079 INFO:teuthology.orchestra.run.smithi066.stdout:Install 2 Packages 2024-04-26T03:36:50.079 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:50.080 INFO:teuthology.orchestra.run.smithi066.stdout:Total download size: 341 k 2024-04-26T03:36:50.080 INFO:teuthology.orchestra.run.smithi066.stdout:Installed size: 1.5 M 2024-04-26T03:36:50.080 INFO:teuthology.orchestra.run.smithi066.stdout:Downloading Packages: 2024-04-26T03:36:50.207 INFO:teuthology.orchestra.run.smithi042.stdout:rbd-fuse-19.0.0-2457.gddeaa917.el9.x86_64.rpm 398 kB/s | 86 kB 00:00 2024-04-26T03:36:50.208 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:50.208 INFO:teuthology.orchestra.run.smithi042.stdout:Total 395 kB/s | 86 kB 00:00 2024-04-26T03:36:50.208 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-04-26T03:36:50.216 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-04-26T03:36:50.216 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-04-26T03:36:50.282 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-04-26T03:36:50.282 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-04-26T03:36:50.349 INFO:teuthology.orchestra.run.smithi066.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 530 kB/s | 77 kB 00:00 2024-04-26T03:36:50.430 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-04-26T03:36:50.431 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:51 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:36:50.539 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : rbd-fuse-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:50.550 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-26T03:36:50.551 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:36:50.551 INFO:teuthology.orchestra.run.smithi092.stdout: Package Architecture Version Repository Size 2024-04-26T03:36:50.551 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:36:50.551 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-04-26T03:36:50.551 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 86 k 2024-04-26T03:36:50.551 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:50.552 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-04-26T03:36:50.552 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:36:50.552 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-04-26T03:36:50.552 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:50.552 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 86 k 2024-04-26T03:36:50.552 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 231 k 2024-04-26T03:36:50.552 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-04-26T03:36:50.610 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:51 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:36:50.650 INFO:teuthology.orchestra.run.smithi066.stdout:(2/2): ceph-volume-19.0.0-2457.gddeaa917.el9.no 591 kB/s | 264 kB 00:00 2024-04-26T03:36:50.651 INFO:teuthology.orchestra.run.smithi066.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:50.652 INFO:teuthology.orchestra.run.smithi066.stdout:Total 595 kB/s | 341 kB 00:00 2024-04-26T03:36:50.663 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction check 2024-04-26T03:36:50.674 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction check succeeded. 2024-04-26T03:36:50.674 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction test 2024-04-26T03:36:50.692 INFO:teuthology.orchestra.run.smithi028.stdout:Package librados2-2:19.0.0-2457.gddeaa917.el9.x86_64 is already installed. 2024-04-26T03:36:50.701 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction test succeeded. 2024-04-26T03:36:50.702 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction 2024-04-26T03:36:50.736 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-04-26T03:36:50.737 INFO:teuthology.orchestra.run.smithi028.stdout:Nothing to do. 2024-04-26T03:36:50.737 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-04-26T03:36:50.806 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install librbd1 2024-04-26T03:36:50.847 INFO:teuthology.orchestra.run.smithi066.stdout: Preparing : 1/1 2024-04-26T03:36:50.883 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: rbd-fuse-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:50.890 INFO:teuthology.orchestra.run.smithi092.stdout:rbd-fuse-19.0.0-2457.gddeaa917.el9.x86_64.rpm 254 kB/s | 86 kB 00:00 2024-04-26T03:36:50.890 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:50.891 INFO:teuthology.orchestra.run.smithi092.stdout:Total 253 kB/s | 86 kB 00:00 2024-04-26T03:36:50.891 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-04-26T03:36:50.897 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-04-26T03:36:50.898 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-04-26T03:36:50.962 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-04-26T03:36:50.962 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-04-26T03:36:51.033 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-04-26T03:36:51.069 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-volume-2:19.0.0-2457.gddeaa917.el9.noarch 2/2 2024-04-26T03:36:51.085 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-volume-2:19.0.0-2457.gddeaa917.el9.noarch 2/2 2024-04-26T03:36:51.085 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T03:36:51.085 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T03:36:51.085 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:51.098 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : rbd-fuse-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:51.098 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:51.098 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-04-26T03:36:51.098 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-fuse-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:36:51.098 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:51.098 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-04-26T03:36:51.103 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-04-26T03:36:51.223 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-fuse-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:51.337 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:52 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:36:51.343 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install rbd-mirror 2024-04-26T03:36:51.420 INFO:teuthology.orchestra.run.smithi028.stdout:Package librbd1-2:19.0.0-2457.gddeaa917.el9.x86_64 is already installed. 2024-04-26T03:36:51.462 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-04-26T03:36:51.464 INFO:teuthology.orchestra.run.smithi028.stdout:Nothing to do. 2024-04-26T03:36:51.464 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-04-26T03:36:51.530 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install python3-rados 2024-04-26T03:36:51.545 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: rbd-fuse-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:51.742 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-volume-2:19.0.0-2457.gddeaa917.el9.noarch 1/2 2024-04-26T03:36:51.824 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-fuse-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:51.824 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:51.824 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-04-26T03:36:51.824 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:36:51.824 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:51.824 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-26T03:36:51.880 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:53 ago on Fri 26 Apr 2024 03:34:58 AM UTC. 2024-04-26T03:36:52.007 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-04-26T03:36:52.007 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:36:52.007 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repo Size 2024-04-26T03:36:52.007 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:36:52.008 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-04-26T03:36:52.008 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-mirror x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 3.1 M 2024-04-26T03:36:52.008 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:52.008 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-04-26T03:36:52.008 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:36:52.008 INFO:teuthology.orchestra.run.smithi042.stdout:Install 1 Package 2024-04-26T03:36:52.008 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:52.008 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 3.1 M 2024-04-26T03:36:52.008 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 13 M 2024-04-26T03:36:52.008 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-04-26T03:36:52.019 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-04-26T03:36:52.020 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:52.020 INFO:teuthology.orchestra.run.smithi066.stdout:Installed: 2024-04-26T03:36:52.020 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-volume-2:19.0.0-2457.gddeaa917.el9.noarch 2024-04-26T03:36:52.020 INFO:teuthology.orchestra.run.smithi066.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T03:36:52.020 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:52.020 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-04-26T03:36:52.065 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install rbd-mirror 2024-04-26T03:36:52.069 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:53 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:36:52.150 INFO:teuthology.orchestra.run.smithi028.stdout:Package python3-rados-2:19.0.0-2457.gddeaa917.el9.x86_64 is already installed. 2024-04-26T03:36:52.193 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-04-26T03:36:52.195 INFO:teuthology.orchestra.run.smithi028.stdout:Nothing to do. 2024-04-26T03:36:52.195 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-04-26T03:36:52.267 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install python3-rgw 2024-04-26T03:36:52.316 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum -y install librados-devel 2024-04-26T03:36:52.594 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:53 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:36:52.718 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-26T03:36:52.718 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:36:52.718 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repo Size 2024-04-26T03:36:52.718 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:36:52.718 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-04-26T03:36:52.718 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 3.1 M 2024-04-26T03:36:52.718 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:52.719 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-04-26T03:36:52.719 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:36:52.719 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-04-26T03:36:52.719 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:52.719 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 3.1 M 2024-04-26T03:36:52.719 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 13 M 2024-04-26T03:36:52.719 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-04-26T03:36:52.801 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:53 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:36:52.854 INFO:teuthology.orchestra.run.smithi066.stdout:Last metadata expiration check: 0:01:51 ago on Fri 26 Apr 2024 03:35:01 AM UTC. 2024-04-26T03:36:52.883 INFO:teuthology.orchestra.run.smithi028.stdout:Package python3-rgw-2:19.0.0-2457.gddeaa917.el9.x86_64 is already installed. 2024-04-26T03:36:52.926 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-04-26T03:36:52.927 INFO:teuthology.orchestra.run.smithi028.stdout:Nothing to do. 2024-04-26T03:36:52.927 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-04-26T03:36:52.979 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-04-26T03:36:52.980 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:36:52.980 INFO:teuthology.orchestra.run.smithi066.stdout: Package Arch Version Repo Size 2024-04-26T03:36:52.980 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:36:52.980 INFO:teuthology.orchestra.run.smithi066.stdout:Installing: 2024-04-26T03:36:52.980 INFO:teuthology.orchestra.run.smithi066.stdout: librados-devel x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 127 k 2024-04-26T03:36:52.980 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:52.980 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction Summary 2024-04-26T03:36:52.980 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:36:52.980 INFO:teuthology.orchestra.run.smithi066.stdout:Install 1 Package 2024-04-26T03:36:52.980 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:52.980 INFO:teuthology.orchestra.run.smithi066.stdout:Total download size: 127 k 2024-04-26T03:36:52.980 INFO:teuthology.orchestra.run.smithi066.stdout:Installed size: 456 k 2024-04-26T03:36:52.981 INFO:teuthology.orchestra.run.smithi066.stdout:Downloading Packages: 2024-04-26T03:36:53.000 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install python3-cephfs 2024-04-26T03:36:53.192 INFO:teuthology.orchestra.run.smithi066.stdout:librados-devel-19.0.0-2457.gddeaa917.el9.x86_64 601 kB/s | 127 kB 00:00 2024-04-26T03:36:53.192 INFO:teuthology.orchestra.run.smithi066.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:53.192 INFO:teuthology.orchestra.run.smithi066.stdout:Total 596 kB/s | 127 kB 00:00 2024-04-26T03:36:53.193 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction check 2024-04-26T03:36:53.199 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction check succeeded. 2024-04-26T03:36:53.199 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction test 2024-04-26T03:36:53.202 INFO:teuthology.orchestra.run.smithi092.stdout:rbd-mirror-19.0.0-2457.gddeaa917.el9.x86_64.rpm 6.4 MB/s | 3.1 MB 00:00 2024-04-26T03:36:53.203 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:53.203 INFO:teuthology.orchestra.run.smithi092.stdout:Total 6.4 MB/s | 3.1 MB 00:00 2024-04-26T03:36:53.203 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-04-26T03:36:53.213 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-04-26T03:36:53.214 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-04-26T03:36:53.265 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction test succeeded. 2024-04-26T03:36:53.266 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction 2024-04-26T03:36:53.296 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-04-26T03:36:53.297 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-04-26T03:36:53.420 INFO:teuthology.orchestra.run.smithi066.stdout: Preparing : 1/1 2024-04-26T03:36:53.521 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librados-devel-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:53.540 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:54 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:36:53.578 INFO:teuthology.orchestra.run.smithi042.stdout:rbd-mirror-19.0.0-2457.gddeaa917.el9.x86_64.rpm 2.0 MB/s | 3.1 MB 00:01 2024-04-26T03:36:53.579 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:53.579 INFO:teuthology.orchestra.run.smithi042.stdout:Total 2.0 MB/s | 3.1 MB 00:01 2024-04-26T03:36:53.580 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-04-26T03:36:53.590 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-04-26T03:36:53.590 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-04-26T03:36:53.623 INFO:teuthology.orchestra.run.smithi028.stdout:Package python3-cephfs-2:19.0.0-2457.gddeaa917.el9.x86_64 is already installed. 2024-04-26T03:36:53.666 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-04-26T03:36:53.667 INFO:teuthology.orchestra.run.smithi028.stdout:Nothing to do. 2024-04-26T03:36:53.667 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-04-26T03:36:53.677 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-04-26T03:36:53.677 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-04-26T03:36:53.736 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-04-26T03:36:53.740 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install python3-rbd 2024-04-26T03:36:53.895 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-mirror-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:53.920 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:53.921 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T03:36:53.921 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T03:36:53.921 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T03:36:53.921 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T03:36:53.921 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:53.958 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librados-devel-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:54.130 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-04-26T03:36:54.196 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : rbd-mirror-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:54.196 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librados-devel-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:54.196 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:54.196 INFO:teuthology.orchestra.run.smithi066.stdout:Installed: 2024-04-26T03:36:54.196 INFO:teuthology.orchestra.run.smithi066.stdout: librados-devel-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:36:54.196 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:54.196 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-04-26T03:36:54.222 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:54.222 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T03:36:54.222 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T03:36:54.223 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T03:36:54.223 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T03:36:54.223 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:54.279 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:55 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:36:54.363 INFO:teuthology.orchestra.run.smithi028.stdout:Package python3-rbd-2:19.0.0-2457.gddeaa917.el9.x86_64 is already installed. 2024-04-26T03:36:54.406 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-04-26T03:36:54.407 INFO:teuthology.orchestra.run.smithi028.stdout:Nothing to do. 2024-04-26T03:36:54.407 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-04-26T03:36:54.423 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum -y install libcephfs2 2024-04-26T03:36:54.484 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install rbd-fuse 2024-04-26T03:36:54.857 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-mirror-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:54.857 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:54.857 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-04-26T03:36:54.857 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:36:54.857 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:54.858 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-26T03:36:54.961 INFO:teuthology.orchestra.run.smithi066.stdout:Last metadata expiration check: 0:01:53 ago on Fri 26 Apr 2024 03:35:01 AM UTC. 2024-04-26T03:36:55.029 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:56 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:36:55.042 INFO:teuthology.orchestra.run.smithi066.stdout:Package libcephfs2-2:19.0.0-2457.gddeaa917.el9.x86_64 is already installed. 2024-04-26T03:36:55.057 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install rbd-nbd 2024-04-26T03:36:55.085 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-04-26T03:36:55.086 INFO:teuthology.orchestra.run.smithi066.stdout:Nothing to do. 2024-04-26T03:36:55.087 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-04-26T03:36:55.155 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-04-26T03:36:55.155 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:36:55.155 INFO:teuthology.orchestra.run.smithi028.stdout: Package Architecture Version Repository Size 2024-04-26T03:36:55.155 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:36:55.155 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-04-26T03:36:55.155 INFO:teuthology.orchestra.run.smithi028.stdout: rbd-fuse x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 86 k 2024-04-26T03:36:55.156 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:55.156 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-04-26T03:36:55.156 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:36:55.156 INFO:teuthology.orchestra.run.smithi028.stdout:Install 1 Package 2024-04-26T03:36:55.156 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:55.156 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 86 k 2024-04-26T03:36:55.157 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 231 k 2024-04-26T03:36:55.157 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-04-26T03:36:55.181 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum -y install libcephfs-devel 2024-04-26T03:36:55.454 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : rbd-mirror-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:55.454 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:55.454 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-04-26T03:36:55.454 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-mirror-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:36:55.454 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:55.454 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-04-26T03:36:55.576 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:56 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:36:55.612 INFO:teuthology.orchestra.run.smithi028.stdout:rbd-fuse-19.0.0-2457.gddeaa917.el9.x86_64.rpm 188 kB/s | 86 kB 00:00 2024-04-26T03:36:55.613 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:55.613 INFO:teuthology.orchestra.run.smithi028.stdout:Total 187 kB/s | 86 kB 00:00 2024-04-26T03:36:55.614 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-04-26T03:36:55.621 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-04-26T03:36:55.621 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-04-26T03:36:55.678 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install rbd-nbd 2024-04-26T03:36:55.688 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-04-26T03:36:55.689 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-04-26T03:36:55.699 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-04-26T03:36:55.700 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:36:55.701 INFO:teuthology.orchestra.run.smithi092.stdout: Package Architecture Version Repository Size 2024-04-26T03:36:55.701 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:36:55.701 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-04-26T03:36:55.701 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 170 k 2024-04-26T03:36:55.701 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:55.701 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-04-26T03:36:55.701 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-04-26T03:36:55.701 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-04-26T03:36:55.701 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:55.701 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 170 k 2024-04-26T03:36:55.701 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 494 k 2024-04-26T03:36:55.702 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-04-26T03:36:55.721 INFO:teuthology.orchestra.run.smithi066.stdout:Last metadata expiration check: 0:01:54 ago on Fri 26 Apr 2024 03:35:01 AM UTC. 2024-04-26T03:36:55.836 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-04-26T03:36:55.846 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-04-26T03:36:55.847 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:36:55.847 INFO:teuthology.orchestra.run.smithi066.stdout: Package Arch Version Repo Size 2024-04-26T03:36:55.847 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:36:55.847 INFO:teuthology.orchestra.run.smithi066.stdout:Installing: 2024-04-26T03:36:55.847 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs-devel x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 32 k 2024-04-26T03:36:55.848 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:55.848 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction Summary 2024-04-26T03:36:55.848 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:36:55.848 INFO:teuthology.orchestra.run.smithi066.stdout:Install 1 Package 2024-04-26T03:36:55.848 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:55.848 INFO:teuthology.orchestra.run.smithi066.stdout:Total download size: 32 k 2024-04-26T03:36:55.848 INFO:teuthology.orchestra.run.smithi066.stdout:Installed size: 142 k 2024-04-26T03:36:55.848 INFO:teuthology.orchestra.run.smithi066.stdout:Downloading Packages: 2024-04-26T03:36:55.928 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : rbd-fuse-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:56.010 INFO:teuthology.orchestra.run.smithi066.stdout:libcephfs-devel-19.0.0-2457.gddeaa917.el9.x86_6 195 kB/s | 32 kB 00:00 2024-04-26T03:36:56.011 INFO:teuthology.orchestra.run.smithi066.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:56.011 INFO:teuthology.orchestra.run.smithi066.stdout:Total 194 kB/s | 32 kB 00:00 2024-04-26T03:36:56.011 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction check 2024-04-26T03:36:56.015 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction check succeeded. 2024-04-26T03:36:56.015 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction test 2024-04-26T03:36:56.017 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction test succeeded. 2024-04-26T03:36:56.018 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction 2024-04-26T03:36:56.073 INFO:teuthology.orchestra.run.smithi092.stdout:rbd-nbd-19.0.0-2457.gddeaa917.el9.x86_64.rpm 458 kB/s | 170 kB 00:00 2024-04-26T03:36:56.073 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:56.074 INFO:teuthology.orchestra.run.smithi092.stdout:Total 456 kB/s | 170 kB 00:00 2024-04-26T03:36:56.074 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-04-26T03:36:56.082 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-04-26T03:36:56.082 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-04-26T03:36:56.096 INFO:teuthology.orchestra.run.smithi066.stdout: Preparing : 1/1 2024-04-26T03:36:56.144 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-04-26T03:36:56.145 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-04-26T03:36:56.201 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libcephfs-devel-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:56.216 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:58 ago on Fri 26 Apr 2024 03:34:58 AM UTC. 2024-04-26T03:36:56.301 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-04-26T03:36:56.313 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: rbd-fuse-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:56.343 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-04-26T03:36:56.344 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:36:56.344 INFO:teuthology.orchestra.run.smithi042.stdout: Package Architecture Version Repository Size 2024-04-26T03:36:56.344 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:36:56.344 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-04-26T03:36:56.344 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-nbd x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 170 k 2024-04-26T03:36:56.345 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:56.345 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-04-26T03:36:56.345 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-04-26T03:36:56.345 INFO:teuthology.orchestra.run.smithi042.stdout:Install 1 Package 2024-04-26T03:36:56.345 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:56.345 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 170 k 2024-04-26T03:36:56.345 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 494 k 2024-04-26T03:36:56.345 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-04-26T03:36:56.414 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-nbd-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:56.536 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:56.558 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : rbd-fuse-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:56.559 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:56.559 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-04-26T03:36:56.559 INFO:teuthology.orchestra.run.smithi028.stdout: rbd-fuse-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:36:56.559 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:56.559 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-04-26T03:36:56.691 INFO:teuthology.orchestra.run.smithi042.stdout:rbd-nbd-19.0.0-2457.gddeaa917.el9.x86_64.rpm 493 kB/s | 170 kB 00:00 2024-04-26T03:36:56.692 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:56.692 INFO:teuthology.orchestra.run.smithi042.stdout:Total 491 kB/s | 170 kB 00:00 2024-04-26T03:36:56.692 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-04-26T03:36:56.700 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-04-26T03:36:56.701 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-04-26T03:36:56.759 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libcephfs-devel-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:56.759 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:56.760 INFO:teuthology.orchestra.run.smithi066.stdout:Installed: 2024-04-26T03:36:56.760 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs-devel-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:36:56.760 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:36:56.760 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-04-26T03:36:56.769 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-04-26T03:36:56.769 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-04-26T03:36:56.779 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: rbd-nbd-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:56.797 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install rbd-mirror 2024-04-26T03:36:56.933 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-04-26T03:36:56.934 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum -y install librados2 2024-04-26T03:36:57.031 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-nbd-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:57.031 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:57.031 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-04-26T03:36:57.031 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:36:57.031 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:36:57.031 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-04-26T03:36:57.032 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : rbd-nbd-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:57.245 DEBUG:teuthology.parallel:result is None 2024-04-26T03:36:57.334 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:58 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:36:57.437 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: rbd-nbd-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:57.460 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-04-26T03:36:57.461 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:36:57.462 INFO:teuthology.orchestra.run.smithi028.stdout: Package Arch Version Repo Size 2024-04-26T03:36:57.462 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:36:57.462 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-04-26T03:36:57.462 INFO:teuthology.orchestra.run.smithi028.stdout: rbd-mirror x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 3.1 M 2024-04-26T03:36:57.462 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:57.462 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-04-26T03:36:57.462 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:36:57.462 INFO:teuthology.orchestra.run.smithi028.stdout:Install 1 Package 2024-04-26T03:36:57.462 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:57.463 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 3.1 M 2024-04-26T03:36:57.463 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 13 M 2024-04-26T03:36:57.463 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-04-26T03:36:57.469 INFO:teuthology.orchestra.run.smithi066.stdout:Last metadata expiration check: 0:01:56 ago on Fri 26 Apr 2024 03:35:01 AM UTC. 2024-04-26T03:36:57.552 INFO:teuthology.orchestra.run.smithi066.stdout:Package librados2-2:19.0.0-2457.gddeaa917.el9.x86_64 is already installed. 2024-04-26T03:36:57.595 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-04-26T03:36:57.596 INFO:teuthology.orchestra.run.smithi066.stdout:Nothing to do. 2024-04-26T03:36:57.596 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-04-26T03:36:57.664 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum -y install librbd1 2024-04-26T03:36:57.694 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : rbd-nbd-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:57.694 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:57.694 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-04-26T03:36:57.694 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-nbd-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:36:57.694 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:36:57.694 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-04-26T03:36:57.930 DEBUG:teuthology.parallel:result is None 2024-04-26T03:36:58.195 INFO:teuthology.orchestra.run.smithi066.stdout:Last metadata expiration check: 0:01:57 ago on Fri 26 Apr 2024 03:35:01 AM UTC. 2024-04-26T03:36:58.279 INFO:teuthology.orchestra.run.smithi066.stdout:Package librbd1-2:19.0.0-2457.gddeaa917.el9.x86_64 is already installed. 2024-04-26T03:36:58.323 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-04-26T03:36:58.324 INFO:teuthology.orchestra.run.smithi066.stdout:Nothing to do. 2024-04-26T03:36:58.324 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-04-26T03:36:58.400 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum -y install python3-rados 2024-04-26T03:36:58.415 INFO:teuthology.orchestra.run.smithi028.stdout:rbd-mirror-19.0.0-2457.gddeaa917.el9.x86_64.rpm 3.2 MB/s | 3.1 MB 00:00 2024-04-26T03:36:58.415 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:36:58.416 INFO:teuthology.orchestra.run.smithi028.stdout:Total 3.2 MB/s | 3.1 MB 00:00 2024-04-26T03:36:58.416 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-04-26T03:36:58.427 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-04-26T03:36:58.427 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-04-26T03:36:58.515 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-04-26T03:36:58.515 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-04-26T03:36:58.938 INFO:teuthology.orchestra.run.smithi066.stdout:Last metadata expiration check: 0:01:57 ago on Fri 26 Apr 2024 03:35:01 AM UTC. 2024-04-26T03:36:58.962 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-04-26T03:36:59.015 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : rbd-mirror-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:59.020 INFO:teuthology.orchestra.run.smithi066.stdout:Package python3-rados-2:19.0.0-2457.gddeaa917.el9.x86_64 is already installed. 2024-04-26T03:36:59.042 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:36:59.042 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T03:36:59.043 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T03:36:59.043 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T03:36:59.043 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T03:36:59.043 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:36:59.064 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-04-26T03:36:59.065 INFO:teuthology.orchestra.run.smithi066.stdout:Nothing to do. 2024-04-26T03:36:59.065 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-04-26T03:36:59.141 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum -y install python3-rgw 2024-04-26T03:36:59.674 INFO:teuthology.orchestra.run.smithi066.stdout:Last metadata expiration check: 0:01:58 ago on Fri 26 Apr 2024 03:35:01 AM UTC. 2024-04-26T03:36:59.756 INFO:teuthology.orchestra.run.smithi066.stdout:Package python3-rgw-2:19.0.0-2457.gddeaa917.el9.x86_64 is already installed. 2024-04-26T03:36:59.799 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-04-26T03:36:59.800 INFO:teuthology.orchestra.run.smithi066.stdout:Nothing to do. 2024-04-26T03:36:59.800 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-04-26T03:36:59.871 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum -y install python3-cephfs 2024-04-26T03:37:00.123 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : rbd-mirror-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:37:00.123 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:37:00.124 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-04-26T03:37:00.124 INFO:teuthology.orchestra.run.smithi028.stdout: rbd-mirror-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:37:00.124 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:37:00.124 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-04-26T03:37:00.360 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install rbd-nbd 2024-04-26T03:37:00.410 INFO:teuthology.orchestra.run.smithi066.stdout:Last metadata expiration check: 0:01:59 ago on Fri 26 Apr 2024 03:35:01 AM UTC. 2024-04-26T03:37:00.492 INFO:teuthology.orchestra.run.smithi066.stdout:Package python3-cephfs-2:19.0.0-2457.gddeaa917.el9.x86_64 is already installed. 2024-04-26T03:37:00.536 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-04-26T03:37:00.537 INFO:teuthology.orchestra.run.smithi066.stdout:Nothing to do. 2024-04-26T03:37:00.537 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-04-26T03:37:00.605 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum -y install python3-rbd 2024-04-26T03:37:00.897 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:02:01 ago on Fri 26 Apr 2024 03:34:59 AM UTC. 2024-04-26T03:37:01.024 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-04-26T03:37:01.025 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:37:01.025 INFO:teuthology.orchestra.run.smithi028.stdout: Package Architecture Version Repository Size 2024-04-26T03:37:01.025 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:37:01.025 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-04-26T03:37:01.025 INFO:teuthology.orchestra.run.smithi028.stdout: rbd-nbd x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 170 k 2024-04-26T03:37:01.025 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:37:01.025 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-04-26T03:37:01.025 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-04-26T03:37:01.026 INFO:teuthology.orchestra.run.smithi028.stdout:Install 1 Package 2024-04-26T03:37:01.026 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:37:01.026 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 170 k 2024-04-26T03:37:01.026 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 494 k 2024-04-26T03:37:01.026 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-04-26T03:37:01.141 INFO:teuthology.orchestra.run.smithi066.stdout:Last metadata expiration check: 0:02:00 ago on Fri 26 Apr 2024 03:35:01 AM UTC. 2024-04-26T03:37:01.224 INFO:teuthology.orchestra.run.smithi066.stdout:Package python3-rbd-2:19.0.0-2457.gddeaa917.el9.x86_64 is already installed. 2024-04-26T03:37:01.251 INFO:teuthology.orchestra.run.smithi028.stdout:rbd-nbd-19.0.0-2457.gddeaa917.el9.x86_64.rpm 756 kB/s | 170 kB 00:00 2024-04-26T03:37:01.252 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:37:01.252 INFO:teuthology.orchestra.run.smithi028.stdout:Total 752 kB/s | 170 kB 00:00 2024-04-26T03:37:01.252 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-04-26T03:37:01.261 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-04-26T03:37:01.261 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-04-26T03:37:01.267 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-04-26T03:37:01.268 INFO:teuthology.orchestra.run.smithi066.stdout:Nothing to do. 2024-04-26T03:37:01.269 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-04-26T03:37:01.328 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-04-26T03:37:01.328 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-04-26T03:37:01.340 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum -y install rbd-fuse 2024-04-26T03:37:01.487 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-04-26T03:37:01.592 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : rbd-nbd-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:37:01.872 INFO:teuthology.orchestra.run.smithi066.stdout:Last metadata expiration check: 0:02:00 ago on Fri 26 Apr 2024 03:35:01 AM UTC. 2024-04-26T03:37:01.998 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-04-26T03:37:01.999 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:37:01.999 INFO:teuthology.orchestra.run.smithi066.stdout: Package Architecture Version Repository Size 2024-04-26T03:37:01.999 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:37:01.999 INFO:teuthology.orchestra.run.smithi066.stdout:Installing: 2024-04-26T03:37:01.999 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-fuse x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 86 k 2024-04-26T03:37:01.999 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:37:01.999 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction Summary 2024-04-26T03:37:01.999 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:37:01.999 INFO:teuthology.orchestra.run.smithi066.stdout:Install 1 Package 2024-04-26T03:37:01.999 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:37:02.000 INFO:teuthology.orchestra.run.smithi066.stdout:Total download size: 86 k 2024-04-26T03:37:02.000 INFO:teuthology.orchestra.run.smithi066.stdout:Installed size: 231 k 2024-04-26T03:37:02.000 INFO:teuthology.orchestra.run.smithi066.stdout:Downloading Packages: 2024-04-26T03:37:02.003 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: rbd-nbd-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:37:02.257 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : rbd-nbd-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:37:02.257 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:37:02.257 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-04-26T03:37:02.257 INFO:teuthology.orchestra.run.smithi028.stdout: rbd-nbd-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:37:02.258 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:37:02.258 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-04-26T03:37:02.322 INFO:teuthology.orchestra.run.smithi066.stdout:rbd-fuse-19.0.0-2457.gddeaa917.el9.x86_64.rpm 266 kB/s | 86 kB 00:00 2024-04-26T03:37:02.323 INFO:teuthology.orchestra.run.smithi066.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:37:02.323 INFO:teuthology.orchestra.run.smithi066.stdout:Total 265 kB/s | 86 kB 00:00 2024-04-26T03:37:02.323 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction check 2024-04-26T03:37:02.330 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction check succeeded. 2024-04-26T03:37:02.330 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction test 2024-04-26T03:37:02.398 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction test succeeded. 2024-04-26T03:37:02.398 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction 2024-04-26T03:37:02.537 DEBUG:teuthology.parallel:result is None 2024-04-26T03:37:02.550 INFO:teuthology.orchestra.run.smithi066.stdout: Preparing : 1/1 2024-04-26T03:37:02.637 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : rbd-fuse-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:37:02.991 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: rbd-fuse-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:37:03.268 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : rbd-fuse-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:37:03.268 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:37:03.268 INFO:teuthology.orchestra.run.smithi066.stdout:Installed: 2024-04-26T03:37:03.268 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-fuse-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:37:03.268 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:37:03.268 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-04-26T03:37:03.517 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum -y install rbd-mirror 2024-04-26T03:37:04.074 INFO:teuthology.orchestra.run.smithi066.stdout:Last metadata expiration check: 0:02:03 ago on Fri 26 Apr 2024 03:35:01 AM UTC. 2024-04-26T03:37:04.202 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-04-26T03:37:04.203 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:37:04.203 INFO:teuthology.orchestra.run.smithi066.stdout: Package Arch Version Repo Size 2024-04-26T03:37:04.203 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:37:04.203 INFO:teuthology.orchestra.run.smithi066.stdout:Installing: 2024-04-26T03:37:04.203 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-mirror x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 3.1 M 2024-04-26T03:37:04.203 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:37:04.203 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction Summary 2024-04-26T03:37:04.203 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:37:04.203 INFO:teuthology.orchestra.run.smithi066.stdout:Install 1 Package 2024-04-26T03:37:04.204 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:37:04.204 INFO:teuthology.orchestra.run.smithi066.stdout:Total download size: 3.1 M 2024-04-26T03:37:04.204 INFO:teuthology.orchestra.run.smithi066.stdout:Installed size: 13 M 2024-04-26T03:37:04.204 INFO:teuthology.orchestra.run.smithi066.stdout:Downloading Packages: 2024-04-26T03:37:04.659 INFO:teuthology.orchestra.run.smithi066.stdout:rbd-mirror-19.0.0-2457.gddeaa917.el9.x86_64.rpm 6.8 MB/s | 3.1 MB 00:00 2024-04-26T03:37:04.659 INFO:teuthology.orchestra.run.smithi066.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:37:04.660 INFO:teuthology.orchestra.run.smithi066.stdout:Total 6.8 MB/s | 3.1 MB 00:00 2024-04-26T03:37:04.660 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction check 2024-04-26T03:37:04.670 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction check succeeded. 2024-04-26T03:37:04.670 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction test 2024-04-26T03:37:04.755 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction test succeeded. 2024-04-26T03:37:04.755 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction 2024-04-26T03:37:05.207 INFO:teuthology.orchestra.run.smithi066.stdout: Preparing : 1/1 2024-04-26T03:37:05.276 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : rbd-mirror-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:37:05.303 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:37:05.303 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T03:37:05.303 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T03:37:05.303 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T03:37:05.304 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T03:37:05.304 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:37:06.310 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : rbd-mirror-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:37:06.310 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:37:06.310 INFO:teuthology.orchestra.run.smithi066.stdout:Installed: 2024-04-26T03:37:06.310 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-mirror-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:37:06.310 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:37:06.310 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-04-26T03:37:06.520 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum -y install rbd-nbd 2024-04-26T03:37:07.055 INFO:teuthology.orchestra.run.smithi066.stdout:Last metadata expiration check: 0:02:06 ago on Fri 26 Apr 2024 03:35:01 AM UTC. 2024-04-26T03:37:07.179 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-04-26T03:37:07.179 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:37:07.179 INFO:teuthology.orchestra.run.smithi066.stdout: Package Architecture Version Repository Size 2024-04-26T03:37:07.179 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:37:07.179 INFO:teuthology.orchestra.run.smithi066.stdout:Installing: 2024-04-26T03:37:07.179 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-nbd x86_64 2:19.0.0-2457.gddeaa917.el9 ceph 170 k 2024-04-26T03:37:07.179 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:37:07.180 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction Summary 2024-04-26T03:37:07.180 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-04-26T03:37:07.180 INFO:teuthology.orchestra.run.smithi066.stdout:Install 1 Package 2024-04-26T03:37:07.180 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:37:07.180 INFO:teuthology.orchestra.run.smithi066.stdout:Total download size: 170 k 2024-04-26T03:37:07.180 INFO:teuthology.orchestra.run.smithi066.stdout:Installed size: 494 k 2024-04-26T03:37:07.180 INFO:teuthology.orchestra.run.smithi066.stdout:Downloading Packages: 2024-04-26T03:37:07.404 INFO:teuthology.orchestra.run.smithi066.stdout:rbd-nbd-19.0.0-2457.gddeaa917.el9.x86_64.rpm 765 kB/s | 170 kB 00:00 2024-04-26T03:37:07.404 INFO:teuthology.orchestra.run.smithi066.stdout:-------------------------------------------------------------------------------- 2024-04-26T03:37:07.404 INFO:teuthology.orchestra.run.smithi066.stdout:Total 760 kB/s | 170 kB 00:00 2024-04-26T03:37:07.405 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction check 2024-04-26T03:37:07.413 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction check succeeded. 2024-04-26T03:37:07.414 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction test 2024-04-26T03:37:07.481 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction test succeeded. 2024-04-26T03:37:07.481 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction 2024-04-26T03:37:07.640 INFO:teuthology.orchestra.run.smithi066.stdout: Preparing : 1/1 2024-04-26T03:37:07.753 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : rbd-nbd-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:37:08.130 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: rbd-nbd-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:37:08.369 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : rbd-nbd-2:19.0.0-2457.gddeaa917.el9.x86_64 1/1 2024-04-26T03:37:08.369 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:37:08.369 INFO:teuthology.orchestra.run.smithi066.stdout:Installed: 2024-04-26T03:37:08.369 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-nbd-2:19.0.0-2457.gddeaa917.el9.x86_64 2024-04-26T03:37:08.369 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:37:08.369 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-04-26T03:37:08.623 DEBUG:teuthology.parallel:result is None 2024-04-26T03:37:08.623 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ddeaa91797339a19172c3036ff48cca58c12f448 2024-04-26T03:37:08.863 DEBUG:teuthology.orchestra.run.smithi028:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T03:37:08.889 INFO:teuthology.orchestra.run.smithi028.stdout:19.0.0-2457.gddeaa917.el9 2024-04-26T03:37:08.889 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2457.gddeaa917.el9 2024-04-26T03:37:08.889 INFO:teuthology.task.install:The correct ceph version 19.0.0-2457.gddeaa917 is installed. 2024-04-26T03:37:08.891 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ddeaa91797339a19172c3036ff48cca58c12f448 2024-04-26T03:37:09.034 DEBUG:teuthology.orchestra.run.smithi042:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T03:37:09.059 INFO:teuthology.orchestra.run.smithi042.stdout:19.0.0-2457.gddeaa917.el9 2024-04-26T03:37:09.059 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2457.gddeaa917.el9 2024-04-26T03:37:09.059 INFO:teuthology.task.install:The correct ceph version 19.0.0-2457.gddeaa917 is installed. 2024-04-26T03:37:09.061 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ddeaa91797339a19172c3036ff48cca58c12f448 2024-04-26T03:37:09.326 DEBUG:teuthology.orchestra.run.smithi066:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T03:37:09.349 INFO:teuthology.orchestra.run.smithi066.stdout:19.0.0-2457.gddeaa917.el9 2024-04-26T03:37:09.350 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2457.gddeaa917.el9 2024-04-26T03:37:09.350 INFO:teuthology.task.install:The correct ceph version 19.0.0-2457.gddeaa917 is installed. 2024-04-26T03:37:09.352 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ddeaa91797339a19172c3036ff48cca58c12f448 2024-04-26T03:37:09.646 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T03:37:09.671 INFO:teuthology.orchestra.run.smithi092.stdout:19.0.0-2457.gddeaa917.el9 2024-04-26T03:37:09.671 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2457.gddeaa917.el9 2024-04-26T03:37:09.672 INFO:teuthology.task.install:The correct ceph version 19.0.0-2457.gddeaa917 is installed. 2024-04-26T03:37:09.674 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-26T03:37:09.674 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T03:37:09.674 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T03:37:09.706 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-26T03:37:09.706 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T03:37:09.737 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-26T03:37:09.737 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T03:37:09.766 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-26T03:37:09.766 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T03:37:09.795 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-26T03:37:09.795 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T03:37:09.796 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T03:37:09.825 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T03:37:09.895 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-26T03:37:09.896 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T03:37:09.927 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T03:37:09.998 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-26T03:37:09.998 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T03:37:10.029 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T03:37:10.098 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-26T03:37:10.098 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T03:37:10.125 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T03:37:10.193 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-26T03:37:10.193 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T03:37:10.193 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T03:37:10.222 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T03:37:10.292 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-26T03:37:10.292 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T03:37:10.322 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T03:37:10.391 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-26T03:37:10.391 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T03:37:10.420 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T03:37:10.490 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-26T03:37:10.490 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T03:37:10.517 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T03:37:10.583 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-26T03:37:10.584 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T03:37:10.584 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T03:37:10.613 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T03:37:10.681 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-26T03:37:10.682 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T03:37:10.711 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T03:37:10.780 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-26T03:37:10.780 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T03:37:10.809 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T03:37:10.878 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-26T03:37:10.878 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T03:37:10.905 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T03:37:10.972 INFO:teuthology.run_tasks:Running task ceph... 2024-04-26T03:37:11.059 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-26T03:37:11.059 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 777 /var/log/ceph 2024-04-26T03:37:11.062 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 777 /var/log/ceph 2024-04-26T03:37:11.065 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 777 /var/log/ceph 2024-04-26T03:37:11.068 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 777 /var/log/ceph 2024-04-26T03:37:11.095 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-26T03:37:11.095 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-26T03:37:11.131 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-26T03:37:11.134 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-26T03:37:11.137 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-26T03:37:11.162 INFO:tasks.ceph:Creating extra log directories... 2024-04-26T03:37:11.163 DEBUG:teuthology.orchestra.run.smithi028:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-26T03:37:11.200 DEBUG:teuthology.orchestra.run.smithi042:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-26T03:37:11.203 DEBUG:teuthology.orchestra.run.smithi066:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-26T03:37:11.205 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-26T03:37:11.245 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-26T03:37:11.246 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 2500, 'ms tcp read timeout': 5}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-26T03:37:11.246 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi/7674443', 'branch': 'wip-yuri2-testing-2024-04-24-0914-squid', 'description': 'rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{default} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7674443', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-yuri2-testing-2024-04-24-0914-squid'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 2500, 'ms tcp read timeout': 5}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED'], 'sha1': 'ddeaa91797339a19172c3036ff48cca58c12f448'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'ddeaa91797339a19172c3036ff48cca58c12f448'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri2-testing-2024-04-24-0914-squid', 'sha1': 'ddeaa91797339a19172c3036ff48cca58c12f448'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 99, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'osd.0', 'osd.4', 'osd.8', 'osd.12'], ['mon.b', 'osd.1', 'osd.5', 'osd.9', 'osd.13'], ['mon.c', 'osd.2', 'osd.6', 'osd.10', 'osd.14'], ['mgr.x', 'osd.3', 'osd.7', 'osd.11', 'osd.15', 'client.0']], 'seed': 1848, 'sha1': 'ddeaa91797339a19172c3036ff48cca58c12f448', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rados', 'suite_branch': 'wip-yuri2-testing-2024-04-24-0914-squid', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_ddeaa91797339a19172c3036ff48cca58c12f448/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'ddeaa91797339a19172c3036ff48cca58c12f448', 'targets': {'smithi028.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMGSA8wL+tzr81Ty8AFOYZ/VZPP0hQA/31h6+l9a3of3/3q7foqrrN1IEF7uwrbRWhaHalBEgBrqInqu0oYTjDg=', 'smithi042.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLmTDwfubsCA2qsSwg0HoXpo1FxmcM1ACY9Kyrv1mcT2zdktDFxFCg1Z5/UtTfRJTkTC+gK6Arv1lFcFejRR67o=', 'smithi066.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJrHq4izvgILDjyK2UbhRKaS+aQoC3LdnaJp7pcHbiE2W+Md+Hn0aQJtzgBciGIqEvVj5cMwdWhIjMrle/rN0HM=', 'smithi092.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCBimfGZaOgOmcVYzVt14hT9zje/dFFyZ+b2S7egISjZzQp49FrINmAsYOOeOaFUs4m7HgNIMUR6moCu3clamJw='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 2500, 'ms tcp read timeout': 5}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED'], 'sha1': 'ddeaa91797339a19172c3036ff48cca58c12f448', 'cluster': 'ceph'}}, {'thrashosds': {'aggressive_pg_num_changes': False, 'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'min_in': 8, 'timeout': 1200}}, {'rados': {'clients': ['client.0'], 'ec_pool': True, 'erasure_code_profile': {'c': 2, 'crush-failure-domain': 'osd', 'k': 4, 'm': 3, 'name': 'shecprofile', 'plugin': 'shec'}, 'objects': 50, 'op_weights': {'append': 100, 'copy_from': 50, 'delete': 50, 'read': 100, 'rmattr': 25, 'rollback': 50, 'setattr': 25, 'snap_create': 50, 'snap_remove': 50, 'write': 0}, 'ops': 400, 'write_append_excl': False}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-26_01:23:56', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-26T03:37:11.246 DEBUG:teuthology.orchestra.run.smithi028:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-26T03:37:11.268 DEBUG:teuthology.orchestra.run.smithi042:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-26T03:37:11.272 DEBUG:teuthology.orchestra.run.smithi066:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-26T03:37:11.274 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-26T03:37:11.290 DEBUG:teuthology.orchestra.run.smithi028:> sudo install -d -m0777 -- /var/run/ceph 2024-04-26T03:37:11.325 DEBUG:teuthology.orchestra.run.smithi042:> sudo install -d -m0777 -- /var/run/ceph 2024-04-26T03:37:11.331 DEBUG:teuthology.orchestra.run.smithi066:> sudo install -d -m0777 -- /var/run/ceph 2024-04-26T03:37:11.333 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m0777 -- /var/run/ceph 2024-04-26T03:37:11.360 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T03:37:11.360 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T03:37:11.407 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T03:37:11.407 DEBUG:teuthology.orchestra.run.smithi028:> stat /dev/vg_nvme/lv_1 2024-04-26T03:37:11.463 INFO:teuthology.orchestra.run.smithi028.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T03:37:11.464 INFO:teuthology.orchestra.run.smithi028.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T03:37:11.464 INFO:teuthology.orchestra.run.smithi028.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-26T03:37:11.464 INFO:teuthology.orchestra.run.smithi028.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T03:37:11.464 INFO:teuthology.orchestra.run.smithi028.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T03:37:11.464 INFO:teuthology.orchestra.run.smithi028.stdout:Access: 2024-04-26 03:36:59.293923829 +0000 2024-04-26T03:37:11.464 INFO:teuthology.orchestra.run.smithi028.stdout:Modify: 2024-04-26 03:34:18.316495953 +0000 2024-04-26T03:37:11.464 INFO:teuthology.orchestra.run.smithi028.stdout:Change: 2024-04-26 03:34:18.316495953 +0000 2024-04-26T03:37:11.464 INFO:teuthology.orchestra.run.smithi028.stdout: Birth: 2024-04-26 03:34:18.316495953 +0000 2024-04-26T03:37:11.464 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T03:37:11.528 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records in 2024-04-26T03:37:11.528 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records out 2024-04-26T03:37:11.528 INFO:teuthology.orchestra.run.smithi028.stderr:512 bytes copied, 0.000110075 s, 4.7 MB/s 2024-04-26T03:37:11.530 DEBUG:teuthology.orchestra.run.smithi028:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T03:37:11.588 DEBUG:teuthology.orchestra.run.smithi028:> stat /dev/vg_nvme/lv_2 2024-04-26T03:37:11.645 INFO:teuthology.orchestra.run.smithi028.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T03:37:11.645 INFO:teuthology.orchestra.run.smithi028.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T03:37:11.645 INFO:teuthology.orchestra.run.smithi028.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-26T03:37:11.645 INFO:teuthology.orchestra.run.smithi028.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T03:37:11.645 INFO:teuthology.orchestra.run.smithi028.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T03:37:11.645 INFO:teuthology.orchestra.run.smithi028.stdout:Access: 2024-04-26 03:36:59.293923829 +0000 2024-04-26T03:37:11.646 INFO:teuthology.orchestra.run.smithi028.stdout:Modify: 2024-04-26 03:34:18.672485595 +0000 2024-04-26T03:37:11.646 INFO:teuthology.orchestra.run.smithi028.stdout:Change: 2024-04-26 03:34:18.672485595 +0000 2024-04-26T03:37:11.646 INFO:teuthology.orchestra.run.smithi028.stdout: Birth: 2024-04-26 03:34:18.672485595 +0000 2024-04-26T03:37:11.646 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T03:37:11.708 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records in 2024-04-26T03:37:11.709 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records out 2024-04-26T03:37:11.709 INFO:teuthology.orchestra.run.smithi028.stderr:512 bytes copied, 8.4147e-05 s, 6.1 MB/s 2024-04-26T03:37:11.710 DEBUG:teuthology.orchestra.run.smithi028:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T03:37:11.766 DEBUG:teuthology.orchestra.run.smithi028:> stat /dev/vg_nvme/lv_3 2024-04-26T03:37:11.821 INFO:teuthology.orchestra.run.smithi028.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T03:37:11.822 INFO:teuthology.orchestra.run.smithi028.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T03:37:11.822 INFO:teuthology.orchestra.run.smithi028.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-26T03:37:11.822 INFO:teuthology.orchestra.run.smithi028.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T03:37:11.822 INFO:teuthology.orchestra.run.smithi028.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T03:37:11.822 INFO:teuthology.orchestra.run.smithi028.stdout:Access: 2024-04-26 03:36:59.294923800 +0000 2024-04-26T03:37:11.822 INFO:teuthology.orchestra.run.smithi028.stdout:Modify: 2024-04-26 03:34:19.012475702 +0000 2024-04-26T03:37:11.822 INFO:teuthology.orchestra.run.smithi028.stdout:Change: 2024-04-26 03:34:19.012475702 +0000 2024-04-26T03:37:11.822 INFO:teuthology.orchestra.run.smithi028.stdout: Birth: 2024-04-26 03:34:19.012475702 +0000 2024-04-26T03:37:11.822 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T03:37:11.886 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records in 2024-04-26T03:37:11.886 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records out 2024-04-26T03:37:11.886 INFO:teuthology.orchestra.run.smithi028.stderr:512 bytes copied, 9.1362e-05 s, 5.6 MB/s 2024-04-26T03:37:11.887 DEBUG:teuthology.orchestra.run.smithi028:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T03:37:11.944 DEBUG:teuthology.orchestra.run.smithi028:> stat /dev/vg_nvme/lv_4 2024-04-26T03:37:12.000 INFO:teuthology.orchestra.run.smithi028.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T03:37:12.000 INFO:teuthology.orchestra.run.smithi028.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T03:37:12.000 INFO:teuthology.orchestra.run.smithi028.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-26T03:37:12.000 INFO:teuthology.orchestra.run.smithi028.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T03:37:12.000 INFO:teuthology.orchestra.run.smithi028.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T03:37:12.001 INFO:teuthology.orchestra.run.smithi028.stdout:Access: 2024-04-26 03:36:59.294923800 +0000 2024-04-26T03:37:12.001 INFO:teuthology.orchestra.run.smithi028.stdout:Modify: 2024-04-26 03:34:19.329466479 +0000 2024-04-26T03:37:12.001 INFO:teuthology.orchestra.run.smithi028.stdout:Change: 2024-04-26 03:34:19.329466479 +0000 2024-04-26T03:37:12.001 INFO:teuthology.orchestra.run.smithi028.stdout: Birth: 2024-04-26 03:34:19.329466479 +0000 2024-04-26T03:37:12.001 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T03:37:12.065 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records in 2024-04-26T03:37:12.065 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records out 2024-04-26T03:37:12.065 INFO:teuthology.orchestra.run.smithi028.stderr:512 bytes copied, 8.9156e-05 s, 5.7 MB/s 2024-04-26T03:37:12.066 DEBUG:teuthology.orchestra.run.smithi028:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T03:37:12.122 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2024-04-26T03:37:12.122 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-26T03:37:12.122 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T03:37:12.142 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T03:37:12.142 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_1 2024-04-26T03:37:12.201 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T03:37:12.201 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T03:37:12.201 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-26T03:37:12.201 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T03:37:12.202 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T03:37:12.202 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-04-26 03:36:54.492493535 +0000 2024-04-26T03:37:12.202 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-04-26 03:34:20.494537530 +0000 2024-04-26T03:37:12.202 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-04-26 03:34:20.494537530 +0000 2024-04-26T03:37:12.202 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-04-26 03:34:20.494537530 +0000 2024-04-26T03:37:12.202 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T03:37:12.269 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-04-26T03:37:12.269 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-04-26T03:37:12.269 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000120872 s, 4.2 MB/s 2024-04-26T03:37:12.271 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T03:37:12.330 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_2 2024-04-26T03:37:12.388 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T03:37:12.388 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T03:37:12.388 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-26T03:37:12.388 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T03:37:12.388 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T03:37:12.388 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-04-26 03:36:54.493493527 +0000 2024-04-26T03:37:12.388 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-04-26 03:34:20.834535262 +0000 2024-04-26T03:37:12.388 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-04-26 03:34:20.834535262 +0000 2024-04-26T03:37:12.389 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-04-26 03:34:20.834535262 +0000 2024-04-26T03:37:12.389 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T03:37:12.456 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-04-26T03:37:12.456 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-04-26T03:37:12.456 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.0001197 s, 4.3 MB/s 2024-04-26T03:37:12.457 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T03:37:12.518 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_3 2024-04-26T03:37:12.576 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T03:37:12.576 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T03:37:12.576 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-26T03:37:12.576 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T03:37:12.576 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T03:37:12.577 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-04-26 03:36:54.493493527 +0000 2024-04-26T03:37:12.577 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-04-26 03:34:21.168533033 +0000 2024-04-26T03:37:12.577 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-04-26 03:34:21.168533033 +0000 2024-04-26T03:37:12.577 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-04-26 03:34:21.168533033 +0000 2024-04-26T03:37:12.577 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T03:37:12.644 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-04-26T03:37:12.644 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-04-26T03:37:12.644 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000118162 s, 4.3 MB/s 2024-04-26T03:37:12.645 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T03:37:12.705 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_4 2024-04-26T03:37:12.764 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T03:37:12.865 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T03:37:12.865 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-04-26T03:37:12.865 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T03:37:12.865 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T03:37:12.865 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-04-26 03:36:54.494493519 +0000 2024-04-26T03:37:12.866 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-04-26 03:34:21.520530685 +0000 2024-04-26T03:37:12.866 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-04-26 03:34:21.520530685 +0000 2024-04-26T03:37:12.866 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-04-26 03:34:21.520530685 +0000 2024-04-26T03:37:12.866 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T03:37:12.894 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-04-26T03:37:12.895 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-04-26T03:37:12.895 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000115358 s, 4.4 MB/s 2024-04-26T03:37:12.896 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T03:37:12.954 INFO:tasks.ceph:osd dev map: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2024-04-26T03:37:12.954 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-26T03:37:12.955 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T03:37:12.974 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T03:37:12.974 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_1 2024-04-26T03:37:13.032 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T03:37:13.033 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T03:37:13.033 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-26T03:37:13.033 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T03:37:13.033 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T03:37:13.033 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-04-26 03:37:05.546344900 +0000 2024-04-26T03:37:13.033 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-04-26 03:34:20.811094815 +0000 2024-04-26T03:37:13.033 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-04-26 03:34:20.811094815 +0000 2024-04-26T03:37:13.033 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-04-26 03:34:20.811094815 +0000 2024-04-26T03:37:13.034 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T03:37:13.101 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-04-26T03:37:13.101 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-04-26T03:37:13.101 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000259868 s, 2.0 MB/s 2024-04-26T03:37:13.102 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T03:37:13.162 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_2 2024-04-26T03:37:13.219 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T03:37:13.220 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T03:37:13.220 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-26T03:37:13.220 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T03:37:13.220 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T03:37:13.220 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-04-26 03:37:05.546344900 +0000 2024-04-26T03:37:13.220 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-04-26 03:34:21.159091949 +0000 2024-04-26T03:37:13.220 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-04-26 03:34:21.159091949 +0000 2024-04-26T03:37:13.220 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-04-26 03:34:21.159091949 +0000 2024-04-26T03:37:13.220 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T03:37:13.285 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-04-26T03:37:13.285 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-04-26T03:37:13.286 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000223401 s, 2.3 MB/s 2024-04-26T03:37:13.287 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T03:37:13.345 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_3 2024-04-26T03:37:13.401 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T03:37:13.402 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T03:37:13.402 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-26T03:37:13.402 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T03:37:13.402 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T03:37:13.402 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-04-26 03:37:05.547344892 +0000 2024-04-26T03:37:13.402 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-04-26 03:34:21.508089073 +0000 2024-04-26T03:37:13.402 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-04-26 03:34:21.508089073 +0000 2024-04-26T03:37:13.402 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-04-26 03:34:21.508089073 +0000 2024-04-26T03:37:13.402 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T03:37:13.467 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-04-26T03:37:13.467 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-04-26T03:37:13.467 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000218596 s, 2.3 MB/s 2024-04-26T03:37:13.469 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T03:37:13.526 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_4 2024-04-26T03:37:13.583 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T03:37:13.583 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T03:37:13.583 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-26T03:37:13.583 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T03:37:13.583 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T03:37:13.583 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-04-26 03:37:05.547344892 +0000 2024-04-26T03:37:13.584 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-04-26 03:34:21.859086181 +0000 2024-04-26T03:37:13.584 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-04-26 03:34:21.859086181 +0000 2024-04-26T03:37:13.584 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-04-26 03:34:21.859086181 +0000 2024-04-26T03:37:13.584 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T03:37:13.647 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-04-26T03:37:13.647 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-04-26T03:37:13.647 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000195127 s, 2.6 MB/s 2024-04-26T03:37:13.648 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T03:37:13.704 INFO:tasks.ceph:osd dev map: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2024-04-26T03:37:13.705 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-26T03:37:13.705 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T03:37:13.723 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T03:37:13.723 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_1 2024-04-26T03:37:13.780 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T03:37:13.780 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T03:37:13.780 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-26T03:37:13.780 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T03:37:13.780 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T03:37:13.781 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-04-26 03:36:54.159256605 +0000 2024-04-26T03:37:13.781 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-04-26 03:34:13.721395835 +0000 2024-04-26T03:37:13.781 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-04-26 03:34:13.721395835 +0000 2024-04-26T03:37:13.781 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-04-26 03:34:13.721395835 +0000 2024-04-26T03:37:13.781 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T03:37:13.846 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-04-26T03:37:13.846 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-04-26T03:37:13.846 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000223859 s, 2.3 MB/s 2024-04-26T03:37:13.847 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T03:37:13.905 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_2 2024-04-26T03:37:13.962 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T03:37:13.962 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T03:37:13.962 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-26T03:37:13.963 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T03:37:13.963 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T03:37:13.963 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-04-26 03:36:54.159256605 +0000 2024-04-26T03:37:13.963 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-04-26 03:34:14.059385997 +0000 2024-04-26T03:37:13.963 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-04-26 03:34:14.059385997 +0000 2024-04-26T03:37:13.963 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-04-26 03:34:14.059385997 +0000 2024-04-26T03:37:13.963 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T03:37:14.027 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-04-26T03:37:14.027 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-04-26T03:37:14.027 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000212833 s, 2.4 MB/s 2024-04-26T03:37:14.028 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T03:37:14.086 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_3 2024-04-26T03:37:14.143 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T03:37:14.143 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T03:37:14.144 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-26T03:37:14.144 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T03:37:14.144 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T03:37:14.144 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-04-26 03:36:54.159256605 +0000 2024-04-26T03:37:14.144 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-04-26 03:34:14.393376275 +0000 2024-04-26T03:37:14.144 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-04-26 03:34:14.393376275 +0000 2024-04-26T03:37:14.144 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-04-26 03:34:14.393376275 +0000 2024-04-26T03:37:14.145 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T03:37:14.208 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-04-26T03:37:14.208 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-04-26T03:37:14.208 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.00019359 s, 2.6 MB/s 2024-04-26T03:37:14.209 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T03:37:14.266 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_4 2024-04-26T03:37:14.322 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T03:37:14.323 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T03:37:14.323 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-26T03:37:14.323 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T03:37:14.323 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T03:37:14.323 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-04-26 03:36:54.160256576 +0000 2024-04-26T03:37:14.323 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-04-26 03:34:14.728366523 +0000 2024-04-26T03:37:14.323 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-04-26 03:34:14.728366523 +0000 2024-04-26T03:37:14.323 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-04-26 03:34:14.728366523 +0000 2024-04-26T03:37:14.323 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T03:37:14.387 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-04-26T03:37:14.387 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-04-26T03:37:14.387 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.0001917 s, 2.7 MB/s 2024-04-26T03:37:14.388 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T03:37:14.445 INFO:tasks.ceph:osd dev map: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2024-04-26T03:37:14.445 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi028.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi042.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi066.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi092.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'}} 2024-04-26T03:37:14.445 INFO:tasks.ceph:Generating config... 2024-04-26T03:37:14.447 INFO:tasks.ceph:[osd] debug monc = 20 2024-04-26T03:37:14.447 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-26T03:37:14.447 INFO:tasks.ceph:[osd] bluestore compression algorithm = zlib 2024-04-26T03:37:14.447 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-04-26T03:37:14.447 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-26T03:37:14.447 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2024-04-26T03:37:14.447 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-26T03:37:14.447 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-26T03:37:14.447 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-26T03:37:14.447 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-26T03:37:14.447 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-26T03:37:14.448 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-26T03:37:14.448 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-26T03:37:14.448 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-26T03:37:14.448 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-04-26T03:37:14.448 INFO:tasks.ceph:[osd] osd debug reject backfill probability = 0.3 2024-04-26T03:37:14.448 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2024-04-26T03:37:14.448 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2024-04-26T03:37:14.448 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-26T03:37:14.448 INFO:tasks.ceph:[osd] osd max backfills = 3 2024-04-26T03:37:14.448 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-04-26T03:37:14.448 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2024-04-26T03:37:14.449 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2024-04-26T03:37:14.449 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-26T03:37:14.449 INFO:tasks.ceph:[osd] osd op queue = debug_random 2024-04-26T03:37:14.449 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2024-04-26T03:37:14.449 INFO:tasks.ceph:[osd] osd scrub max interval = 120 2024-04-26T03:37:14.449 INFO:tasks.ceph:[osd] osd scrub min interval = 60 2024-04-26T03:37:14.449 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-04-26T03:37:14.449 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-04-26T03:37:14.449 INFO:tasks.ceph:[global] mon election default strategy = 1 2024-04-26T03:37:14.449 INFO:tasks.ceph:[global] ms inject socket failures = 2500 2024-04-26T03:37:14.449 INFO:tasks.ceph:[global] ms tcp read timeout = 5 2024-04-26T03:37:14.449 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-26T03:37:14.449 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-26T03:37:14.449 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-26T03:37:14.449 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-26T03:37:14.450 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-26T03:37:14.450 INFO:tasks.ceph:[mon] mon scrub interval = 300 2024-04-26T03:37:14.450 INFO:tasks.ceph:Setting up mon.a... 2024-04-26T03:37:14.450 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-26T03:37:14.504 INFO:teuthology.orchestra.run.smithi028.stdout:creating /etc/ceph/ceph.keyring 2024-04-26T03:37:14.507 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-26T03:37:14.553 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T03:37:14.578 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.28'), ('mon.b', '172.21.15.42'), ('mon.c', '172.21.15.66')] 2024-04-26T03:37:14.579 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.28,172.21.15.42,172.21.15.66', 'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 2500, 'ms tcp read timeout': 5}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': 120, 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub min interval': 60, 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon scrub interval': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}} 2024-04-26T03:37:14.580 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T03:37:14.580 DEBUG:teuthology.orchestra.run.smithi028:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-26T03:37:14.635 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.28 --add b 172.21.15.42 --add c 172.21.15.66 --print /home/ubuntu/cephtest/ceph.monmap 2024-04-26T03:37:14.721 INFO:teuthology.orchestra.run.smithi028.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-26T03:37:14.721 INFO:teuthology.orchestra.run.smithi028.stdout:monmaptool: generated fsid c5a13c99-5c9a-4b8b-a893-4f6f4211677a 2024-04-26T03:37:14.721 INFO:teuthology.orchestra.run.smithi028.stdout:setting min_mon_release = quincy 2024-04-26T03:37:14.721 INFO:teuthology.orchestra.run.smithi028.stdout:epoch 0 2024-04-26T03:37:14.721 INFO:teuthology.orchestra.run.smithi028.stdout:fsid c5a13c99-5c9a-4b8b-a893-4f6f4211677a 2024-04-26T03:37:14.721 INFO:teuthology.orchestra.run.smithi028.stdout:last_changed 2024-04-26T03:37:14.720345+0000 2024-04-26T03:37:14.721 INFO:teuthology.orchestra.run.smithi028.stdout:created 2024-04-26T03:37:14.720345+0000 2024-04-26T03:37:14.721 INFO:teuthology.orchestra.run.smithi028.stdout:min_mon_release 17 (quincy) 2024-04-26T03:37:14.721 INFO:teuthology.orchestra.run.smithi028.stdout:election_strategy: 1 2024-04-26T03:37:14.721 INFO:teuthology.orchestra.run.smithi028.stdout:0: [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] mon.a 2024-04-26T03:37:14.721 INFO:teuthology.orchestra.run.smithi028.stdout:1: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.b 2024-04-26T03:37:14.722 INFO:teuthology.orchestra.run.smithi028.stdout:2: [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] mon.c 2024-04-26T03:37:14.722 INFO:teuthology.orchestra.run.smithi028.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-26T03:37:14.723 DEBUG:teuthology.orchestra.run.smithi028:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-26T03:37:14.738 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID c5a13c99-5c9a-4b8b-a893-4f6f4211677a... 2024-04-26T03:37:14.739 DEBUG:teuthology.orchestra.run.smithi028:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-26T03:37:14.780 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-26T03:37:14.782 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-26T03:37:14.785 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-26T03:37:14.819 INFO:teuthology.orchestra.run.smithi028.stdout:[global] 2024-04-26T03:37:14.820 INFO:teuthology.orchestra.run.smithi028.stdout: chdir = "" 2024-04-26T03:37:14.820 INFO:teuthology.orchestra.run.smithi028.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-26T03:37:14.820 INFO:teuthology.orchestra.run.smithi028.stdout: auth supported = cephx 2024-04-26T03:37:14.820 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:37:14.820 INFO:teuthology.orchestra.run.smithi028.stdout: filestore xattr use omap = true 2024-04-26T03:37:14.820 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:37:14.820 INFO:teuthology.orchestra.run.smithi028.stdout: mon clock drift allowed = 1.000 2024-04-26T03:37:14.820 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:37:14.820 INFO:teuthology.orchestra.run.smithi028.stdout: osd crush chooseleaf type = 0 2024-04-26T03:37:14.820 INFO:teuthology.orchestra.run.smithi028.stdout: auth debug = true 2024-04-26T03:37:14.820 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:37:14.820 INFO:teuthology.orchestra.run.smithi028.stdout: ms die on old message = true 2024-04-26T03:37:14.820 INFO:teuthology.orchestra.run.smithi028.stdout: ms die on bug = true 2024-04-26T03:37:14.820 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:37:14.821 INFO:teuthology.orchestra.run.smithi028.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-26T03:37:14.821 INFO:teuthology.orchestra.run.smithi028.stdout: mon pg warn max object skew = 0 2024-04-26T03:37:14.821 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:37:14.821 INFO:teuthology.orchestra.run.smithi028.stdout: # disable pg_autoscaler by default for new pools 2024-04-26T03:37:14.821 INFO:teuthology.orchestra.run.smithi028.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-26T03:37:14.821 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:37:14.821 INFO:teuthology.orchestra.run.smithi028.stdout: osd pool default size = 2 2024-04-26T03:37:14.821 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:37:14.821 INFO:teuthology.orchestra.run.smithi028.stdout: mon osd allow primary affinity = true 2024-04-26T03:37:14.821 INFO:teuthology.orchestra.run.smithi028.stdout: mon osd allow pg remap = true 2024-04-26T03:37:14.821 INFO:teuthology.orchestra.run.smithi028.stdout: mon warn on legacy crush tunables = false 2024-04-26T03:37:14.821 INFO:teuthology.orchestra.run.smithi028.stdout: mon warn on crush straw calc version zero = false 2024-04-26T03:37:14.821 INFO:teuthology.orchestra.run.smithi028.stdout: mon warn on no sortbitwise = false 2024-04-26T03:37:14.821 INFO:teuthology.orchestra.run.smithi028.stdout: mon warn on osd down out interval zero = false 2024-04-26T03:37:14.822 INFO:teuthology.orchestra.run.smithi028.stdout: mon warn on too few osds = false 2024-04-26T03:37:14.822 INFO:teuthology.orchestra.run.smithi028.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-26T03:37:14.822 INFO:teuthology.orchestra.run.smithi028.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-26T03:37:14.822 INFO:teuthology.orchestra.run.smithi028.stdout: mon_allow_pool_size_one = true 2024-04-26T03:37:14.822 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:37:14.822 INFO:teuthology.orchestra.run.smithi028.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-26T03:37:14.822 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:37:14.822 INFO:teuthology.orchestra.run.smithi028.stdout: osd default data pool replay window = 5 2024-04-26T03:37:14.822 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:37:14.822 INFO:teuthology.orchestra.run.smithi028.stdout: mon allow pool delete = true 2024-04-26T03:37:14.822 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:37:14.822 INFO:teuthology.orchestra.run.smithi028.stdout: mon cluster log file level = debug 2024-04-26T03:37:14.823 INFO:teuthology.orchestra.run.smithi028.stdout: debug asserts on shutdown = true 2024-04-26T03:37:14.823 INFO:teuthology.orchestra.run.smithi028.stdout: mon health detail to clog = false 2024-04-26T03:37:14.823 INFO:teuthology.orchestra.run.smithi028.stdout: mon host = "172.21.15.28,172.21.15.42,172.21.15.66" 2024-04-26T03:37:14.823 INFO:teuthology.orchestra.run.smithi028.stdout: mon client directed command retry = 5 2024-04-26T03:37:14.823 INFO:teuthology.orchestra.run.smithi028.stdout: mon election default strategy = 1 2024-04-26T03:37:14.823 INFO:teuthology.orchestra.run.smithi028.stdout: ms inject socket failures = 2500 2024-04-26T03:37:14.823 INFO:teuthology.orchestra.run.smithi028.stdout: ms tcp read timeout = 5 2024-04-26T03:37:14.823 INFO:teuthology.orchestra.run.smithi028.stdout: fsid = c5a13c99-5c9a-4b8b-a893-4f6f4211677a 2024-04-26T03:37:14.823 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:37:14.823 INFO:teuthology.orchestra.run.smithi028.stdout:[osd] 2024-04-26T03:37:14.823 INFO:teuthology.orchestra.run.smithi028.stdout: osd journal size = 100 2024-04-26T03:37:14.823 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:37:14.823 INFO:teuthology.orchestra.run.smithi028.stdout: osd scrub load threshold = 5.0 2024-04-26T03:37:14.823 INFO:teuthology.orchestra.run.smithi028.stdout: osd scrub max interval = 120 2024-04-26T03:37:14.824 INFO:teuthology.orchestra.run.smithi028.stdout: osd mclock profile = high_recovery_ops 2024-04-26T03:37:14.824 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:37:14.824 INFO:teuthology.orchestra.run.smithi028.stdout: osd recover clone overlap = true 2024-04-26T03:37:14.824 INFO:teuthology.orchestra.run.smithi028.stdout: osd recovery max chunk = 1048576 2024-04-26T03:37:14.824 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:37:14.824 INFO:teuthology.orchestra.run.smithi028.stdout: osd debug shutdown = true 2024-04-26T03:37:14.824 INFO:teuthology.orchestra.run.smithi028.stdout: osd debug op order = true 2024-04-26T03:37:14.824 INFO:teuthology.orchestra.run.smithi028.stdout: osd debug verify stray on activate = true 2024-04-26T03:37:14.824 INFO:teuthology.orchestra.run.smithi028.stdout: osd debug trim objects = true 2024-04-26T03:37:14.824 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:37:14.824 INFO:teuthology.orchestra.run.smithi028.stdout: osd open classes on start = true 2024-04-26T03:37:14.824 INFO:teuthology.orchestra.run.smithi028.stdout: osd debug pg log writeout = true 2024-04-26T03:37:14.824 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:37:14.824 INFO:teuthology.orchestra.run.smithi028.stdout: osd deep scrub update digest min age = 30 2024-04-26T03:37:14.825 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:37:14.825 INFO:teuthology.orchestra.run.smithi028.stdout: osd map max advance = 10 2024-04-26T03:37:14.825 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:37:14.825 INFO:teuthology.orchestra.run.smithi028.stdout: journal zero on create = true 2024-04-26T03:37:14.825 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:37:14.825 INFO:teuthology.orchestra.run.smithi028.stdout: filestore ondisk finisher threads = 3 2024-04-26T03:37:14.825 INFO:teuthology.orchestra.run.smithi028.stdout: filestore apply finisher threads = 3 2024-04-26T03:37:14.825 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:37:14.825 INFO:teuthology.orchestra.run.smithi028.stdout: bdev debug aio = true 2024-04-26T03:37:14.825 INFO:teuthology.orchestra.run.smithi028.stdout: osd debug misdirected ops = true 2024-04-26T03:37:14.825 INFO:teuthology.orchestra.run.smithi028.stdout: debug monc = 20 2024-04-26T03:37:14.826 INFO:teuthology.orchestra.run.smithi028.stdout: bluestore block size = 96636764160 2024-04-26T03:37:14.826 INFO:teuthology.orchestra.run.smithi028.stdout: bluestore compression algorithm = zlib 2024-04-26T03:37:14.826 INFO:teuthology.orchestra.run.smithi028.stdout: bluestore compression mode = aggressive 2024-04-26T03:37:14.826 INFO:teuthology.orchestra.run.smithi028.stdout: bluestore fsck on mount = True 2024-04-26T03:37:14.826 INFO:teuthology.orchestra.run.smithi028.stdout: bluestore zero block detection = True 2024-04-26T03:37:14.826 INFO:teuthology.orchestra.run.smithi028.stdout: debug bluefs = 20 2024-04-26T03:37:14.826 INFO:teuthology.orchestra.run.smithi028.stdout: debug bluestore = 20 2024-04-26T03:37:14.826 INFO:teuthology.orchestra.run.smithi028.stdout: debug ms = 1 2024-04-26T03:37:14.826 INFO:teuthology.orchestra.run.smithi028.stdout: debug osd = 20 2024-04-26T03:37:14.826 INFO:teuthology.orchestra.run.smithi028.stdout: debug rocksdb = 10 2024-04-26T03:37:14.826 INFO:teuthology.orchestra.run.smithi028.stdout: mon osd backfillfull_ratio = 0.85 2024-04-26T03:37:14.830 INFO:teuthology.orchestra.run.smithi028.stdout: mon osd full ratio = 0.9 2024-04-26T03:37:14.830 INFO:teuthology.orchestra.run.smithi028.stdout: mon osd nearfull ratio = 0.8 2024-04-26T03:37:14.830 INFO:teuthology.orchestra.run.smithi028.stdout: osd blocked scrub grace period = 3600 2024-04-26T03:37:14.830 INFO:teuthology.orchestra.run.smithi028.stdout: osd debug reject backfill probability = 0.3 2024-04-26T03:37:14.830 INFO:teuthology.orchestra.run.smithi028.stdout: osd debug verify cached snaps = True 2024-04-26T03:37:14.830 INFO:teuthology.orchestra.run.smithi028.stdout: osd debug verify missing on start = True 2024-04-26T03:37:14.830 INFO:teuthology.orchestra.run.smithi028.stdout: osd failsafe full ratio = 0.95 2024-04-26T03:37:14.830 INFO:teuthology.orchestra.run.smithi028.stdout: osd max backfills = 3 2024-04-26T03:37:14.830 INFO:teuthology.orchestra.run.smithi028.stdout: osd max markdown count = 1000 2024-04-26T03:37:14.830 INFO:teuthology.orchestra.run.smithi028.stdout: osd mclock override recovery settings = True 2024-04-26T03:37:14.830 INFO:teuthology.orchestra.run.smithi028.stdout: osd objectstore = bluestore 2024-04-26T03:37:14.830 INFO:teuthology.orchestra.run.smithi028.stdout: osd op queue = debug_random 2024-04-26T03:37:14.830 INFO:teuthology.orchestra.run.smithi028.stdout: osd op queue cut off = debug_random 2024-04-26T03:37:14.831 INFO:teuthology.orchestra.run.smithi028.stdout: osd scrub min interval = 60 2024-04-26T03:37:14.831 INFO:teuthology.orchestra.run.smithi028.stdout: osd shutdown pgref assert = True 2024-04-26T03:37:14.831 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:37:14.831 INFO:teuthology.orchestra.run.smithi028.stdout:[mgr] 2024-04-26T03:37:14.831 INFO:teuthology.orchestra.run.smithi028.stdout: debug ms = 1 2024-04-26T03:37:14.831 INFO:teuthology.orchestra.run.smithi028.stdout: debug mgr = 20 2024-04-26T03:37:14.831 INFO:teuthology.orchestra.run.smithi028.stdout: debug mon = 20 2024-04-26T03:37:14.831 INFO:teuthology.orchestra.run.smithi028.stdout: debug auth = 20 2024-04-26T03:37:14.831 INFO:teuthology.orchestra.run.smithi028.stdout: mon reweight min pgs per osd = 4 2024-04-26T03:37:14.831 INFO:teuthology.orchestra.run.smithi028.stdout: mon reweight min bytes per osd = 10 2024-04-26T03:37:14.831 INFO:teuthology.orchestra.run.smithi028.stdout: mgr/telemetry/nag = false 2024-04-26T03:37:14.831 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:37:14.831 INFO:teuthology.orchestra.run.smithi028.stdout:[mon] 2024-04-26T03:37:14.831 INFO:teuthology.orchestra.run.smithi028.stdout: debug ms = 1 2024-04-26T03:37:14.832 INFO:teuthology.orchestra.run.smithi028.stdout: debug mon = 20 2024-04-26T03:37:14.832 INFO:teuthology.orchestra.run.smithi028.stdout: debug paxos = 20 2024-04-26T03:37:14.832 INFO:teuthology.orchestra.run.smithi028.stdout: debug auth = 20 2024-04-26T03:37:14.832 INFO:teuthology.orchestra.run.smithi028.stdout: mon data avail warn = 5 2024-04-26T03:37:14.832 INFO:teuthology.orchestra.run.smithi028.stdout: mon mgr mkfs grace = 240 2024-04-26T03:37:14.832 INFO:teuthology.orchestra.run.smithi028.stdout: mon reweight min pgs per osd = 4 2024-04-26T03:37:14.832 INFO:teuthology.orchestra.run.smithi028.stdout: mon osd reporter subtree level = osd 2024-04-26T03:37:14.832 INFO:teuthology.orchestra.run.smithi028.stdout: mon osd prime pg temp = true 2024-04-26T03:37:14.832 INFO:teuthology.orchestra.run.smithi028.stdout: mon reweight min bytes per osd = 10 2024-04-26T03:37:14.832 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:37:14.832 INFO:teuthology.orchestra.run.smithi028.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-26T03:37:14.832 INFO:teuthology.orchestra.run.smithi028.stdout: auth mon ticket ttl = 660 # 11m 2024-04-26T03:37:14.832 INFO:teuthology.orchestra.run.smithi028.stdout: auth service ticket ttl = 240 # 4m 2024-04-26T03:37:14.833 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:37:14.833 INFO:teuthology.orchestra.run.smithi028.stdout: # don't complain about insecure global_id in the test suite 2024-04-26T03:37:14.833 INFO:teuthology.orchestra.run.smithi028.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-26T03:37:14.833 INFO:teuthology.orchestra.run.smithi028.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-26T03:37:14.833 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:37:14.833 INFO:teuthology.orchestra.run.smithi028.stdout: # 1m isn't quite enough 2024-04-26T03:37:14.833 INFO:teuthology.orchestra.run.smithi028.stdout: mon_down_mkfs_grace = 2m 2024-04-26T03:37:14.833 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:37:14.833 INFO:teuthology.orchestra.run.smithi028.stdout: mon_warn_on_filestore_osds = false 2024-04-26T03:37:14.833 INFO:teuthology.orchestra.run.smithi028.stdout: mon scrub interval = 300 2024-04-26T03:37:14.833 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:37:14.833 INFO:teuthology.orchestra.run.smithi028.stdout:[client] 2024-04-26T03:37:14.833 INFO:teuthology.orchestra.run.smithi028.stdout: rgw cache enabled = true 2024-04-26T03:37:14.833 INFO:teuthology.orchestra.run.smithi028.stdout: rgw enable ops log = true 2024-04-26T03:37:14.834 INFO:teuthology.orchestra.run.smithi028.stdout: rgw enable usage log = true 2024-04-26T03:37:14.834 INFO:teuthology.orchestra.run.smithi028.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-26T03:37:14.834 INFO:teuthology.orchestra.run.smithi028.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-26T03:37:14.834 INFO:teuthology.orchestra.run.smithi028.stdout:[mon.a] 2024-04-26T03:37:14.834 INFO:teuthology.orchestra.run.smithi028.stdout:[mon.b] 2024-04-26T03:37:14.834 INFO:teuthology.orchestra.run.smithi028.stdout:[mon.c] 2024-04-26T03:37:14.835 INFO:teuthology.orchestra.run.smithi066.stdout:[global] 2024-04-26T03:37:14.836 INFO:teuthology.orchestra.run.smithi066.stdout: chdir = "" 2024-04-26T03:37:14.836 INFO:teuthology.orchestra.run.smithi066.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-26T03:37:14.836 INFO:teuthology.orchestra.run.smithi066.stdout: auth supported = cephx 2024-04-26T03:37:14.836 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:37:14.836 INFO:teuthology.orchestra.run.smithi066.stdout: filestore xattr use omap = true 2024-04-26T03:37:14.838 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:37:14.838 INFO:teuthology.orchestra.run.smithi066.stdout: mon clock drift allowed = 1.000 2024-04-26T03:37:14.838 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:37:14.838 INFO:teuthology.orchestra.run.smithi066.stdout: osd crush chooseleaf type = 0 2024-04-26T03:37:14.838 INFO:teuthology.orchestra.run.smithi066.stdout: auth debug = true 2024-04-26T03:37:14.838 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:37:14.838 INFO:teuthology.orchestra.run.smithi066.stdout: ms die on old message = true 2024-04-26T03:37:14.838 INFO:teuthology.orchestra.run.smithi066.stdout: ms die on bug = true 2024-04-26T03:37:14.838 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:37:14.838 INFO:teuthology.orchestra.run.smithi066.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-26T03:37:14.838 INFO:teuthology.orchestra.run.smithi066.stdout: mon pg warn max object skew = 0 2024-04-26T03:37:14.838 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:37:14.838 INFO:teuthology.orchestra.run.smithi066.stdout: # disable pg_autoscaler by default for new pools 2024-04-26T03:37:14.839 INFO:teuthology.orchestra.run.smithi066.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-26T03:37:14.839 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:37:14.839 INFO:teuthology.orchestra.run.smithi066.stdout: osd pool default size = 2 2024-04-26T03:37:14.839 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:37:14.839 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd allow primary affinity = true 2024-04-26T03:37:14.839 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd allow pg remap = true 2024-04-26T03:37:14.839 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on legacy crush tunables = false 2024-04-26T03:37:14.839 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on crush straw calc version zero = false 2024-04-26T03:37:14.839 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on no sortbitwise = false 2024-04-26T03:37:14.839 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on osd down out interval zero = false 2024-04-26T03:37:14.839 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on too few osds = false 2024-04-26T03:37:14.839 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-26T03:37:14.839 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-26T03:37:14.839 INFO:teuthology.orchestra.run.smithi066.stdout: mon_allow_pool_size_one = true 2024-04-26T03:37:14.840 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:37:14.840 INFO:teuthology.orchestra.run.smithi066.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-26T03:37:14.840 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:37:14.840 INFO:teuthology.orchestra.run.smithi066.stdout: osd default data pool replay window = 5 2024-04-26T03:37:14.840 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:37:14.840 INFO:teuthology.orchestra.run.smithi066.stdout: mon allow pool delete = true 2024-04-26T03:37:14.840 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:37:14.840 INFO:teuthology.orchestra.run.smithi066.stdout: mon cluster log file level = debug 2024-04-26T03:37:14.840 INFO:teuthology.orchestra.run.smithi066.stdout: debug asserts on shutdown = true 2024-04-26T03:37:14.840 INFO:teuthology.orchestra.run.smithi066.stdout: mon health detail to clog = false 2024-04-26T03:37:14.840 INFO:teuthology.orchestra.run.smithi066.stdout: mon host = "172.21.15.28,172.21.15.42,172.21.15.66" 2024-04-26T03:37:14.840 INFO:teuthology.orchestra.run.smithi066.stdout: mon client directed command retry = 5 2024-04-26T03:37:14.840 INFO:teuthology.orchestra.run.smithi066.stdout: mon election default strategy = 1 2024-04-26T03:37:14.840 INFO:teuthology.orchestra.run.smithi066.stdout: ms inject socket failures = 2500 2024-04-26T03:37:14.840 INFO:teuthology.orchestra.run.smithi066.stdout: ms tcp read timeout = 5 2024-04-26T03:37:14.841 INFO:teuthology.orchestra.run.smithi066.stdout: fsid = c5a13c99-5c9a-4b8b-a893-4f6f4211677a 2024-04-26T03:37:14.841 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:37:14.841 INFO:teuthology.orchestra.run.smithi066.stdout:[osd] 2024-04-26T03:37:14.841 INFO:teuthology.orchestra.run.smithi066.stdout: osd journal size = 100 2024-04-26T03:37:14.841 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:37:14.841 INFO:teuthology.orchestra.run.smithi066.stdout: osd scrub load threshold = 5.0 2024-04-26T03:37:14.841 INFO:teuthology.orchestra.run.smithi066.stdout: osd scrub max interval = 120 2024-04-26T03:37:14.841 INFO:teuthology.orchestra.run.smithi066.stdout: osd mclock profile = high_recovery_ops 2024-04-26T03:37:14.841 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:37:14.841 INFO:teuthology.orchestra.run.smithi066.stdout: osd recover clone overlap = true 2024-04-26T03:37:14.841 INFO:teuthology.orchestra.run.smithi066.stdout: osd recovery max chunk = 1048576 2024-04-26T03:37:14.841 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:37:14.841 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug shutdown = true 2024-04-26T03:37:14.841 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug op order = true 2024-04-26T03:37:14.842 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug verify stray on activate = true 2024-04-26T03:37:14.842 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug trim objects = true 2024-04-26T03:37:14.842 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:37:14.842 INFO:teuthology.orchestra.run.smithi066.stdout: osd open classes on start = true 2024-04-26T03:37:14.842 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug pg log writeout = true 2024-04-26T03:37:14.842 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:37:14.842 INFO:teuthology.orchestra.run.smithi066.stdout: osd deep scrub update digest min age = 30 2024-04-26T03:37:14.842 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:37:14.842 INFO:teuthology.orchestra.run.smithi066.stdout: osd map max advance = 10 2024-04-26T03:37:14.842 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:37:14.842 INFO:teuthology.orchestra.run.smithi066.stdout: journal zero on create = true 2024-04-26T03:37:14.842 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:37:14.842 INFO:teuthology.orchestra.run.smithi066.stdout: filestore ondisk finisher threads = 3 2024-04-26T03:37:14.843 INFO:teuthology.orchestra.run.smithi066.stdout: filestore apply finisher threads = 3 2024-04-26T03:37:14.843 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:37:14.843 INFO:teuthology.orchestra.run.smithi066.stdout: bdev debug aio = true 2024-04-26T03:37:14.843 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug misdirected ops = true 2024-04-26T03:37:14.843 INFO:teuthology.orchestra.run.smithi066.stdout: debug monc = 20 2024-04-26T03:37:14.843 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore block size = 96636764160 2024-04-26T03:37:14.843 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore compression algorithm = zlib 2024-04-26T03:37:14.843 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore compression mode = aggressive 2024-04-26T03:37:14.843 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore fsck on mount = True 2024-04-26T03:37:14.843 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore zero block detection = True 2024-04-26T03:37:14.843 INFO:teuthology.orchestra.run.smithi066.stdout: debug bluefs = 20 2024-04-26T03:37:14.843 INFO:teuthology.orchestra.run.smithi066.stdout: debug bluestore = 20 2024-04-26T03:37:14.843 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-04-26T03:37:14.843 INFO:teuthology.orchestra.run.smithi066.stdout: debug osd = 20 2024-04-26T03:37:14.843 INFO:teuthology.orchestra.run.smithi066.stdout: debug rocksdb = 10 2024-04-26T03:37:14.844 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd backfillfull_ratio = 0.85 2024-04-26T03:37:14.844 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd full ratio = 0.9 2024-04-26T03:37:14.844 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd nearfull ratio = 0.8 2024-04-26T03:37:14.844 INFO:teuthology.orchestra.run.smithi066.stdout: osd blocked scrub grace period = 3600 2024-04-26T03:37:14.844 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug reject backfill probability = 0.3 2024-04-26T03:37:14.844 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug verify cached snaps = True 2024-04-26T03:37:14.844 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug verify missing on start = True 2024-04-26T03:37:14.844 INFO:teuthology.orchestra.run.smithi066.stdout: osd failsafe full ratio = 0.95 2024-04-26T03:37:14.844 INFO:teuthology.orchestra.run.smithi066.stdout: osd max backfills = 3 2024-04-26T03:37:14.844 INFO:teuthology.orchestra.run.smithi066.stdout: osd max markdown count = 1000 2024-04-26T03:37:14.844 INFO:teuthology.orchestra.run.smithi066.stdout: osd mclock override recovery settings = True 2024-04-26T03:37:14.844 INFO:teuthology.orchestra.run.smithi066.stdout: osd objectstore = bluestore 2024-04-26T03:37:14.844 INFO:teuthology.orchestra.run.smithi066.stdout: osd op queue = debug_random 2024-04-26T03:37:14.844 INFO:teuthology.orchestra.run.smithi066.stdout: osd op queue cut off = debug_random 2024-04-26T03:37:14.845 INFO:teuthology.orchestra.run.smithi066.stdout: osd scrub min interval = 60 2024-04-26T03:37:14.845 INFO:teuthology.orchestra.run.smithi066.stdout: osd shutdown pgref assert = True 2024-04-26T03:37:14.845 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:37:14.845 INFO:teuthology.orchestra.run.smithi066.stdout:[mgr] 2024-04-26T03:37:14.845 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-04-26T03:37:14.845 INFO:teuthology.orchestra.run.smithi066.stdout: debug mgr = 20 2024-04-26T03:37:14.845 INFO:teuthology.orchestra.run.smithi066.stdout: debug mon = 20 2024-04-26T03:37:14.845 INFO:teuthology.orchestra.run.smithi066.stdout: debug auth = 20 2024-04-26T03:37:14.845 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min pgs per osd = 4 2024-04-26T03:37:14.845 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min bytes per osd = 10 2024-04-26T03:37:14.845 INFO:teuthology.orchestra.run.smithi066.stdout: mgr/telemetry/nag = false 2024-04-26T03:37:14.845 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:37:14.845 INFO:teuthology.orchestra.run.smithi066.stdout:[mon] 2024-04-26T03:37:14.845 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-04-26T03:37:14.846 INFO:teuthology.orchestra.run.smithi066.stdout: debug mon = 20 2024-04-26T03:37:14.846 INFO:teuthology.orchestra.run.smithi066.stdout: debug paxos = 20 2024-04-26T03:37:14.846 INFO:teuthology.orchestra.run.smithi066.stdout: debug auth = 20 2024-04-26T03:37:14.846 INFO:teuthology.orchestra.run.smithi066.stdout: mon data avail warn = 5 2024-04-26T03:37:14.846 INFO:teuthology.orchestra.run.smithi066.stdout: mon mgr mkfs grace = 240 2024-04-26T03:37:14.846 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min pgs per osd = 4 2024-04-26T03:37:14.846 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd reporter subtree level = osd 2024-04-26T03:37:14.846 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd prime pg temp = true 2024-04-26T03:37:14.846 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min bytes per osd = 10 2024-04-26T03:37:14.846 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:37:14.846 INFO:teuthology.orchestra.run.smithi066.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-26T03:37:14.846 INFO:teuthology.orchestra.run.smithi066.stdout: auth mon ticket ttl = 660 # 11m 2024-04-26T03:37:14.846 INFO:teuthology.orchestra.run.smithi066.stdout: auth service ticket ttl = 240 # 4m 2024-04-26T03:37:14.846 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:37:14.847 INFO:teuthology.orchestra.run.smithi066.stdout: # don't complain about insecure global_id in the test suite 2024-04-26T03:37:14.847 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-26T03:37:14.847 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-26T03:37:14.847 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:37:14.847 INFO:teuthology.orchestra.run.smithi066.stdout: # 1m isn't quite enough 2024-04-26T03:37:14.847 INFO:teuthology.orchestra.run.smithi066.stdout: mon_down_mkfs_grace = 2m 2024-04-26T03:37:14.847 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:37:14.847 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_filestore_osds = false 2024-04-26T03:37:14.847 INFO:teuthology.orchestra.run.smithi066.stdout: mon scrub interval = 300 2024-04-26T03:37:14.847 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-26T03:37:14.847 INFO:teuthology.orchestra.run.smithi066.stdout:[client] 2024-04-26T03:37:14.847 INFO:teuthology.orchestra.run.smithi066.stdout: rgw cache enabled = true 2024-04-26T03:37:14.847 INFO:teuthology.orchestra.run.smithi066.stdout: rgw enable ops log = true 2024-04-26T03:37:14.847 INFO:teuthology.orchestra.run.smithi066.stdout: rgw enable usage log = true 2024-04-26T03:37:14.848 INFO:teuthology.orchestra.run.smithi066.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-26T03:37:14.848 INFO:teuthology.orchestra.run.smithi066.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-26T03:37:14.848 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.a] 2024-04-26T03:37:14.848 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.b] 2024-04-26T03:37:14.848 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.c] 2024-04-26T03:37:14.848 INFO:teuthology.orchestra.run.smithi092.stdout:[global] 2024-04-26T03:37:14.848 INFO:teuthology.orchestra.run.smithi092.stdout: chdir = "" 2024-04-26T03:37:14.848 INFO:teuthology.orchestra.run.smithi092.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-26T03:37:14.848 INFO:teuthology.orchestra.run.smithi092.stdout: auth supported = cephx 2024-04-26T03:37:14.848 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:37:14.848 INFO:teuthology.orchestra.run.smithi092.stdout: filestore xattr use omap = true 2024-04-26T03:37:14.848 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:37:14.849 INFO:teuthology.orchestra.run.smithi092.stdout: mon clock drift allowed = 1.000 2024-04-26T03:37:14.849 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:37:14.849 INFO:teuthology.orchestra.run.smithi092.stdout: osd crush chooseleaf type = 0 2024-04-26T03:37:14.849 INFO:teuthology.orchestra.run.smithi092.stdout: auth debug = true 2024-04-26T03:37:14.849 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:37:14.849 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on old message = true 2024-04-26T03:37:14.849 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on bug = true 2024-04-26T03:37:14.849 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:37:14.849 INFO:teuthology.orchestra.run.smithi092.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-26T03:37:14.849 INFO:teuthology.orchestra.run.smithi092.stdout: mon pg warn max object skew = 0 2024-04-26T03:37:14.849 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:37:14.849 INFO:teuthology.orchestra.run.smithi092.stdout: # disable pg_autoscaler by default for new pools 2024-04-26T03:37:14.849 INFO:teuthology.orchestra.run.smithi092.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-26T03:37:14.849 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:37:14.850 INFO:teuthology.orchestra.run.smithi092.stdout: osd pool default size = 2 2024-04-26T03:37:14.850 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:37:14.850 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd allow primary affinity = true 2024-04-26T03:37:14.850 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd allow pg remap = true 2024-04-26T03:37:14.850 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on legacy crush tunables = false 2024-04-26T03:37:14.850 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on crush straw calc version zero = false 2024-04-26T03:37:14.850 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on no sortbitwise = false 2024-04-26T03:37:14.850 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on osd down out interval zero = false 2024-04-26T03:37:14.850 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on too few osds = false 2024-04-26T03:37:14.850 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-26T03:37:14.850 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-26T03:37:14.850 INFO:teuthology.orchestra.run.smithi092.stdout: mon_allow_pool_size_one = true 2024-04-26T03:37:14.850 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:37:14.851 INFO:teuthology.orchestra.run.smithi092.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-26T03:37:14.851 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:37:14.851 INFO:teuthology.orchestra.run.smithi092.stdout: osd default data pool replay window = 5 2024-04-26T03:37:14.851 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:37:14.851 INFO:teuthology.orchestra.run.smithi092.stdout: mon allow pool delete = true 2024-04-26T03:37:14.851 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:37:14.851 INFO:teuthology.orchestra.run.smithi092.stdout: mon cluster log file level = debug 2024-04-26T03:37:14.851 INFO:teuthology.orchestra.run.smithi092.stdout: debug asserts on shutdown = true 2024-04-26T03:37:14.851 INFO:teuthology.orchestra.run.smithi092.stdout: mon health detail to clog = false 2024-04-26T03:37:14.851 INFO:teuthology.orchestra.run.smithi092.stdout: mon host = "172.21.15.28,172.21.15.42,172.21.15.66" 2024-04-26T03:37:14.851 INFO:teuthology.orchestra.run.smithi092.stdout: mon client directed command retry = 5 2024-04-26T03:37:14.851 INFO:teuthology.orchestra.run.smithi092.stdout: mon election default strategy = 1 2024-04-26T03:37:14.851 INFO:teuthology.orchestra.run.smithi092.stdout: ms inject socket failures = 2500 2024-04-26T03:37:14.851 INFO:teuthology.orchestra.run.smithi092.stdout: ms tcp read timeout = 5 2024-04-26T03:37:14.852 INFO:teuthology.orchestra.run.smithi092.stdout: fsid = c5a13c99-5c9a-4b8b-a893-4f6f4211677a 2024-04-26T03:37:14.852 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:37:14.852 INFO:teuthology.orchestra.run.smithi092.stdout:[osd] 2024-04-26T03:37:14.852 INFO:teuthology.orchestra.run.smithi092.stdout: osd journal size = 100 2024-04-26T03:37:14.852 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:37:14.852 INFO:teuthology.orchestra.run.smithi092.stdout: osd scrub load threshold = 5.0 2024-04-26T03:37:14.852 INFO:teuthology.orchestra.run.smithi092.stdout: osd scrub max interval = 120 2024-04-26T03:37:14.852 INFO:teuthology.orchestra.run.smithi092.stdout: osd mclock profile = high_recovery_ops 2024-04-26T03:37:14.852 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:37:14.852 INFO:teuthology.orchestra.run.smithi092.stdout: osd recover clone overlap = true 2024-04-26T03:37:14.852 INFO:teuthology.orchestra.run.smithi092.stdout: osd recovery max chunk = 1048576 2024-04-26T03:37:14.852 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:37:14.852 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug shutdown = true 2024-04-26T03:37:14.852 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug op order = true 2024-04-26T03:37:14.852 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug verify stray on activate = true 2024-04-26T03:37:14.852 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug trim objects = true 2024-04-26T03:37:14.853 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:37:14.853 INFO:teuthology.orchestra.run.smithi092.stdout: osd open classes on start = true 2024-04-26T03:37:14.853 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug pg log writeout = true 2024-04-26T03:37:14.853 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:37:14.853 INFO:teuthology.orchestra.run.smithi092.stdout: osd deep scrub update digest min age = 30 2024-04-26T03:37:14.853 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:37:14.853 INFO:teuthology.orchestra.run.smithi092.stdout: osd map max advance = 10 2024-04-26T03:37:14.853 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:37:14.853 INFO:teuthology.orchestra.run.smithi092.stdout: journal zero on create = true 2024-04-26T03:37:14.853 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:37:14.853 INFO:teuthology.orchestra.run.smithi092.stdout: filestore ondisk finisher threads = 3 2024-04-26T03:37:14.853 INFO:teuthology.orchestra.run.smithi092.stdout: filestore apply finisher threads = 3 2024-04-26T03:37:14.853 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:37:14.853 INFO:teuthology.orchestra.run.smithi092.stdout: bdev debug aio = true 2024-04-26T03:37:14.854 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug misdirected ops = true 2024-04-26T03:37:14.854 INFO:teuthology.orchestra.run.smithi092.stdout: debug monc = 20 2024-04-26T03:37:14.854 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore block size = 96636764160 2024-04-26T03:37:14.854 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore compression algorithm = zlib 2024-04-26T03:37:14.854 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore compression mode = aggressive 2024-04-26T03:37:14.854 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore fsck on mount = True 2024-04-26T03:37:14.854 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore zero block detection = True 2024-04-26T03:37:14.854 INFO:teuthology.orchestra.run.smithi092.stdout: debug bluefs = 20 2024-04-26T03:37:14.854 INFO:teuthology.orchestra.run.smithi092.stdout: debug bluestore = 20 2024-04-26T03:37:14.854 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2024-04-26T03:37:14.854 INFO:teuthology.orchestra.run.smithi092.stdout: debug osd = 20 2024-04-26T03:37:14.854 INFO:teuthology.orchestra.run.smithi092.stdout: debug rocksdb = 10 2024-04-26T03:37:14.854 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd backfillfull_ratio = 0.85 2024-04-26T03:37:14.854 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd full ratio = 0.9 2024-04-26T03:37:14.855 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd nearfull ratio = 0.8 2024-04-26T03:37:14.855 INFO:teuthology.orchestra.run.smithi092.stdout: osd blocked scrub grace period = 3600 2024-04-26T03:37:14.855 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug reject backfill probability = 0.3 2024-04-26T03:37:14.855 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug verify cached snaps = True 2024-04-26T03:37:14.855 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug verify missing on start = True 2024-04-26T03:37:14.855 INFO:teuthology.orchestra.run.smithi092.stdout: osd failsafe full ratio = 0.95 2024-04-26T03:37:14.855 INFO:teuthology.orchestra.run.smithi092.stdout: osd max backfills = 3 2024-04-26T03:37:14.855 INFO:teuthology.orchestra.run.smithi092.stdout: osd max markdown count = 1000 2024-04-26T03:37:14.855 INFO:teuthology.orchestra.run.smithi092.stdout: osd mclock override recovery settings = True 2024-04-26T03:37:14.855 INFO:teuthology.orchestra.run.smithi092.stdout: osd objectstore = bluestore 2024-04-26T03:37:14.855 INFO:teuthology.orchestra.run.smithi092.stdout: osd op queue = debug_random 2024-04-26T03:37:14.855 INFO:teuthology.orchestra.run.smithi092.stdout: osd op queue cut off = debug_random 2024-04-26T03:37:14.855 INFO:teuthology.orchestra.run.smithi092.stdout: osd scrub min interval = 60 2024-04-26T03:37:14.855 INFO:teuthology.orchestra.run.smithi092.stdout: osd shutdown pgref assert = True 2024-04-26T03:37:14.856 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:37:14.856 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr] 2024-04-26T03:37:14.856 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2024-04-26T03:37:14.856 INFO:teuthology.orchestra.run.smithi092.stdout: debug mgr = 20 2024-04-26T03:37:14.856 INFO:teuthology.orchestra.run.smithi092.stdout: debug mon = 20 2024-04-26T03:37:14.856 INFO:teuthology.orchestra.run.smithi092.stdout: debug auth = 20 2024-04-26T03:37:14.856 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min pgs per osd = 4 2024-04-26T03:37:14.856 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min bytes per osd = 10 2024-04-26T03:37:14.856 INFO:teuthology.orchestra.run.smithi092.stdout: mgr/telemetry/nag = false 2024-04-26T03:37:14.856 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:37:14.856 INFO:teuthology.orchestra.run.smithi092.stdout:[mon] 2024-04-26T03:37:14.856 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2024-04-26T03:37:14.856 INFO:teuthology.orchestra.run.smithi092.stdout: debug mon = 20 2024-04-26T03:37:14.856 INFO:teuthology.orchestra.run.smithi092.stdout: debug paxos = 20 2024-04-26T03:37:14.857 INFO:teuthology.orchestra.run.smithi092.stdout: debug auth = 20 2024-04-26T03:37:14.857 INFO:teuthology.orchestra.run.smithi092.stdout: mon data avail warn = 5 2024-04-26T03:37:14.857 INFO:teuthology.orchestra.run.smithi092.stdout: mon mgr mkfs grace = 240 2024-04-26T03:37:14.857 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min pgs per osd = 4 2024-04-26T03:37:14.857 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd reporter subtree level = osd 2024-04-26T03:37:14.857 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd prime pg temp = true 2024-04-26T03:37:14.857 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min bytes per osd = 10 2024-04-26T03:37:14.857 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:37:14.857 INFO:teuthology.orchestra.run.smithi092.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-26T03:37:14.857 INFO:teuthology.orchestra.run.smithi092.stdout: auth mon ticket ttl = 660 # 11m 2024-04-26T03:37:14.857 INFO:teuthology.orchestra.run.smithi092.stdout: auth service ticket ttl = 240 # 4m 2024-04-26T03:37:14.857 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:37:14.857 INFO:teuthology.orchestra.run.smithi092.stdout: # don't complain about insecure global_id in the test suite 2024-04-26T03:37:14.857 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-26T03:37:14.857 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-26T03:37:14.857 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:37:14.858 INFO:teuthology.orchestra.run.smithi092.stdout: # 1m isn't quite enough 2024-04-26T03:37:14.858 INFO:teuthology.orchestra.run.smithi092.stdout: mon_down_mkfs_grace = 2m 2024-04-26T03:37:14.858 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:37:14.858 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_filestore_osds = false 2024-04-26T03:37:14.858 INFO:teuthology.orchestra.run.smithi092.stdout: mon scrub interval = 300 2024-04-26T03:37:14.858 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-04-26T03:37:14.858 INFO:teuthology.orchestra.run.smithi092.stdout:[client] 2024-04-26T03:37:14.858 INFO:teuthology.orchestra.run.smithi092.stdout: rgw cache enabled = true 2024-04-26T03:37:14.858 INFO:teuthology.orchestra.run.smithi092.stdout: rgw enable ops log = true 2024-04-26T03:37:14.858 INFO:teuthology.orchestra.run.smithi092.stdout: rgw enable usage log = true 2024-04-26T03:37:14.858 INFO:teuthology.orchestra.run.smithi092.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-26T03:37:14.858 INFO:teuthology.orchestra.run.smithi092.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-26T03:37:14.858 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.a] 2024-04-26T03:37:14.858 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.b] 2024-04-26T03:37:14.859 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.c] 2024-04-26T03:37:14.859 INFO:teuthology.orchestra.run.smithi042.stdout:[global] 2024-04-26T03:37:14.859 INFO:teuthology.orchestra.run.smithi042.stdout: chdir = "" 2024-04-26T03:37:14.859 INFO:teuthology.orchestra.run.smithi042.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-26T03:37:14.859 INFO:teuthology.orchestra.run.smithi042.stdout: auth supported = cephx 2024-04-26T03:37:14.859 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:37:14.859 INFO:teuthology.orchestra.run.smithi042.stdout: filestore xattr use omap = true 2024-04-26T03:37:14.859 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:37:14.859 INFO:teuthology.orchestra.run.smithi042.stdout: mon clock drift allowed = 1.000 2024-04-26T03:37:14.859 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:37:14.859 INFO:teuthology.orchestra.run.smithi042.stdout: osd crush chooseleaf type = 0 2024-04-26T03:37:14.859 INFO:teuthology.orchestra.run.smithi042.stdout: auth debug = true 2024-04-26T03:37:14.859 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:37:14.860 INFO:teuthology.orchestra.run.smithi042.stdout: ms die on old message = true 2024-04-26T03:37:14.860 INFO:teuthology.orchestra.run.smithi042.stdout: ms die on bug = true 2024-04-26T03:37:14.860 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:37:14.860 INFO:teuthology.orchestra.run.smithi042.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-26T03:37:14.860 INFO:teuthology.orchestra.run.smithi042.stdout: mon pg warn max object skew = 0 2024-04-26T03:37:14.860 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:37:14.860 INFO:teuthology.orchestra.run.smithi042.stdout: # disable pg_autoscaler by default for new pools 2024-04-26T03:37:14.860 INFO:teuthology.orchestra.run.smithi042.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-26T03:37:14.860 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:37:14.860 INFO:teuthology.orchestra.run.smithi042.stdout: osd pool default size = 2 2024-04-26T03:37:14.860 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:37:14.860 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd allow primary affinity = true 2024-04-26T03:37:14.860 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd allow pg remap = true 2024-04-26T03:37:14.861 INFO:teuthology.orchestra.run.smithi042.stdout: mon warn on legacy crush tunables = false 2024-04-26T03:37:14.861 INFO:teuthology.orchestra.run.smithi042.stdout: mon warn on crush straw calc version zero = false 2024-04-26T03:37:14.861 INFO:teuthology.orchestra.run.smithi042.stdout: mon warn on no sortbitwise = false 2024-04-26T03:37:14.861 INFO:teuthology.orchestra.run.smithi042.stdout: mon warn on osd down out interval zero = false 2024-04-26T03:37:14.861 INFO:teuthology.orchestra.run.smithi042.stdout: mon warn on too few osds = false 2024-04-26T03:37:14.861 INFO:teuthology.orchestra.run.smithi042.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-26T03:37:14.861 INFO:teuthology.orchestra.run.smithi042.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-26T03:37:14.861 INFO:teuthology.orchestra.run.smithi042.stdout: mon_allow_pool_size_one = true 2024-04-26T03:37:14.861 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:37:14.861 INFO:teuthology.orchestra.run.smithi042.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-26T03:37:14.861 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:37:14.861 INFO:teuthology.orchestra.run.smithi042.stdout: osd default data pool replay window = 5 2024-04-26T03:37:14.861 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:37:14.861 INFO:teuthology.orchestra.run.smithi042.stdout: mon allow pool delete = true 2024-04-26T03:37:14.862 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:37:14.862 INFO:teuthology.orchestra.run.smithi042.stdout: mon cluster log file level = debug 2024-04-26T03:37:14.862 INFO:teuthology.orchestra.run.smithi042.stdout: debug asserts on shutdown = true 2024-04-26T03:37:14.862 INFO:teuthology.orchestra.run.smithi042.stdout: mon health detail to clog = false 2024-04-26T03:37:14.862 INFO:teuthology.orchestra.run.smithi042.stdout: mon host = "172.21.15.28,172.21.15.42,172.21.15.66" 2024-04-26T03:37:14.862 INFO:teuthology.orchestra.run.smithi042.stdout: mon client directed command retry = 5 2024-04-26T03:37:14.862 INFO:teuthology.orchestra.run.smithi042.stdout: mon election default strategy = 1 2024-04-26T03:37:14.862 INFO:teuthology.orchestra.run.smithi042.stdout: ms inject socket failures = 2500 2024-04-26T03:37:14.862 INFO:teuthology.orchestra.run.smithi042.stdout: ms tcp read timeout = 5 2024-04-26T03:37:14.862 INFO:teuthology.orchestra.run.smithi042.stdout: fsid = c5a13c99-5c9a-4b8b-a893-4f6f4211677a 2024-04-26T03:37:14.862 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:37:14.862 INFO:teuthology.orchestra.run.smithi042.stdout:[osd] 2024-04-26T03:37:14.862 INFO:teuthology.orchestra.run.smithi042.stdout: osd journal size = 100 2024-04-26T03:37:14.862 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:37:14.863 INFO:teuthology.orchestra.run.smithi042.stdout: osd scrub load threshold = 5.0 2024-04-26T03:37:14.863 INFO:teuthology.orchestra.run.smithi042.stdout: osd scrub max interval = 120 2024-04-26T03:37:14.863 INFO:teuthology.orchestra.run.smithi042.stdout: osd mclock profile = high_recovery_ops 2024-04-26T03:37:14.863 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:37:14.863 INFO:teuthology.orchestra.run.smithi042.stdout: osd recover clone overlap = true 2024-04-26T03:37:14.863 INFO:teuthology.orchestra.run.smithi042.stdout: osd recovery max chunk = 1048576 2024-04-26T03:37:14.863 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:37:14.863 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug shutdown = true 2024-04-26T03:37:14.863 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug op order = true 2024-04-26T03:37:14.863 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug verify stray on activate = true 2024-04-26T03:37:14.863 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug trim objects = true 2024-04-26T03:37:14.863 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:37:14.863 INFO:teuthology.orchestra.run.smithi042.stdout: osd open classes on start = true 2024-04-26T03:37:14.863 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug pg log writeout = true 2024-04-26T03:37:14.863 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:37:14.864 INFO:teuthology.orchestra.run.smithi042.stdout: osd deep scrub update digest min age = 30 2024-04-26T03:37:14.864 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:37:14.864 INFO:teuthology.orchestra.run.smithi042.stdout: osd map max advance = 10 2024-04-26T03:37:14.864 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:37:14.864 INFO:teuthology.orchestra.run.smithi042.stdout: journal zero on create = true 2024-04-26T03:37:14.864 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:37:14.864 INFO:teuthology.orchestra.run.smithi042.stdout: filestore ondisk finisher threads = 3 2024-04-26T03:37:14.864 INFO:teuthology.orchestra.run.smithi042.stdout: filestore apply finisher threads = 3 2024-04-26T03:37:14.864 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:37:14.864 INFO:teuthology.orchestra.run.smithi042.stdout: bdev debug aio = true 2024-04-26T03:37:14.864 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug misdirected ops = true 2024-04-26T03:37:14.864 INFO:teuthology.orchestra.run.smithi042.stdout: debug monc = 20 2024-04-26T03:37:14.864 INFO:teuthology.orchestra.run.smithi042.stdout: bluestore block size = 96636764160 2024-04-26T03:37:14.864 INFO:teuthology.orchestra.run.smithi042.stdout: bluestore compression algorithm = zlib 2024-04-26T03:37:14.865 INFO:teuthology.orchestra.run.smithi042.stdout: bluestore compression mode = aggressive 2024-04-26T03:37:14.865 INFO:teuthology.orchestra.run.smithi042.stdout: bluestore fsck on mount = True 2024-04-26T03:37:14.865 INFO:teuthology.orchestra.run.smithi042.stdout: bluestore zero block detection = True 2024-04-26T03:37:14.865 INFO:teuthology.orchestra.run.smithi042.stdout: debug bluefs = 20 2024-04-26T03:37:14.865 INFO:teuthology.orchestra.run.smithi042.stdout: debug bluestore = 20 2024-04-26T03:37:14.865 INFO:teuthology.orchestra.run.smithi042.stdout: debug ms = 1 2024-04-26T03:37:14.865 INFO:teuthology.orchestra.run.smithi042.stdout: debug osd = 20 2024-04-26T03:37:14.865 INFO:teuthology.orchestra.run.smithi042.stdout: debug rocksdb = 10 2024-04-26T03:37:14.865 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd backfillfull_ratio = 0.85 2024-04-26T03:37:14.865 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd full ratio = 0.9 2024-04-26T03:37:14.865 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd nearfull ratio = 0.8 2024-04-26T03:37:14.865 INFO:teuthology.orchestra.run.smithi042.stdout: osd blocked scrub grace period = 3600 2024-04-26T03:37:14.865 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug reject backfill probability = 0.3 2024-04-26T03:37:14.865 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug verify cached snaps = True 2024-04-26T03:37:14.865 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug verify missing on start = True 2024-04-26T03:37:14.866 INFO:teuthology.orchestra.run.smithi042.stdout: osd failsafe full ratio = 0.95 2024-04-26T03:37:14.866 INFO:teuthology.orchestra.run.smithi042.stdout: osd max backfills = 3 2024-04-26T03:37:14.866 INFO:teuthology.orchestra.run.smithi042.stdout: osd max markdown count = 1000 2024-04-26T03:37:14.866 INFO:teuthology.orchestra.run.smithi042.stdout: osd mclock override recovery settings = True 2024-04-26T03:37:14.866 INFO:teuthology.orchestra.run.smithi042.stdout: osd objectstore = bluestore 2024-04-26T03:37:14.866 INFO:teuthology.orchestra.run.smithi042.stdout: osd op queue = debug_random 2024-04-26T03:37:14.866 INFO:teuthology.orchestra.run.smithi042.stdout: osd op queue cut off = debug_random 2024-04-26T03:37:14.866 INFO:teuthology.orchestra.run.smithi042.stdout: osd scrub min interval = 60 2024-04-26T03:37:14.866 INFO:teuthology.orchestra.run.smithi042.stdout: osd shutdown pgref assert = True 2024-04-26T03:37:14.866 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:37:14.866 INFO:teuthology.orchestra.run.smithi042.stdout:[mgr] 2024-04-26T03:37:14.866 INFO:teuthology.orchestra.run.smithi042.stdout: debug ms = 1 2024-04-26T03:37:14.866 INFO:teuthology.orchestra.run.smithi042.stdout: debug mgr = 20 2024-04-26T03:37:14.866 INFO:teuthology.orchestra.run.smithi042.stdout: debug mon = 20 2024-04-26T03:37:14.867 INFO:teuthology.orchestra.run.smithi042.stdout: debug auth = 20 2024-04-26T03:37:14.867 INFO:teuthology.orchestra.run.smithi042.stdout: mon reweight min pgs per osd = 4 2024-04-26T03:37:14.867 INFO:teuthology.orchestra.run.smithi042.stdout: mon reweight min bytes per osd = 10 2024-04-26T03:37:14.867 INFO:teuthology.orchestra.run.smithi042.stdout: mgr/telemetry/nag = false 2024-04-26T03:37:14.867 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:37:14.867 INFO:teuthology.orchestra.run.smithi042.stdout:[mon] 2024-04-26T03:37:14.867 INFO:teuthology.orchestra.run.smithi042.stdout: debug ms = 1 2024-04-26T03:37:14.867 INFO:teuthology.orchestra.run.smithi042.stdout: debug mon = 20 2024-04-26T03:37:14.867 INFO:teuthology.orchestra.run.smithi042.stdout: debug paxos = 20 2024-04-26T03:37:14.867 INFO:teuthology.orchestra.run.smithi042.stdout: debug auth = 20 2024-04-26T03:37:14.867 INFO:teuthology.orchestra.run.smithi042.stdout: mon data avail warn = 5 2024-04-26T03:37:14.867 INFO:teuthology.orchestra.run.smithi042.stdout: mon mgr mkfs grace = 240 2024-04-26T03:37:14.867 INFO:teuthology.orchestra.run.smithi042.stdout: mon reweight min pgs per osd = 4 2024-04-26T03:37:14.867 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd reporter subtree level = osd 2024-04-26T03:37:14.867 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd prime pg temp = true 2024-04-26T03:37:14.868 INFO:teuthology.orchestra.run.smithi042.stdout: mon reweight min bytes per osd = 10 2024-04-26T03:37:14.868 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:37:14.868 INFO:teuthology.orchestra.run.smithi042.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-26T03:37:14.868 INFO:teuthology.orchestra.run.smithi042.stdout: auth mon ticket ttl = 660 # 11m 2024-04-26T03:37:14.868 INFO:teuthology.orchestra.run.smithi042.stdout: auth service ticket ttl = 240 # 4m 2024-04-26T03:37:14.868 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:37:14.868 INFO:teuthology.orchestra.run.smithi042.stdout: # don't complain about insecure global_id in the test suite 2024-04-26T03:37:14.868 INFO:teuthology.orchestra.run.smithi042.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-26T03:37:14.868 INFO:teuthology.orchestra.run.smithi042.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-26T03:37:14.868 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:37:14.868 INFO:teuthology.orchestra.run.smithi042.stdout: # 1m isn't quite enough 2024-04-26T03:37:14.868 INFO:teuthology.orchestra.run.smithi042.stdout: mon_down_mkfs_grace = 2m 2024-04-26T03:37:14.868 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:37:14.868 INFO:teuthology.orchestra.run.smithi042.stdout: mon_warn_on_filestore_osds = false 2024-04-26T03:37:14.869 INFO:teuthology.orchestra.run.smithi042.stdout: mon scrub interval = 300 2024-04-26T03:37:14.869 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-26T03:37:14.869 INFO:teuthology.orchestra.run.smithi042.stdout:[client] 2024-04-26T03:37:14.869 INFO:teuthology.orchestra.run.smithi042.stdout: rgw cache enabled = true 2024-04-26T03:37:14.869 INFO:teuthology.orchestra.run.smithi042.stdout: rgw enable ops log = true 2024-04-26T03:37:14.869 INFO:teuthology.orchestra.run.smithi042.stdout: rgw enable usage log = true 2024-04-26T03:37:14.869 INFO:teuthology.orchestra.run.smithi042.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-26T03:37:14.869 INFO:teuthology.orchestra.run.smithi042.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-26T03:37:14.869 INFO:teuthology.orchestra.run.smithi042.stdout:[mon.a] 2024-04-26T03:37:14.869 INFO:teuthology.orchestra.run.smithi042.stdout:[mon.b] 2024-04-26T03:37:14.869 INFO:teuthology.orchestra.run.smithi042.stdout:[mon.c] 2024-04-26T03:37:14.872 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-26T03:37:14.872 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-26T03:37:14.965 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-26T03:37:14.965 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T03:37:14.965 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-26T03:37:14.981 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T03:37:14.981 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-26T03:37:15.037 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi028.front.sepia.ceph.com 2024-04-26T03:37:15.037 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T03:37:15.037 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-26T03:37:15.037 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T03:37:15.110 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T03:37:15.111 DEBUG:teuthology.orchestra.run.smithi028:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-26T03:37:15.166 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi042.front.sepia.ceph.com 2024-04-26T03:37:15.166 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-26T03:37:15.167 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-26T03:37:15.167 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T03:37:15.206 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-26T03:37:15.206 DEBUG:teuthology.orchestra.run.smithi042:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-26T03:37:15.265 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi066.front.sepia.ceph.com 2024-04-26T03:37:15.265 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-26T03:37:15.265 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-26T03:37:15.266 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T03:37:15.305 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-26T03:37:15.305 DEBUG:teuthology.orchestra.run.smithi066:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-26T03:37:15.363 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi092.front.sepia.ceph.com 2024-04-26T03:37:15.363 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-26T03:37:15.363 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-26T03:37:15.363 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T03:37:15.400 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-26T03:37:15.400 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-26T03:37:15.457 INFO:tasks.ceph:Setting up mon nodes... 2024-04-26T03:37:15.457 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-26T03:37:15.458 DEBUG:teuthology.orchestra.run.smithi028:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-26T03:37:15.508 INFO:teuthology.orchestra.run.smithi028.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-26T03:37:15.510 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-26T03:37:15.584 INFO:teuthology.orchestra.run.smithi092.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-26T03:37:15.587 INFO:tasks.ceph:Setting up mds nodes... 2024-04-26T03:37:15.587 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-26T03:37:15.587 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-26T03:37:15.627 INFO:teuthology.orchestra.run.smithi092.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-26T03:37:15.639 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-26T03:37:15.639 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi028.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi042.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi066.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi092.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'}} 2024-04-26T03:37:15.639 DEBUG:teuthology.orchestra.run.smithi028:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-26T03:37:15.664 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2024-04-26T03:37:15.664 INFO:tasks.ceph:role: osd.0 2024-04-26T03:37:15.665 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi028.front.sepia.ceph.com 2024-04-26T03:37:15.665 DEBUG:teuthology.orchestra.run.smithi028:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-26T03:37:15.729 INFO:teuthology.orchestra.run.smithi028.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T03:37:15.729 INFO:teuthology.orchestra.run.smithi028.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T03:37:15.729 INFO:teuthology.orchestra.run.smithi028.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T03:37:15.730 INFO:teuthology.orchestra.run.smithi028.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T03:37:15.730 INFO:teuthology.orchestra.run.smithi028.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T03:37:15.730 INFO:teuthology.orchestra.run.smithi028.stdout: = sunit=0 swidth=0 blks 2024-04-26T03:37:15.730 INFO:teuthology.orchestra.run.smithi028.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T03:37:15.730 INFO:teuthology.orchestra.run.smithi028.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T03:37:15.730 INFO:teuthology.orchestra.run.smithi028.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T03:37:15.730 INFO:teuthology.orchestra.run.smithi028.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T03:37:15.762 INFO:teuthology.orchestra.run.smithi028.stdout:Discarding blocks...Done. 2024-04-26T03:37:15.763 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi028.front.sepia.ceph.com -o noatime 2024-04-26T03:37:15.764 DEBUG:teuthology.orchestra.run.smithi028:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-26T03:37:15.838 DEBUG:teuthology.orchestra.run.smithi028:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-26T03:37:15.905 DEBUG:teuthology.orchestra.run.smithi028:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-26T03:37:15.971 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2024-04-26T03:37:15.971 INFO:tasks.ceph:role: osd.4 2024-04-26T03:37:15.971 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi028.front.sepia.ceph.com 2024-04-26T03:37:15.971 DEBUG:teuthology.orchestra.run.smithi028:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-26T03:37:16.035 INFO:teuthology.orchestra.run.smithi028.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T03:37:16.035 INFO:teuthology.orchestra.run.smithi028.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T03:37:16.035 INFO:teuthology.orchestra.run.smithi028.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T03:37:16.035 INFO:teuthology.orchestra.run.smithi028.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T03:37:16.035 INFO:teuthology.orchestra.run.smithi028.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T03:37:16.035 INFO:teuthology.orchestra.run.smithi028.stdout: = sunit=0 swidth=0 blks 2024-04-26T03:37:16.035 INFO:teuthology.orchestra.run.smithi028.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T03:37:16.035 INFO:teuthology.orchestra.run.smithi028.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T03:37:16.035 INFO:teuthology.orchestra.run.smithi028.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T03:37:16.035 INFO:teuthology.orchestra.run.smithi028.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T03:37:16.067 INFO:teuthology.orchestra.run.smithi028.stdout:Discarding blocks...Done. 2024-04-26T03:37:16.069 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi028.front.sepia.ceph.com -o noatime 2024-04-26T03:37:16.069 DEBUG:teuthology.orchestra.run.smithi028:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-4 2024-04-26T03:37:16.139 DEBUG:teuthology.orchestra.run.smithi028:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-26T03:37:16.207 DEBUG:teuthology.orchestra.run.smithi028:> sudo mkdir -p /var/lib/ceph/osd/ceph-8 2024-04-26T03:37:16.272 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2024-04-26T03:37:16.273 INFO:tasks.ceph:role: osd.8 2024-04-26T03:37:16.273 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi028.front.sepia.ceph.com 2024-04-26T03:37:16.273 DEBUG:teuthology.orchestra.run.smithi028:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-26T03:37:16.337 INFO:teuthology.orchestra.run.smithi028.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T03:37:16.337 INFO:teuthology.orchestra.run.smithi028.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T03:37:16.337 INFO:teuthology.orchestra.run.smithi028.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T03:37:16.337 INFO:teuthology.orchestra.run.smithi028.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T03:37:16.337 INFO:teuthology.orchestra.run.smithi028.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T03:37:16.337 INFO:teuthology.orchestra.run.smithi028.stdout: = sunit=0 swidth=0 blks 2024-04-26T03:37:16.337 INFO:teuthology.orchestra.run.smithi028.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T03:37:16.337 INFO:teuthology.orchestra.run.smithi028.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T03:37:16.337 INFO:teuthology.orchestra.run.smithi028.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T03:37:16.338 INFO:teuthology.orchestra.run.smithi028.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T03:37:16.370 INFO:teuthology.orchestra.run.smithi028.stdout:Discarding blocks...Done. 2024-04-26T03:37:16.372 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi028.front.sepia.ceph.com -o noatime 2024-04-26T03:37:16.372 DEBUG:teuthology.orchestra.run.smithi028:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-8 2024-04-26T03:37:16.444 DEBUG:teuthology.orchestra.run.smithi028:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-8 2024-04-26T03:37:16.513 DEBUG:teuthology.orchestra.run.smithi028:> sudo mkdir -p /var/lib/ceph/osd/ceph-12 2024-04-26T03:37:16.578 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2024-04-26T03:37:16.578 INFO:tasks.ceph:role: osd.12 2024-04-26T03:37:16.578 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi028.front.sepia.ceph.com 2024-04-26T03:37:16.578 DEBUG:teuthology.orchestra.run.smithi028:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-26T03:37:16.644 INFO:teuthology.orchestra.run.smithi028.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T03:37:16.644 INFO:teuthology.orchestra.run.smithi028.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T03:37:16.644 INFO:teuthology.orchestra.run.smithi028.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T03:37:16.644 INFO:teuthology.orchestra.run.smithi028.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T03:37:16.644 INFO:teuthology.orchestra.run.smithi028.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T03:37:16.644 INFO:teuthology.orchestra.run.smithi028.stdout: = sunit=0 swidth=0 blks 2024-04-26T03:37:16.644 INFO:teuthology.orchestra.run.smithi028.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T03:37:16.644 INFO:teuthology.orchestra.run.smithi028.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T03:37:16.644 INFO:teuthology.orchestra.run.smithi028.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T03:37:16.644 INFO:teuthology.orchestra.run.smithi028.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T03:37:16.676 INFO:teuthology.orchestra.run.smithi028.stdout:Discarding blocks...Done. 2024-04-26T03:37:16.678 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi028.front.sepia.ceph.com -o noatime 2024-04-26T03:37:16.678 DEBUG:teuthology.orchestra.run.smithi028:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-12 2024-04-26T03:37:16.748 DEBUG:teuthology.orchestra.run.smithi028:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-12 2024-04-26T03:37:16.816 DEBUG:teuthology.orchestra.run.smithi028:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T03:37:16.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:37:16.903+0000 7fcb2f1dc640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-26T03:37:16.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:37:16.903+0000 7fcb2f1dc640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-26T03:37:16.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:37:16.903+0000 7fcb2f1dc640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T03:37:16.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:37:16.903+0000 7fcb2f1dc640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T03:37:16.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:37:16.903+0000 7fcb2f1dc640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T03:37:16.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:37:16.904+0000 7fcb2f1dc640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-26T03:37:19.759 DEBUG:teuthology.orchestra.run.smithi028:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-26T03:37:19.789 DEBUG:teuthology.orchestra.run.smithi028:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T03:37:19.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:37:19.876+0000 7f7790621640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-26T03:37:19.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:37:19.876+0000 7f7790621640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-26T03:37:19.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:37:19.876+0000 7f7790621640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T03:37:19.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:37:19.876+0000 7f7790621640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T03:37:19.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:37:19.876+0000 7f7790621640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T03:37:19.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:37:19.877+0000 7f7790621640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-26T03:37:22.728 DEBUG:teuthology.orchestra.run.smithi028:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-26T03:37:22.757 DEBUG:teuthology.orchestra.run.smithi028:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 8 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T03:37:22.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:37:22.846+0000 7fa602021640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-8/keyring: can't open /var/lib/ceph/osd/ceph-8/keyring: (2) No such file or directory 2024-04-26T03:37:22.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:37:22.846+0000 7fa602021640 -1 created new key in keyring /var/lib/ceph/osd/ceph-8/keyring 2024-04-26T03:37:22.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:37:22.846+0000 7fa602021640 -1 bluestore(/var/lib/ceph/osd/ceph-8/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-8/block: (2) No such file or directory 2024-04-26T03:37:22.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:37:22.846+0000 7fa602021640 -1 bluestore(/var/lib/ceph/osd/ceph-8/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-8/block: (2) No such file or directory 2024-04-26T03:37:22.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:37:22.846+0000 7fa602021640 -1 bluestore(/var/lib/ceph/osd/ceph-8/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-8/block: (2) No such file or directory 2024-04-26T03:37:22.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:37:22.847+0000 7fa602021640 -1 bluestore(/var/lib/ceph/osd/ceph-8) _read_fsid unparsable uuid 2024-04-26T03:37:25.692 DEBUG:teuthology.orchestra.run.smithi028:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-8 2024-04-26T03:37:25.721 DEBUG:teuthology.orchestra.run.smithi028:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 12 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T03:37:25.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:37:25.813+0000 7f5e7e61f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-12/keyring: can't open /var/lib/ceph/osd/ceph-12/keyring: (2) No such file or directory 2024-04-26T03:37:25.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:37:25.813+0000 7f5e7e61f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-12/keyring 2024-04-26T03:37:25.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:37:25.813+0000 7f5e7e61f640 -1 bluestore(/var/lib/ceph/osd/ceph-12/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-12/block: (2) No such file or directory 2024-04-26T03:37:25.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:37:25.813+0000 7f5e7e61f640 -1 bluestore(/var/lib/ceph/osd/ceph-12/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-12/block: (2) No such file or directory 2024-04-26T03:37:25.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:37:25.813+0000 7f5e7e61f640 -1 bluestore(/var/lib/ceph/osd/ceph-12/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-12/block: (2) No such file or directory 2024-04-26T03:37:25.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:37:25.813+0000 7f5e7e61f640 -1 bluestore(/var/lib/ceph/osd/ceph-12) _read_fsid unparsable uuid 2024-04-26T03:37:28.667 DEBUG:teuthology.orchestra.run.smithi028:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-12 2024-04-26T03:37:28.696 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-26T03:37:28.726 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2024-04-26T03:37:28.726 INFO:tasks.ceph:role: osd.1 2024-04-26T03:37:28.726 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi042.front.sepia.ceph.com 2024-04-26T03:37:28.726 DEBUG:teuthology.orchestra.run.smithi042:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-26T03:37:28.794 INFO:teuthology.orchestra.run.smithi042.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T03:37:28.795 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T03:37:28.795 INFO:teuthology.orchestra.run.smithi042.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T03:37:28.795 INFO:teuthology.orchestra.run.smithi042.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T03:37:28.795 INFO:teuthology.orchestra.run.smithi042.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T03:37:28.795 INFO:teuthology.orchestra.run.smithi042.stdout: = sunit=0 swidth=0 blks 2024-04-26T03:37:28.795 INFO:teuthology.orchestra.run.smithi042.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T03:37:28.795 INFO:teuthology.orchestra.run.smithi042.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T03:37:28.795 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T03:37:28.795 INFO:teuthology.orchestra.run.smithi042.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T03:37:28.827 INFO:teuthology.orchestra.run.smithi042.stdout:Discarding blocks...Done. 2024-04-26T03:37:28.829 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi042.front.sepia.ceph.com -o noatime 2024-04-26T03:37:28.829 DEBUG:teuthology.orchestra.run.smithi042:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-1 2024-04-26T03:37:28.898 DEBUG:teuthology.orchestra.run.smithi042:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-26T03:37:28.969 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-26T03:37:29.035 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2024-04-26T03:37:29.035 INFO:tasks.ceph:role: osd.5 2024-04-26T03:37:29.035 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi042.front.sepia.ceph.com 2024-04-26T03:37:29.035 DEBUG:teuthology.orchestra.run.smithi042:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-26T03:37:29.099 INFO:teuthology.orchestra.run.smithi042.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T03:37:29.099 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T03:37:29.099 INFO:teuthology.orchestra.run.smithi042.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T03:37:29.099 INFO:teuthology.orchestra.run.smithi042.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T03:37:29.099 INFO:teuthology.orchestra.run.smithi042.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T03:37:29.099 INFO:teuthology.orchestra.run.smithi042.stdout: = sunit=0 swidth=0 blks 2024-04-26T03:37:29.099 INFO:teuthology.orchestra.run.smithi042.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T03:37:29.099 INFO:teuthology.orchestra.run.smithi042.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T03:37:29.099 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T03:37:29.100 INFO:teuthology.orchestra.run.smithi042.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T03:37:29.130 INFO:teuthology.orchestra.run.smithi042.stdout:Discarding blocks...Done. 2024-04-26T03:37:29.132 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi042.front.sepia.ceph.com -o noatime 2024-04-26T03:37:29.132 DEBUG:teuthology.orchestra.run.smithi042:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-26T03:37:29.201 DEBUG:teuthology.orchestra.run.smithi042:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-26T03:37:29.270 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/osd/ceph-9 2024-04-26T03:37:29.335 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2024-04-26T03:37:29.335 INFO:tasks.ceph:role: osd.9 2024-04-26T03:37:29.335 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi042.front.sepia.ceph.com 2024-04-26T03:37:29.336 DEBUG:teuthology.orchestra.run.smithi042:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-26T03:37:29.401 INFO:teuthology.orchestra.run.smithi042.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T03:37:29.401 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T03:37:29.401 INFO:teuthology.orchestra.run.smithi042.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T03:37:29.401 INFO:teuthology.orchestra.run.smithi042.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T03:37:29.401 INFO:teuthology.orchestra.run.smithi042.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T03:37:29.401 INFO:teuthology.orchestra.run.smithi042.stdout: = sunit=0 swidth=0 blks 2024-04-26T03:37:29.401 INFO:teuthology.orchestra.run.smithi042.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T03:37:29.401 INFO:teuthology.orchestra.run.smithi042.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T03:37:29.401 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T03:37:29.401 INFO:teuthology.orchestra.run.smithi042.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T03:37:29.433 INFO:teuthology.orchestra.run.smithi042.stdout:Discarding blocks...Done. 2024-04-26T03:37:29.435 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi042.front.sepia.ceph.com -o noatime 2024-04-26T03:37:29.435 DEBUG:teuthology.orchestra.run.smithi042:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-9 2024-04-26T03:37:29.502 DEBUG:teuthology.orchestra.run.smithi042:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-9 2024-04-26T03:37:29.571 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/osd/ceph-13 2024-04-26T03:37:29.635 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2024-04-26T03:37:29.635 INFO:tasks.ceph:role: osd.13 2024-04-26T03:37:29.635 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi042.front.sepia.ceph.com 2024-04-26T03:37:29.635 DEBUG:teuthology.orchestra.run.smithi042:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-26T03:37:29.699 INFO:teuthology.orchestra.run.smithi042.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T03:37:29.699 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T03:37:29.699 INFO:teuthology.orchestra.run.smithi042.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T03:37:29.699 INFO:teuthology.orchestra.run.smithi042.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T03:37:29.699 INFO:teuthology.orchestra.run.smithi042.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T03:37:29.699 INFO:teuthology.orchestra.run.smithi042.stdout: = sunit=0 swidth=0 blks 2024-04-26T03:37:29.699 INFO:teuthology.orchestra.run.smithi042.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T03:37:29.699 INFO:teuthology.orchestra.run.smithi042.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T03:37:29.699 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T03:37:29.700 INFO:teuthology.orchestra.run.smithi042.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T03:37:29.731 INFO:teuthology.orchestra.run.smithi042.stdout:Discarding blocks...Done. 2024-04-26T03:37:29.733 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi042.front.sepia.ceph.com -o noatime 2024-04-26T03:37:29.733 DEBUG:teuthology.orchestra.run.smithi042:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-13 2024-04-26T03:37:29.802 DEBUG:teuthology.orchestra.run.smithi042:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-13 2024-04-26T03:37:29.871 DEBUG:teuthology.orchestra.run.smithi042:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T03:37:29.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-26T03:37:29.961+0000 7fd7c7df1640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-26T03:37:29.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-26T03:37:29.961+0000 7fd7c7df1640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-26T03:37:29.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-26T03:37:29.961+0000 7fd7c7df1640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T03:37:29.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-26T03:37:29.961+0000 7fd7c7df1640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T03:37:29.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-26T03:37:29.961+0000 7fd7c7df1640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T03:37:29.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-26T03:37:29.962+0000 7fd7c7df1640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-26T03:37:32.806 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-26T03:37:32.835 DEBUG:teuthology.orchestra.run.smithi042:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T03:37:32.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-26T03:37:32.923+0000 7f716381f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-26T03:37:32.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-26T03:37:32.924+0000 7f716381f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-26T03:37:32.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-26T03:37:32.924+0000 7f716381f640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T03:37:32.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-26T03:37:32.924+0000 7f716381f640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T03:37:32.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-26T03:37:32.924+0000 7f716381f640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T03:37:32.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-26T03:37:32.924+0000 7f716381f640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-26T03:37:35.773 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-26T03:37:35.803 DEBUG:teuthology.orchestra.run.smithi042:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 9 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T03:37:35.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-26T03:37:35.891+0000 7f56fe80e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-9/keyring: can't open /var/lib/ceph/osd/ceph-9/keyring: (2) No such file or directory 2024-04-26T03:37:35.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-26T03:37:35.891+0000 7f56fe80e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-9/keyring 2024-04-26T03:37:35.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-26T03:37:35.891+0000 7f56fe80e640 -1 bluestore(/var/lib/ceph/osd/ceph-9/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-9/block: (2) No such file or directory 2024-04-26T03:37:35.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-26T03:37:35.891+0000 7f56fe80e640 -1 bluestore(/var/lib/ceph/osd/ceph-9/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-9/block: (2) No such file or directory 2024-04-26T03:37:35.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-26T03:37:35.891+0000 7f56fe80e640 -1 bluestore(/var/lib/ceph/osd/ceph-9/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-9/block: (2) No such file or directory 2024-04-26T03:37:35.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-26T03:37:35.892+0000 7f56fe80e640 -1 bluestore(/var/lib/ceph/osd/ceph-9) _read_fsid unparsable uuid 2024-04-26T03:37:38.746 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-9 2024-04-26T03:37:38.774 DEBUG:teuthology.orchestra.run.smithi042:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 13 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T03:37:38.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-26T03:37:38.859+0000 7f8ade80c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-13/keyring: can't open /var/lib/ceph/osd/ceph-13/keyring: (2) No such file or directory 2024-04-26T03:37:38.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-26T03:37:38.859+0000 7f8ade80c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-13/keyring 2024-04-26T03:37:38.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-26T03:37:38.859+0000 7f8ade80c640 -1 bluestore(/var/lib/ceph/osd/ceph-13/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-13/block: (2) No such file or directory 2024-04-26T03:37:38.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-26T03:37:38.859+0000 7f8ade80c640 -1 bluestore(/var/lib/ceph/osd/ceph-13/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-13/block: (2) No such file or directory 2024-04-26T03:37:38.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-26T03:37:38.859+0000 7f8ade80c640 -1 bluestore(/var/lib/ceph/osd/ceph-13/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-13/block: (2) No such file or directory 2024-04-26T03:37:38.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-26T03:37:38.860+0000 7f8ade80c640 -1 bluestore(/var/lib/ceph/osd/ceph-13) _read_fsid unparsable uuid 2024-04-26T03:37:41.720 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-13 2024-04-26T03:37:41.749 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-26T03:37:41.779 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2024-04-26T03:37:41.779 INFO:tasks.ceph:role: osd.2 2024-04-26T03:37:41.779 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi066.front.sepia.ceph.com 2024-04-26T03:37:41.779 DEBUG:teuthology.orchestra.run.smithi066:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-26T03:37:41.855 INFO:teuthology.orchestra.run.smithi066.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T03:37:41.856 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T03:37:41.856 INFO:teuthology.orchestra.run.smithi066.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T03:37:41.856 INFO:teuthology.orchestra.run.smithi066.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T03:37:41.856 INFO:teuthology.orchestra.run.smithi066.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T03:37:41.856 INFO:teuthology.orchestra.run.smithi066.stdout: = sunit=0 swidth=0 blks 2024-04-26T03:37:41.856 INFO:teuthology.orchestra.run.smithi066.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T03:37:41.856 INFO:teuthology.orchestra.run.smithi066.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T03:37:41.856 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T03:37:41.856 INFO:teuthology.orchestra.run.smithi066.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T03:37:42.355 INFO:teuthology.orchestra.run.smithi066.stdout:Discarding blocks...Done. 2024-04-26T03:37:42.357 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi066.front.sepia.ceph.com -o noatime 2024-04-26T03:37:42.357 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-2 2024-04-26T03:37:42.496 DEBUG:teuthology.orchestra.run.smithi066:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-26T03:37:42.530 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-26T03:37:42.595 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2024-04-26T03:37:42.595 INFO:tasks.ceph:role: osd.6 2024-04-26T03:37:42.595 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi066.front.sepia.ceph.com 2024-04-26T03:37:42.595 DEBUG:teuthology.orchestra.run.smithi066:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-26T03:37:42.667 INFO:teuthology.orchestra.run.smithi066.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T03:37:42.668 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T03:37:42.668 INFO:teuthology.orchestra.run.smithi066.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T03:37:42.668 INFO:teuthology.orchestra.run.smithi066.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T03:37:42.668 INFO:teuthology.orchestra.run.smithi066.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T03:37:42.668 INFO:teuthology.orchestra.run.smithi066.stdout: = sunit=0 swidth=0 blks 2024-04-26T03:37:42.668 INFO:teuthology.orchestra.run.smithi066.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T03:37:42.668 INFO:teuthology.orchestra.run.smithi066.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T03:37:42.668 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T03:37:42.668 INFO:teuthology.orchestra.run.smithi066.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T03:37:43.135 INFO:teuthology.orchestra.run.smithi066.stdout:Discarding blocks...Done. 2024-04-26T03:37:43.138 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi066.front.sepia.ceph.com -o noatime 2024-04-26T03:37:43.138 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-6 2024-04-26T03:37:43.219 DEBUG:teuthology.orchestra.run.smithi066:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-26T03:37:43.288 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/osd/ceph-10 2024-04-26T03:37:43.352 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2024-04-26T03:37:43.353 INFO:tasks.ceph:role: osd.10 2024-04-26T03:37:43.353 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi066.front.sepia.ceph.com 2024-04-26T03:37:43.353 DEBUG:teuthology.orchestra.run.smithi066:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-26T03:37:43.425 INFO:teuthology.orchestra.run.smithi066.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T03:37:43.425 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T03:37:43.425 INFO:teuthology.orchestra.run.smithi066.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T03:37:43.425 INFO:teuthology.orchestra.run.smithi066.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T03:37:43.425 INFO:teuthology.orchestra.run.smithi066.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T03:37:43.425 INFO:teuthology.orchestra.run.smithi066.stdout: = sunit=0 swidth=0 blks 2024-04-26T03:37:43.425 INFO:teuthology.orchestra.run.smithi066.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T03:37:43.425 INFO:teuthology.orchestra.run.smithi066.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T03:37:43.425 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T03:37:43.425 INFO:teuthology.orchestra.run.smithi066.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T03:37:43.946 INFO:teuthology.orchestra.run.smithi066.stdout:Discarding blocks...Done. 2024-04-26T03:37:43.948 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi066.front.sepia.ceph.com -o noatime 2024-04-26T03:37:43.948 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-10 2024-04-26T03:37:44.083 DEBUG:teuthology.orchestra.run.smithi066:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-10 2024-04-26T03:37:44.115 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/osd/ceph-14 2024-04-26T03:37:44.182 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2024-04-26T03:37:44.182 INFO:tasks.ceph:role: osd.14 2024-04-26T03:37:44.182 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi066.front.sepia.ceph.com 2024-04-26T03:37:44.182 DEBUG:teuthology.orchestra.run.smithi066:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-26T03:37:44.256 INFO:teuthology.orchestra.run.smithi066.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T03:37:44.256 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T03:37:44.256 INFO:teuthology.orchestra.run.smithi066.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T03:37:44.256 INFO:teuthology.orchestra.run.smithi066.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T03:37:44.256 INFO:teuthology.orchestra.run.smithi066.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T03:37:44.257 INFO:teuthology.orchestra.run.smithi066.stdout: = sunit=0 swidth=0 blks 2024-04-26T03:37:44.257 INFO:teuthology.orchestra.run.smithi066.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T03:37:44.257 INFO:teuthology.orchestra.run.smithi066.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T03:37:44.257 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T03:37:44.257 INFO:teuthology.orchestra.run.smithi066.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T03:37:44.769 INFO:teuthology.orchestra.run.smithi066.stdout:Discarding blocks...Done. 2024-04-26T03:37:44.772 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi066.front.sepia.ceph.com -o noatime 2024-04-26T03:37:44.772 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-14 2024-04-26T03:37:44.869 DEBUG:teuthology.orchestra.run.smithi066:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-14 2024-04-26T03:37:44.902 DEBUG:teuthology.orchestra.run.smithi066:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T03:37:45.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-26T03:37:45.002+0000 7f16d7d25640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-26T03:37:45.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-26T03:37:45.003+0000 7f16d7d25640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-26T03:37:45.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-26T03:37:45.003+0000 7f16d7d25640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T03:37:45.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-26T03:37:45.003+0000 7f16d7d25640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T03:37:45.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-26T03:37:45.003+0000 7f16d7d25640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T03:37:45.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-26T03:37:45.003+0000 7f16d7d25640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-26T03:37:47.862 DEBUG:teuthology.orchestra.run.smithi066:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-26T03:37:47.892 DEBUG:teuthology.orchestra.run.smithi066:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T03:37:47.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-26T03:37:47.980+0000 7f0787018640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-26T03:37:47.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-26T03:37:47.980+0000 7f0787018640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-26T03:37:47.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-26T03:37:47.980+0000 7f0787018640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T03:37:47.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-26T03:37:47.980+0000 7f0787018640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T03:37:47.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-26T03:37:47.980+0000 7f0787018640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T03:37:47.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-26T03:37:47.981+0000 7f0787018640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-26T03:37:50.840 DEBUG:teuthology.orchestra.run.smithi066:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-26T03:37:50.869 DEBUG:teuthology.orchestra.run.smithi066:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 10 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T03:37:50.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-26T03:37:50.957+0000 7f2b4621c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-10/keyring: can't open /var/lib/ceph/osd/ceph-10/keyring: (2) No such file or directory 2024-04-26T03:37:50.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-26T03:37:50.957+0000 7f2b4621c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-10/keyring 2024-04-26T03:37:50.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-26T03:37:50.957+0000 7f2b4621c640 -1 bluestore(/var/lib/ceph/osd/ceph-10/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-10/block: (2) No such file or directory 2024-04-26T03:37:50.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-26T03:37:50.957+0000 7f2b4621c640 -1 bluestore(/var/lib/ceph/osd/ceph-10/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-10/block: (2) No such file or directory 2024-04-26T03:37:50.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-26T03:37:50.957+0000 7f2b4621c640 -1 bluestore(/var/lib/ceph/osd/ceph-10/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-10/block: (2) No such file or directory 2024-04-26T03:37:50.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-26T03:37:50.957+0000 7f2b4621c640 -1 bluestore(/var/lib/ceph/osd/ceph-10) _read_fsid unparsable uuid 2024-04-26T03:37:53.818 DEBUG:teuthology.orchestra.run.smithi066:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-10 2024-04-26T03:37:53.846 DEBUG:teuthology.orchestra.run.smithi066:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 14 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T03:37:53.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-26T03:37:53.932+0000 7f8eec214640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-14/keyring: can't open /var/lib/ceph/osd/ceph-14/keyring: (2) No such file or directory 2024-04-26T03:37:53.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-26T03:37:53.932+0000 7f8eec214640 -1 created new key in keyring /var/lib/ceph/osd/ceph-14/keyring 2024-04-26T03:37:53.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-26T03:37:53.932+0000 7f8eec214640 -1 bluestore(/var/lib/ceph/osd/ceph-14/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-14/block: (2) No such file or directory 2024-04-26T03:37:53.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-26T03:37:53.932+0000 7f8eec214640 -1 bluestore(/var/lib/ceph/osd/ceph-14/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-14/block: (2) No such file or directory 2024-04-26T03:37:53.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-26T03:37:53.932+0000 7f8eec214640 -1 bluestore(/var/lib/ceph/osd/ceph-14/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-14/block: (2) No such file or directory 2024-04-26T03:37:53.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-26T03:37:53.932+0000 7f8eec214640 -1 bluestore(/var/lib/ceph/osd/ceph-14) _read_fsid unparsable uuid 2024-04-26T03:37:56.783 DEBUG:teuthology.orchestra.run.smithi066:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-14 2024-04-26T03:37:56.812 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-26T03:37:56.839 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2024-04-26T03:37:56.839 INFO:tasks.ceph:role: osd.3 2024-04-26T03:37:56.839 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi092.front.sepia.ceph.com 2024-04-26T03:37:56.839 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-26T03:37:56.913 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T03:37:56.913 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T03:37:56.913 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T03:37:56.913 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T03:37:56.913 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T03:37:56.913 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2024-04-26T03:37:56.913 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T03:37:56.914 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T03:37:56.914 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T03:37:56.914 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T03:37:57.412 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2024-04-26T03:37:57.415 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2024-04-26T03:37:57.415 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-3 2024-04-26T03:37:57.552 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-26T03:37:57.583 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-26T03:37:57.648 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2024-04-26T03:37:57.648 INFO:tasks.ceph:role: osd.7 2024-04-26T03:37:57.648 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi092.front.sepia.ceph.com 2024-04-26T03:37:57.649 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-26T03:37:57.719 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T03:37:57.719 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T03:37:57.719 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T03:37:57.719 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T03:37:57.719 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T03:37:57.719 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2024-04-26T03:37:57.719 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T03:37:57.719 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T03:37:57.720 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T03:37:57.720 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T03:37:58.182 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2024-04-26T03:37:58.185 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2024-04-26T03:37:58.185 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-7 2024-04-26T03:37:58.280 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-26T03:37:58.307 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-11 2024-04-26T03:37:58.370 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2024-04-26T03:37:58.371 INFO:tasks.ceph:role: osd.11 2024-04-26T03:37:58.371 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi092.front.sepia.ceph.com 2024-04-26T03:37:58.371 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-26T03:37:58.440 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T03:37:58.440 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T03:37:58.441 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T03:37:58.441 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T03:37:58.441 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T03:37:58.441 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2024-04-26T03:37:58.441 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T03:37:58.441 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T03:37:58.441 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T03:37:58.441 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T03:37:58.924 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2024-04-26T03:37:58.926 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2024-04-26T03:37:58.926 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-11 2024-04-26T03:37:59.073 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-11 2024-04-26T03:37:59.103 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-15 2024-04-26T03:37:59.167 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2024-04-26T03:37:59.167 INFO:tasks.ceph:role: osd.15 2024-04-26T03:37:59.168 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi092.front.sepia.ceph.com 2024-04-26T03:37:59.168 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-26T03:37:59.238 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T03:37:59.238 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T03:37:59.238 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T03:37:59.238 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T03:37:59.238 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T03:37:59.238 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2024-04-26T03:37:59.239 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T03:37:59.239 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T03:37:59.239 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T03:37:59.239 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T03:37:59.759 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2024-04-26T03:37:59.761 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2024-04-26T03:37:59.761 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-15 2024-04-26T03:37:59.908 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-15 2024-04-26T03:37:59.937 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T03:38:00.037 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-26T03:38:00.035+0000 7fdec8c10640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-26T03:38:00.038 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-26T03:38:00.035+0000 7fdec8c10640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-26T03:38:00.038 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-26T03:38:00.035+0000 7fdec8c10640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T03:38:00.038 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-26T03:38:00.036+0000 7fdec8c10640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T03:38:00.038 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-26T03:38:00.036+0000 7fdec8c10640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T03:38:00.038 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-26T03:38:00.036+0000 7fdec8c10640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-26T03:38:02.896 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-26T03:38:02.923 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T03:38:03.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-26T03:38:03.009+0000 7fb124296640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-26T03:38:03.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-26T03:38:03.009+0000 7fb124296640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-26T03:38:03.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-26T03:38:03.009+0000 7fb124296640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T03:38:03.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-26T03:38:03.009+0000 7fb124296640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T03:38:03.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-26T03:38:03.009+0000 7fb124296640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T03:38:03.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-26T03:38:03.010+0000 7fb124296640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-26T03:38:05.861 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-26T03:38:05.890 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 11 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T03:38:05.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-26T03:38:05.979+0000 7f1295c0d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-11/keyring: can't open /var/lib/ceph/osd/ceph-11/keyring: (2) No such file or directory 2024-04-26T03:38:05.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-26T03:38:05.979+0000 7f1295c0d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-11/keyring 2024-04-26T03:38:05.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-26T03:38:05.979+0000 7f1295c0d640 -1 bluestore(/var/lib/ceph/osd/ceph-11/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-11/block: (2) No such file or directory 2024-04-26T03:38:05.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-26T03:38:05.979+0000 7f1295c0d640 -1 bluestore(/var/lib/ceph/osd/ceph-11/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-11/block: (2) No such file or directory 2024-04-26T03:38:05.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-26T03:38:05.979+0000 7f1295c0d640 -1 bluestore(/var/lib/ceph/osd/ceph-11/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-11/block: (2) No such file or directory 2024-04-26T03:38:05.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-26T03:38:05.980+0000 7f1295c0d640 -1 bluestore(/var/lib/ceph/osd/ceph-11) _read_fsid unparsable uuid 2024-04-26T03:38:08.835 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-11 2024-04-26T03:38:08.860 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 15 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T03:38:08.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-26T03:38:08.941+0000 7f3501c07640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-15/keyring: can't open /var/lib/ceph/osd/ceph-15/keyring: (2) No such file or directory 2024-04-26T03:38:08.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-26T03:38:08.942+0000 7f3501c07640 -1 created new key in keyring /var/lib/ceph/osd/ceph-15/keyring 2024-04-26T03:38:08.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-26T03:38:08.942+0000 7f3501c07640 -1 bluestore(/var/lib/ceph/osd/ceph-15/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-15/block: (2) No such file or directory 2024-04-26T03:38:08.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-26T03:38:08.942+0000 7f3501c07640 -1 bluestore(/var/lib/ceph/osd/ceph-15/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-15/block: (2) No such file or directory 2024-04-26T03:38:08.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-26T03:38:08.942+0000 7f3501c07640 -1 bluestore(/var/lib/ceph/osd/ceph-15/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-15/block: (2) No such file or directory 2024-04-26T03:38:08.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-04-26T03:38:08.942+0000 7f3501c07640 -1 bluestore(/var/lib/ceph/osd/ceph-15) _read_fsid unparsable uuid 2024-04-26T03:38:11.791 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-15 2024-04-26T03:38:11.859 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-26T03:38:11.860 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T03:38:11.860 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-26T03:38:11.889 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T03:38:11.889 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-26T03:38:11.958 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T03:38:11.958 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-26T03:38:12.025 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T03:38:12.025 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/osd/ceph-8/keyring of=/dev/stdout 2024-04-26T03:38:12.091 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T03:38:12.091 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/osd/ceph-12/keyring of=/dev/stdout 2024-04-26T03:38:12.158 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-26T03:38:12.159 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-26T03:38:12.189 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-26T03:38:12.189 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-26T03:38:12.258 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-26T03:38:12.258 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-9/keyring of=/dev/stdout 2024-04-26T03:38:12.327 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-26T03:38:12.327 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-13/keyring of=/dev/stdout 2024-04-26T03:38:12.395 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-26T03:38:12.395 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-26T03:38:12.425 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-26T03:38:12.425 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-26T03:38:12.493 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-26T03:38:12.493 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-10/keyring of=/dev/stdout 2024-04-26T03:38:12.562 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-26T03:38:12.562 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-14/keyring of=/dev/stdout 2024-04-26T03:38:12.629 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-26T03:38:12.629 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-26T03:38:12.656 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-26T03:38:12.657 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-26T03:38:12.722 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-26T03:38:12.723 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-26T03:38:12.787 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-26T03:38:12.788 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-11/keyring of=/dev/stdout 2024-04-26T03:38:12.852 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-26T03:38:12.852 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-15/keyring of=/dev/stdout 2024-04-26T03:38:12.916 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-26T03:38:12.916 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-26T03:38:12.971 INFO:tasks.ceph:Adding keys to all mons... 2024-04-26T03:38:12.971 DEBUG:teuthology.orchestra.run.smithi028:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-26T03:38:12.975 DEBUG:teuthology.orchestra.run.smithi042:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-26T03:38:12.978 DEBUG:teuthology.orchestra.run.smithi066:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-26T03:38:12.998 INFO:teuthology.orchestra.run.smithi042.stdout:[mgr.y] 2024-04-26T03:38:12.998 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBrIStmLXxCHhAArFO839Cs5wUtoJ6xSzA6FA== 2024-04-26T03:38:12.998 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.0] 2024-04-26T03:38:12.999 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBsIStmWVXtNRAAb9Wvhvih20sjs8TcsPI96w== 2024-04-26T03:38:12.999 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.4] 2024-04-26T03:38:12.999 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBvIStmr2JUNBAAlhY42BZk9MVs6KQZmLuCCA== 2024-04-26T03:38:12.999 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.8] 2024-04-26T03:38:12.999 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQByIStmuqN5MhAANXuV/WiDo1/qWIatFq70OQ== 2024-04-26T03:38:12.999 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.12] 2024-04-26T03:38:13.000 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQB1IStm7k97MBAA7CSSSNxE3YEd7fl6C1TFTg== 2024-04-26T03:38:13.000 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.1] 2024-04-26T03:38:13.000 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQB5IStmHplgORAANvbHk6bhaMYH+glgJH/vYw== 2024-04-26T03:38:13.000 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.5] 2024-04-26T03:38:13.000 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQB8IStm3Y8hNxAAJ1SyA92KDDp7pzDcQe7pfg== 2024-04-26T03:38:13.000 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.9] 2024-04-26T03:38:13.001 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQB/IStm0Eg1NRAAZgAluLrF2hGJZg+blLarXg== 2024-04-26T03:38:13.001 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.13] 2024-04-26T03:38:13.001 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQCCIStmERdKMxAACmt7V9O1t3YwczYUiN9sFw== 2024-04-26T03:38:13.001 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.2] 2024-04-26T03:38:13.001 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQCJIStmmr8xABAAQqZR7kUjqoafYDteMycKWQ== 2024-04-26T03:38:13.002 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.6] 2024-04-26T03:38:13.002 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQCLIStmy0+IOhAA9ScRsUHgUPPKnMnGAwyANQ== 2024-04-26T03:38:13.002 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.10] 2024-04-26T03:38:13.002 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQCOIStm1KohORAAmHgHHs4dX+MqDJ8WpyESMw== 2024-04-26T03:38:13.002 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.14] 2024-04-26T03:38:13.002 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQCRIStmeDmjNxAAv9FslUOmHsJ32oD4B87Qbw== 2024-04-26T03:38:13.002 INFO:teuthology.orchestra.run.smithi042.stdout:[mgr.x] 2024-04-26T03:38:13.003 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBrIStmO8LOIhAAFQcQQbnQ9qWis3M6vICu8Q== 2024-04-26T03:38:13.003 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.3] 2024-04-26T03:38:13.003 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQCYIStmXbkuAhAA8hIQWm+2l1dnQ24ChKHfiw== 2024-04-26T03:38:13.003 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.7] 2024-04-26T03:38:13.003 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQCbIStmw4yeABAAvFfxUtoajjO3JBiCk79KvQ== 2024-04-26T03:38:13.003 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.11] 2024-04-26T03:38:13.003 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQCdIStm9FdqOhAAgXDBJphw0wUm+1sV0AO1Uw== 2024-04-26T03:38:13.003 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.15] 2024-04-26T03:38:13.003 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQCgIStmPDAzOBAAdiI7WzYosSdIjT/wWp//iQ== 2024-04-26T03:38:13.004 INFO:teuthology.orchestra.run.smithi042.stdout:[client.0] 2024-04-26T03:38:13.004 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBrIStmZrhaJRAAiBIMHIDFQB4yYqSoXmWPDA== 2024-04-26T03:38:13.005 INFO:teuthology.orchestra.run.smithi028.stdout:[mgr.y] 2024-04-26T03:38:13.005 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQBrIStmLXxCHhAArFO839Cs5wUtoJ6xSzA6FA== 2024-04-26T03:38:13.006 INFO:teuthology.orchestra.run.smithi028.stdout:[osd.0] 2024-04-26T03:38:13.006 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQBsIStmWVXtNRAAb9Wvhvih20sjs8TcsPI96w== 2024-04-26T03:38:13.006 INFO:teuthology.orchestra.run.smithi028.stdout:[osd.4] 2024-04-26T03:38:13.006 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQBvIStmr2JUNBAAlhY42BZk9MVs6KQZmLuCCA== 2024-04-26T03:38:13.006 INFO:teuthology.orchestra.run.smithi028.stdout:[osd.8] 2024-04-26T03:38:13.006 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQByIStmuqN5MhAANXuV/WiDo1/qWIatFq70OQ== 2024-04-26T03:38:13.006 INFO:teuthology.orchestra.run.smithi028.stdout:[osd.12] 2024-04-26T03:38:13.006 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQB1IStm7k97MBAA7CSSSNxE3YEd7fl6C1TFTg== 2024-04-26T03:38:13.006 INFO:teuthology.orchestra.run.smithi028.stdout:[osd.1] 2024-04-26T03:38:13.006 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQB5IStmHplgORAANvbHk6bhaMYH+glgJH/vYw== 2024-04-26T03:38:13.006 INFO:teuthology.orchestra.run.smithi028.stdout:[osd.5] 2024-04-26T03:38:13.007 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQB8IStm3Y8hNxAAJ1SyA92KDDp7pzDcQe7pfg== 2024-04-26T03:38:13.007 INFO:teuthology.orchestra.run.smithi028.stdout:[osd.9] 2024-04-26T03:38:13.007 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQB/IStm0Eg1NRAAZgAluLrF2hGJZg+blLarXg== 2024-04-26T03:38:13.007 INFO:teuthology.orchestra.run.smithi028.stdout:[osd.13] 2024-04-26T03:38:13.007 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQCCIStmERdKMxAACmt7V9O1t3YwczYUiN9sFw== 2024-04-26T03:38:13.007 INFO:teuthology.orchestra.run.smithi028.stdout:[osd.2] 2024-04-26T03:38:13.007 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQCJIStmmr8xABAAQqZR7kUjqoafYDteMycKWQ== 2024-04-26T03:38:13.007 INFO:teuthology.orchestra.run.smithi028.stdout:[osd.6] 2024-04-26T03:38:13.007 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQCLIStmy0+IOhAA9ScRsUHgUPPKnMnGAwyANQ== 2024-04-26T03:38:13.007 INFO:teuthology.orchestra.run.smithi028.stdout:[osd.10] 2024-04-26T03:38:13.007 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQCOIStm1KohORAAmHgHHs4dX+MqDJ8WpyESMw== 2024-04-26T03:38:13.007 INFO:teuthology.orchestra.run.smithi028.stdout:[osd.14] 2024-04-26T03:38:13.007 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQCRIStmeDmjNxAAv9FslUOmHsJ32oD4B87Qbw== 2024-04-26T03:38:13.007 INFO:teuthology.orchestra.run.smithi028.stdout:[mgr.x] 2024-04-26T03:38:13.007 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQBrIStmO8LOIhAAFQcQQbnQ9qWis3M6vICu8Q== 2024-04-26T03:38:13.008 INFO:teuthology.orchestra.run.smithi028.stdout:[osd.3] 2024-04-26T03:38:13.008 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQCYIStmXbkuAhAA8hIQWm+2l1dnQ24ChKHfiw== 2024-04-26T03:38:13.008 INFO:teuthology.orchestra.run.smithi028.stdout:[osd.7] 2024-04-26T03:38:13.008 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQCbIStmw4yeABAAvFfxUtoajjO3JBiCk79KvQ== 2024-04-26T03:38:13.008 INFO:teuthology.orchestra.run.smithi028.stdout:[osd.11] 2024-04-26T03:38:13.008 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQCdIStm9FdqOhAAgXDBJphw0wUm+1sV0AO1Uw== 2024-04-26T03:38:13.008 INFO:teuthology.orchestra.run.smithi028.stdout:[osd.15] 2024-04-26T03:38:13.008 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQCgIStmPDAzOBAAdiI7WzYosSdIjT/wWp//iQ== 2024-04-26T03:38:13.008 INFO:teuthology.orchestra.run.smithi028.stdout:[client.0] 2024-04-26T03:38:13.008 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQBrIStmZrhaJRAAiBIMHIDFQB4yYqSoXmWPDA== 2024-04-26T03:38:13.009 INFO:teuthology.orchestra.run.smithi066.stdout:[mgr.y] 2024-04-26T03:38:13.010 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBrIStmLXxCHhAArFO839Cs5wUtoJ6xSzA6FA== 2024-04-26T03:38:13.010 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.0] 2024-04-26T03:38:13.010 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBsIStmWVXtNRAAb9Wvhvih20sjs8TcsPI96w== 2024-04-26T03:38:13.010 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.4] 2024-04-26T03:38:13.010 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBvIStmr2JUNBAAlhY42BZk9MVs6KQZmLuCCA== 2024-04-26T03:38:13.010 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.8] 2024-04-26T03:38:13.011 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQByIStmuqN5MhAANXuV/WiDo1/qWIatFq70OQ== 2024-04-26T03:38:13.011 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.12] 2024-04-26T03:38:13.011 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQB1IStm7k97MBAA7CSSSNxE3YEd7fl6C1TFTg== 2024-04-26T03:38:13.011 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.1] 2024-04-26T03:38:13.011 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQB5IStmHplgORAANvbHk6bhaMYH+glgJH/vYw== 2024-04-26T03:38:13.011 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.5] 2024-04-26T03:38:13.011 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQB8IStm3Y8hNxAAJ1SyA92KDDp7pzDcQe7pfg== 2024-04-26T03:38:13.012 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.9] 2024-04-26T03:38:13.012 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQB/IStm0Eg1NRAAZgAluLrF2hGJZg+blLarXg== 2024-04-26T03:38:13.012 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.13] 2024-04-26T03:38:13.012 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQCCIStmERdKMxAACmt7V9O1t3YwczYUiN9sFw== 2024-04-26T03:38:13.012 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.2] 2024-04-26T03:38:13.012 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQCJIStmmr8xABAAQqZR7kUjqoafYDteMycKWQ== 2024-04-26T03:38:13.012 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.6] 2024-04-26T03:38:13.012 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQCLIStmy0+IOhAA9ScRsUHgUPPKnMnGAwyANQ== 2024-04-26T03:38:13.012 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.10] 2024-04-26T03:38:13.012 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQCOIStm1KohORAAmHgHHs4dX+MqDJ8WpyESMw== 2024-04-26T03:38:13.013 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.14] 2024-04-26T03:38:13.013 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQCRIStmeDmjNxAAv9FslUOmHsJ32oD4B87Qbw== 2024-04-26T03:38:13.013 INFO:teuthology.orchestra.run.smithi066.stdout:[mgr.x] 2024-04-26T03:38:13.013 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBrIStmO8LOIhAAFQcQQbnQ9qWis3M6vICu8Q== 2024-04-26T03:38:13.013 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.3] 2024-04-26T03:38:13.013 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQCYIStmXbkuAhAA8hIQWm+2l1dnQ24ChKHfiw== 2024-04-26T03:38:13.013 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.7] 2024-04-26T03:38:13.013 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQCbIStmw4yeABAAvFfxUtoajjO3JBiCk79KvQ== 2024-04-26T03:38:13.013 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.11] 2024-04-26T03:38:13.013 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQCdIStm9FdqOhAAgXDBJphw0wUm+1sV0AO1Uw== 2024-04-26T03:38:13.013 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.15] 2024-04-26T03:38:13.013 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQCgIStmPDAzOBAAdiI7WzYosSdIjT/wWp//iQ== 2024-04-26T03:38:13.013 INFO:teuthology.orchestra.run.smithi066.stdout:[client.0] 2024-04-26T03:38:13.013 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBrIStmZrhaJRAAiBIMHIDFQB4yYqSoXmWPDA== 2024-04-26T03:38:13.014 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T03:38:13.049 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T03:38:13.052 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T03:38:13.109 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.112 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.114 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.167 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.169 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.172 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.223 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.227 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.229 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.279 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.12 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.281 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.12 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.284 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.12 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.333 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.335 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.337 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.387 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.389 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.391 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.441 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.443 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.445 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.494 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.13 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.496 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.13 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.499 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.13 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.549 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.551 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.554 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.603 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.605 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.608 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.657 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.660 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.662 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.711 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.14 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.714 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.14 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.716 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.14 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.765 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T03:38:13.768 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T03:38:13.770 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T03:38:13.820 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.823 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.825 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.874 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.877 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.879 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.928 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.931 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.934 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.984 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.15 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.987 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.15 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:13.990 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.15 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T03:38:14.040 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T03:38:14.043 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T03:38:14.045 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T03:38:14.097 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-26T03:38:14.098 DEBUG:teuthology.orchestra.run.smithi028:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-26T03:38:14.122 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T03:38:14.216 DEBUG:teuthology.orchestra.run.smithi028:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-26T03:38:14.241 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-26T03:38:14.266 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T03:38:14.362 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-26T03:38:14.386 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-26T03:38:14.412 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T03:38:14.509 DEBUG:teuthology.orchestra.run.smithi066:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-26T03:38:14.533 DEBUG:teuthology.orchestra.run.smithi028:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-26T03:38:14.535 DEBUG:teuthology.orchestra.run.smithi042:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-26T03:38:14.537 DEBUG:teuthology.orchestra.run.smithi066:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-26T03:38:14.588 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-26T03:38:14.588 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-26T03:38:14.588 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-26T03:38:14.591 INFO:tasks.ceph.mon.a:Started 2024-04-26T03:38:14.591 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-26T03:38:14.591 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-26T03:38:14.594 INFO:tasks.ceph.mon.b:Started 2024-04-26T03:38:14.594 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-26T03:38:14.594 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-26T03:38:14.630 INFO:tasks.ceph.mon.c:Started 2024-04-26T03:38:14.630 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-26T03:38:14.631 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-26T03:38:14.631 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-26T03:38:14.633 INFO:tasks.ceph.mgr.y:Started 2024-04-26T03:38:14.633 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-26T03:38:14.634 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-26T03:38:14.637 INFO:tasks.ceph.mgr.x:Started 2024-04-26T03:38:14.638 INFO:tasks.ceph:Setting crush tunables to default 2024-04-26T03:38:14.638 DEBUG:teuthology.orchestra.run.smithi028:> sudo ceph --cluster ceph osd crush tunables default 2024-04-26T03:38:19.997 INFO:teuthology.orchestra.run.smithi028.stderr:adjusted tunables profile to default 2024-04-26T03:38:20.009 INFO:tasks.ceph:check_enable_crimson: False 2024-04-26T03:38:20.009 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-26T03:38:20.009 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T03:38:20.009 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-26T03:38:20.034 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T03:38:20.035 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-26T03:38:20.037 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-04-26T03:38:20.035+0000 7f4d832b4040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T03:38:20.100 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T03:38:20.100 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/osd/ceph-8/fsid of=/dev/stdout 2024-04-26T03:38:20.176 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T03:38:20.177 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/osd/ceph-12/fsid of=/dev/stdout 2024-04-26T03:38:20.189 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-04-26T03:38:20.187+0000 7f4d832b4040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T03:38:20.231 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T03:38:20.228+0000 7f6b87c38040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T03:38:20.242 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-04-26T03:38:20.239+0000 7f4d832b4040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T03:38:20.253 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-26T03:38:20.254 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-26T03:38:20.286 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-26T03:38:20.287 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-26T03:38:20.357 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-26T03:38:20.357 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-9/fsid of=/dev/stdout 2024-04-26T03:38:20.393 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T03:38:20.391+0000 7f6b87c38040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T03:38:20.430 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-26T03:38:20.430 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-13/fsid of=/dev/stdout 2024-04-26T03:38:20.449 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T03:38:20.447+0000 7f6b87c38040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T03:38:20.486 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-04-26T03:38:20.484+0000 7f4d832b4040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T03:38:20.500 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-26T03:38:20.500 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-26T03:38:20.531 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-26T03:38:20.531 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-26T03:38:20.531 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-04-26T03:38:20.529+0000 7f4d832b4040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T03:38:20.599 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-26T03:38:20.600 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-10/fsid of=/dev/stdout 2024-04-26T03:38:20.668 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-26T03:38:20.668 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-14/fsid of=/dev/stdout 2024-04-26T03:38:20.698 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T03:38:20.696+0000 7f6b87c38040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T03:38:20.734 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-26T03:38:20.734 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-26T03:38:20.746 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T03:38:20.744+0000 7f6b87c38040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T03:38:20.758 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-26T03:38:20.758 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-26T03:38:20.823 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-26T03:38:20.824 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-11/fsid of=/dev/stdout 2024-04-26T03:38:20.887 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-04-26T03:38:20.888 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-15/fsid of=/dev/stdout 2024-04-26T03:38:20.950 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new 31b0e7a9-455d-4874-9c60-26efac8d0b1e 0 2024-04-26T03:38:21.168 INFO:teuthology.orchestra.run.smithi092.stdout:0 2024-04-26T03:38:21.180 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new 52819ace-a070-4822-b33e-3c54466e61ff 1 2024-04-26T03:38:21.246 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-04-26T03:38:21.244+0000 7f4d832b4040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T03:38:21.336 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-04-26T03:38:21.334+0000 7f4d832b4040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T03:38:21.355 INFO:teuthology.orchestra.run.smithi092.stdout:1 2024-04-26T03:38:21.366 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new 858f8ac1-2edc-43d5-914c-c1914bdc0c9a 2 2024-04-26T03:38:21.381 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-04-26T03:38:21.379+0000 7f4d832b4040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T03:38:21.465 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T03:38:21.463+0000 7f6b87c38040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T03:38:21.544 INFO:teuthology.orchestra.run.smithi092.stdout:2 2024-04-26T03:38:21.553 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-04-26T03:38:21.551+0000 7f4d832b4040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T03:38:21.555 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new 531a2e30-3351-4f84-804b-5fb5050b84ed 3 2024-04-26T03:38:21.559 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T03:38:21.556+0000 7f6b87c38040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T03:38:21.602 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T03:38:21.601+0000 7f6b87c38040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T03:38:21.609 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-04-26T03:38:21.608+0000 7f4d832b4040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T03:38:21.664 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-04-26T03:38:21.663+0000 7f4d832b4040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T03:38:21.773 INFO:teuthology.orchestra.run.smithi092.stdout:3 2024-04-26T03:38:21.777 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T03:38:21.775+0000 7f6b87c38040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T03:38:21.780 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-04-26T03:38:21.778+0000 7f4d832b4040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T03:38:21.783 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new 6d38ffda-40eb-4fd7-9300-2a09f2673e66 4 2024-04-26T03:38:21.836 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T03:38:21.834+0000 7f6b87c38040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T03:38:21.891 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T03:38:21.889+0000 7f6b87c38040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T03:38:21.999 INFO:teuthology.orchestra.run.smithi092.stdout:4 2024-04-26T03:38:22.010 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new 361aa40d-e53b-42af-afd4-81fbcac28725 5 2024-04-26T03:38:22.011 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T03:38:22.009+0000 7f6b87c38040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T03:38:22.174 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-04-26T03:38:22.172+0000 7f4d832b4040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T03:38:22.189 INFO:teuthology.orchestra.run.smithi092.stdout:5 2024-04-26T03:38:22.200 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new f610ad3e-1fd8-42bd-af4b-0d79c3bb9cf8 6 2024-04-26T03:38:22.376 INFO:teuthology.orchestra.run.smithi092.stdout:6 2024-04-26T03:38:22.387 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new a4bae6fb-e087-4563-a129-6938e18eed8c 7 2024-04-26T03:38:22.410 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T03:38:22.408+0000 7f6b87c38040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T03:38:22.444 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-04-26T03:38:22.443+0000 7f4d832b4040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T03:38:22.500 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-04-26T03:38:22.498+0000 7f4d832b4040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T03:38:22.565 INFO:teuthology.orchestra.run.smithi092.stdout:7 2024-04-26T03:38:22.576 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new 479f3885-452e-4231-bdc2-b79668384ba6 8 2024-04-26T03:38:22.620 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-04-26T03:38:22.619+0000 7f4d832b4040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T03:38:22.684 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T03:38:22.682+0000 7f6b87c38040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T03:38:22.739 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T03:38:22.737+0000 7f6b87c38040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T03:38:22.753 INFO:teuthology.orchestra.run.smithi092.stdout:8 2024-04-26T03:38:22.764 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new 9a603a47-7c5a-47d4-a298-56b3ac991892 9 2024-04-26T03:38:22.861 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T03:38:22.859+0000 7f6b87c38040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T03:38:22.905 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-04-26T03:38:22.903+0000 7f4d832b4040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T03:38:22.940 INFO:teuthology.orchestra.run.smithi092.stdout:9 2024-04-26T03:38:22.951 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new 1cd42945-6ee1-49f0-b17b-4093dd36b4ec 10 2024-04-26T03:38:22.955 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-04-26T03:38:22.953+0000 7f4d832b4040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T03:38:23.117 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-04-26T03:38:23.115+0000 7f4d832b4040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T03:38:23.129 INFO:teuthology.orchestra.run.smithi092.stdout:10 2024-04-26T03:38:23.140 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new b41e8100-1d4f-4b7a-991a-7a27bc29feec 11 2024-04-26T03:38:23.154 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T03:38:23.152+0000 7f6b87c38040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T03:38:23.204 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T03:38:23.202+0000 7f6b87c38040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T03:38:23.259 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-04-26T03:38:23.257+0000 7f4d832b4040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T03:38:23.310 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-04-26T03:38:23.309+0000 7f4d832b4040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T03:38:23.360 INFO:teuthology.orchestra.run.smithi092.stdout:11 2024-04-26T03:38:23.368 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T03:38:23.366+0000 7f6b87c38040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T03:38:23.372 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new 458cf150-32e1-449e-b533-ac6ab5226f85 12 2024-04-26T03:38:23.383 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-04-26T03:38:23.382+0000 7f4d832b4040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T03:38:23.431 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-04-26T03:38:23.430+0000 7f4d832b4040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T03:38:23.513 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T03:38:23.511+0000 7f6b87c38040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T03:38:23.548 INFO:teuthology.orchestra.run.smithi092.stdout:12 2024-04-26T03:38:23.559 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new f008462e-a735-4845-9ecf-dd7d4a6941af 13 2024-04-26T03:38:23.566 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T03:38:23.564+0000 7f6b87c38040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T03:38:23.639 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T03:38:23.637+0000 7f6b87c38040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T03:38:23.686 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T03:38:23.684+0000 7f6b87c38040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T03:38:23.776 INFO:teuthology.orchestra.run.smithi092.stdout:13 2024-04-26T03:38:23.787 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new 6f66ae97-eb8a-48eb-9bcc-20f65fbf690b 14 2024-04-26T03:38:23.922 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-04-26T03:38:23.921+0000 7f4d832b4040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T03:38:23.966 INFO:teuthology.orchestra.run.smithi092.stdout:14 2024-04-26T03:38:23.977 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new b856bfc8-253b-4112-be5c-f67d61d01768 15 2024-04-26T03:38:24.042 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-04-26T03:38:24.040+0000 7f4d832b4040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T03:38:24.152 INFO:teuthology.orchestra.run.smithi092.stdout:15 2024-04-26T03:38:24.163 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-26T03:38:24.163 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-26T03:38:24.166 INFO:tasks.ceph.osd.0:Started 2024-04-26T03:38:24.166 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-26T03:38:24.166 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-26T03:38:24.168 INFO:tasks.ceph.osd.4:Started 2024-04-26T03:38:24.168 INFO:tasks.ceph.osd.8:Restarting daemon 2024-04-26T03:38:24.168 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 8 2024-04-26T03:38:24.170 INFO:tasks.ceph.osd.8:Started 2024-04-26T03:38:24.170 INFO:tasks.ceph.osd.12:Restarting daemon 2024-04-26T03:38:24.170 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 12 2024-04-26T03:38:24.173 INFO:tasks.ceph.osd.12:Started 2024-04-26T03:38:24.173 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-26T03:38:24.173 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-26T03:38:24.175 INFO:tasks.ceph.osd.1:Started 2024-04-26T03:38:24.176 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-26T03:38:24.176 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-26T03:38:24.178 INFO:tasks.ceph.osd.5:Started 2024-04-26T03:38:24.178 INFO:tasks.ceph.osd.9:Restarting daemon 2024-04-26T03:38:24.178 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 9 2024-04-26T03:38:24.180 INFO:tasks.ceph.osd.9:Started 2024-04-26T03:38:24.181 INFO:tasks.ceph.osd.13:Restarting daemon 2024-04-26T03:38:24.181 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 13 2024-04-26T03:38:24.183 INFO:tasks.ceph.osd.13:Started 2024-04-26T03:38:24.183 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-26T03:38:24.183 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-26T03:38:24.186 INFO:tasks.ceph.osd.2:Started 2024-04-26T03:38:24.186 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-26T03:38:24.186 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-26T03:38:24.188 INFO:tasks.ceph.osd.6:Started 2024-04-26T03:38:24.188 INFO:tasks.ceph.osd.10:Restarting daemon 2024-04-26T03:38:24.188 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 10 2024-04-26T03:38:24.191 INFO:tasks.ceph.osd.10:Started 2024-04-26T03:38:24.191 INFO:tasks.ceph.osd.14:Restarting daemon 2024-04-26T03:38:24.191 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 14 2024-04-26T03:38:24.193 INFO:tasks.ceph.osd.14:Started 2024-04-26T03:38:24.193 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-26T03:38:24.193 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-26T03:38:24.196 INFO:tasks.ceph.osd.3:Started 2024-04-26T03:38:24.196 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-26T03:38:24.196 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-26T03:38:24.198 INFO:tasks.ceph.osd.7:Started 2024-04-26T03:38:24.198 INFO:tasks.ceph.osd.11:Restarting daemon 2024-04-26T03:38:24.198 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 11 2024-04-26T03:38:24.200 INFO:tasks.ceph.osd.11:Started 2024-04-26T03:38:24.200 INFO:tasks.ceph.osd.15:Restarting daemon 2024-04-26T03:38:24.200 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 15 2024-04-26T03:38:24.201 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T03:38:24.199+0000 7f6b87c38040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T03:38:24.202 INFO:tasks.ceph.osd.15:Started 2024-04-26T03:38:24.202 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T03:38:24.351 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T03:38:24.349+0000 7f6b87c38040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T03:38:24.397 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:38:24.398 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":18,"fsid":"c5a13c99-5c9a-4b8b-a893-4f6f4211677a","created":"2024-04-26T03:38:19.896648+0000","modified":"2024-04-26T03:38:24.150336+0000","last_up_change":"0.000000","last_in_change":"2024-04-26T03:38:24.150336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"31b0e7a9-455d-4874-9c60-26efac8d0b1e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"52819ace-a070-4822-b33e-3c54466e61ff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"858f8ac1-2edc-43d5-914c-c1914bdc0c9a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"531a2e30-3351-4f84-804b-5fb5050b84ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"6d38ffda-40eb-4fd7-9300-2a09f2673e66","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"361aa40d-e53b-42af-afd4-81fbcac28725","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f610ad3e-1fd8-42bd-af4b-0d79c3bb9cf8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a4bae6fb-e087-4563-a129-6938e18eed8c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"479f3885-452e-4231-bdc2-b79668384ba6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"9a603a47-7c5a-47d4-a298-56b3ac991892","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"1cd42945-6ee1-49f0-b17b-4093dd36b4ec","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"b41e8100-1d4f-4b7a-991a-7a27bc29feec","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"458cf150-32e1-449e-b533-ac6ab5226f85","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"f008462e-a735-4845-9ecf-dd7d4a6941af","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"6f66ae97-eb8a-48eb-9bcc-20f65fbf690b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"b856bfc8-253b-4112-be5c-f67d61d01768","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T03:38:24.408 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-26T03:38:24.409 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-26T03:38:24.764 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:38:24.761+0000 7fba10344640 -1 Falling back to public interface 2024-04-26T03:38:24.783 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-04-26T03:38:24.782+0000 7fe925a15640 -1 Falling back to public interface 2024-04-26T03:38:24.784 INFO:tasks.ceph.osd.4.smithi028.stderr:2024-04-26T03:38:24.782+0000 7fd954207640 -1 Falling back to public interface 2024-04-26T03:38:24.790 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:38:24.788+0000 7fb78f599640 -1 Falling back to public interface 2024-04-26T03:38:24.790 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:38:24.788+0000 7f60d6ca4640 -1 Falling back to public interface 2024-04-26T03:38:24.794 INFO:tasks.ceph.osd.9.smithi042.stderr:2024-04-26T03:38:24.792+0000 7fdac9407640 -1 Falling back to public interface 2024-04-26T03:38:24.794 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:38:24.792+0000 7f0a18e0d640 -1 Falling back to public interface 2024-04-26T03:38:24.795 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:38:24.793+0000 7f0a1783b640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6816/0: (98) Address already in use 2024-04-26T03:38:24.795 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-04-26T03:38:24.793+0000 7fee59a15640 -1 Falling back to public interface 2024-04-26T03:38:24.795 INFO:tasks.ceph.osd.8.smithi028.stderr:2024-04-26T03:38:24.792+0000 7f5bbb02c640 -1 Falling back to public interface 2024-04-26T03:38:24.798 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-04-26T03:38:24.796+0000 7fdc3081b640 -1 Falling back to public interface 2024-04-26T03:38:24.799 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-26T03:38:24.797+0000 7f610e3fc640 -1 Falling back to public interface 2024-04-26T03:38:24.800 INFO:tasks.ceph.osd.10.smithi066.stderr:2024-04-26T03:38:24.798+0000 7f1312d1c640 -1 Falling back to public interface 2024-04-26T03:38:24.802 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-04-26T03:38:24.800+0000 7f745480f640 -1 Falling back to public interface 2024-04-26T03:38:24.806 INFO:tasks.ceph.osd.12.smithi028.stderr:2024-04-26T03:38:24.805+0000 7f1153413640 -1 Falling back to public interface 2024-04-26T03:38:24.807 INFO:tasks.ceph.osd.14.smithi066.stderr:2024-04-26T03:38:24.804+0000 7f5399620640 -1 Falling back to public interface 2024-04-26T03:38:24.810 INFO:tasks.ceph.osd.15.smithi092.stderr:2024-04-26T03:38:24.808+0000 7f6d15e41640 -1 Falling back to public interface 2024-04-26T03:38:25.163 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-04-26T03:38:25.161+0000 7f4d246d4640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-26T03:38:25.163 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-04-26T03:38:25.161+0000 7f4d246d4640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-26T03:38:26.022 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-26T03:38:26.326 INFO:teuthology.misc.health.smithi028.stdout: 2024-04-26T03:38:26.327 INFO:teuthology.misc.health.smithi028.stdout:{"epoch":18,"fsid":"c5a13c99-5c9a-4b8b-a893-4f6f4211677a","created":"2024-04-26T03:38:19.896648+0000","modified":"2024-04-26T03:38:24.150336+0000","last_up_change":"0.000000","last_in_change":"2024-04-26T03:38:24.150336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"31b0e7a9-455d-4874-9c60-26efac8d0b1e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"52819ace-a070-4822-b33e-3c54466e61ff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"858f8ac1-2edc-43d5-914c-c1914bdc0c9a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"531a2e30-3351-4f84-804b-5fb5050b84ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"6d38ffda-40eb-4fd7-9300-2a09f2673e66","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"361aa40d-e53b-42af-afd4-81fbcac28725","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f610ad3e-1fd8-42bd-af4b-0d79c3bb9cf8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a4bae6fb-e087-4563-a129-6938e18eed8c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"479f3885-452e-4231-bdc2-b79668384ba6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"9a603a47-7c5a-47d4-a298-56b3ac991892","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"1cd42945-6ee1-49f0-b17b-4093dd36b4ec","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"b41e8100-1d4f-4b7a-991a-7a27bc29feec","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"458cf150-32e1-449e-b533-ac6ab5226f85","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"f008462e-a735-4845-9ecf-dd7d4a6941af","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"6f66ae97-eb8a-48eb-9bcc-20f65fbf690b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"b856bfc8-253b-4112-be5c-f67d61d01768","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T03:38:26.338 DEBUG:teuthology.misc:0 of 16 OSDs are up 2024-04-26T03:38:28.218 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:38:28.216+0000 7fba10344640 -1 osd.0 0 log_to_monitors true 2024-04-26T03:38:28.231 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:38:28.229+0000 7f60d6ca4640 -1 osd.2 0 log_to_monitors true 2024-04-26T03:38:28.240 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-04-26T03:38:28.239+0000 7fe925a15640 -1 osd.1 0 log_to_monitors true 2024-04-26T03:38:28.259 INFO:tasks.ceph.osd.4.smithi028.stderr:2024-04-26T03:38:28.257+0000 7fd954207640 -1 osd.4 0 log_to_monitors true 2024-04-26T03:38:28.260 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-04-26T03:38:28.258+0000 7f745480f640 -1 osd.11 0 log_to_monitors true 2024-04-26T03:38:28.280 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:38:28.278+0000 7f0a18e0d640 -1 osd.13 0 log_to_monitors true 2024-04-26T03:38:28.281 INFO:tasks.ceph.osd.10.smithi066.stderr:2024-04-26T03:38:28.279+0000 7f1312d1c640 -1 osd.10 0 log_to_monitors true 2024-04-26T03:38:28.294 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-04-26T03:38:28.292+0000 7fdc3081b640 -1 osd.7 0 log_to_monitors true 2024-04-26T03:38:28.307 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-04-26T03:38:28.304+0000 7fee59a15640 -1 osd.6 0 log_to_monitors true 2024-04-26T03:38:28.308 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:38:28.306+0000 7fb78f599640 -1 osd.5 0 log_to_monitors true 2024-04-26T03:38:28.308 INFO:tasks.ceph.osd.12.smithi028.stderr:2024-04-26T03:38:28.306+0000 7f1153413640 -1 osd.12 0 log_to_monitors true 2024-04-26T03:38:28.310 INFO:tasks.ceph.osd.14.smithi066.stderr:2024-04-26T03:38:28.308+0000 7f5399620640 -1 osd.14 0 log_to_monitors true 2024-04-26T03:38:28.311 INFO:tasks.ceph.osd.8.smithi028.stderr:2024-04-26T03:38:28.309+0000 7f5bbb02c640 -1 osd.8 0 log_to_monitors true 2024-04-26T03:38:28.329 INFO:tasks.ceph.osd.9.smithi042.stderr:2024-04-26T03:38:28.328+0000 7fdac9407640 -1 osd.9 0 log_to_monitors true 2024-04-26T03:38:28.343 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-26T03:38:28.341+0000 7f610e3fc640 -1 osd.3 0 log_to_monitors true 2024-04-26T03:38:28.357 INFO:tasks.ceph.osd.15.smithi092.stderr:2024-04-26T03:38:28.355+0000 7f6d15e41640 -1 osd.15 0 log_to_monitors true 2024-04-26T03:38:29.287 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-04-26T03:38:29.286+0000 7f7450178640 -1 osd.11 0 waiting for initial osdmap 2024-04-26T03:38:29.303 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-04-26T03:38:29.302+0000 7f744bfa0640 -1 osd.11 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T03:38:30.289 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-04-26T03:38:30.286+0000 7fee552e8640 -1 osd.6 0 waiting for initial osdmap 2024-04-26T03:38:30.289 INFO:tasks.ceph.osd.14.smithi066.stderr:2024-04-26T03:38:30.286+0000 7f5394ef3640 -1 osd.14 0 waiting for initial osdmap 2024-04-26T03:38:30.289 INFO:tasks.ceph.osd.12.smithi028.stderr:2024-04-26T03:38:30.287+0000 7f114edff640 -1 osd.12 0 waiting for initial osdmap 2024-04-26T03:38:30.290 INFO:tasks.ceph.osd.8.smithi028.stderr:2024-04-26T03:38:30.287+0000 7f5bb65c3640 -1 osd.8 0 waiting for initial osdmap 2024-04-26T03:38:30.290 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:38:30.288+0000 7f0a14822640 -1 osd.13 0 waiting for initial osdmap 2024-04-26T03:38:30.290 INFO:tasks.ceph.osd.10.smithi066.stderr:2024-04-26T03:38:30.289+0000 7f130ecb2640 -1 osd.10 0 waiting for initial osdmap 2024-04-26T03:38:30.295 INFO:tasks.ceph.osd.10.smithi066.stderr:2024-04-26T03:38:30.293+0000 7f130aada640 -1 osd.10 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T03:38:30.296 INFO:tasks.ceph.osd.9.smithi042.stderr:2024-04-26T03:38:30.295+0000 7fdac539d640 -1 osd.9 0 waiting for initial osdmap 2024-04-26T03:38:30.299 INFO:tasks.ceph.osd.15.smithi092.stderr:2024-04-26T03:38:30.298+0000 7f6d11dd7640 -1 osd.15 0 waiting for initial osdmap 2024-04-26T03:38:30.301 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-04-26T03:38:30.300+0000 7fdc2c207640 -1 osd.7 0 waiting for initial osdmap 2024-04-26T03:38:30.301 INFO:tasks.ceph.osd.9.smithi042.stderr:2024-04-26T03:38:30.300+0000 7fdac11c5640 -1 osd.9 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T03:38:30.303 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-04-26T03:38:30.301+0000 7fee51110640 -1 osd.6 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T03:38:30.303 INFO:tasks.ceph.osd.14.smithi066.stderr:2024-04-26T03:38:30.301+0000 7f5390d1b640 -1 osd.14 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T03:38:30.309 INFO:tasks.ceph.osd.8.smithi028.stderr:2024-04-26T03:38:30.307+0000 7f5bb23eb640 -1 osd.8 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T03:38:30.309 INFO:tasks.ceph.osd.12.smithi028.stderr:2024-04-26T03:38:30.307+0000 7f114ac27640 -1 osd.12 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T03:38:30.309 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:38:30.308+0000 7f0a1064a640 -1 osd.13 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T03:38:30.310 INFO:tasks.ceph.osd.15.smithi092.stderr:2024-04-26T03:38:30.308+0000 7f6d0dbff640 -1 osd.15 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T03:38:30.310 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-04-26T03:38:30.308+0000 7fdc2802f640 -1 osd.7 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T03:38:32.055 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:38:32.053+0000 7f60d2c3a640 -1 osd.2 0 waiting for initial osdmap 2024-04-26T03:38:32.066 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:38:32.064+0000 7f60caa5a640 -1 osd.2 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T03:38:32.124 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-04-26T03:38:32.122+0000 7fe9213b2640 -1 osd.1 0 waiting for initial osdmap 2024-04-26T03:38:32.134 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-04-26T03:38:32.133+0000 7fe9191d2640 -1 osd.1 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T03:38:32.195 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:38:32.193+0000 7fba0c2da640 -1 osd.0 0 waiting for initial osdmap 2024-04-26T03:38:32.208 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:38:32.206+0000 7fba040fa640 -1 osd.0 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T03:38:32.209 INFO:tasks.ceph.osd.4.smithi028.stderr:2024-04-26T03:38:32.207+0000 7fd95019d640 -1 osd.4 0 waiting for initial osdmap 2024-04-26T03:38:32.223 INFO:tasks.ceph.osd.4.smithi028.stderr:2024-04-26T03:38:32.222+0000 7fd947fbd640 -1 osd.4 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T03:38:32.907 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-26T03:38:32.905+0000 7f610aba6640 -1 osd.3 0 waiting for initial osdmap 2024-04-26T03:38:32.920 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-26T03:38:32.919+0000 7f61021b2640 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T03:38:33.174 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:38:33.172+0000 7fb78b52f640 -1 osd.5 0 waiting for initial osdmap 2024-04-26T03:38:33.192 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:38:33.191+0000 7fb78334f640 -1 osd.5 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T03:38:33.945 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-26T03:38:34.253 INFO:teuthology.misc.health.smithi028.stdout: 2024-04-26T03:38:34.253 INFO:teuthology.misc.health.smithi028.stdout:{"epoch":24,"fsid":"c5a13c99-5c9a-4b8b-a893-4f6f4211677a","created":"2024-04-26T03:38:19.896648+0000","modified":"2024-04-26T03:38:33.308561+0000","last_up_change":"2024-04-26T03:38:33.308561+0000","last_in_change":"2024-04-26T03:38:24.150336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T03:38:32.306696+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"31b0e7a9-455d-4874-9c60-26efac8d0b1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2377656090},{"type":"v1","addr":"172.21.15.28:6801","nonce":2377656090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2377656090},{"type":"v1","addr":"172.21.15.28:6803","nonce":2377656090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2377656090},{"type":"v1","addr":"172.21.15.28:6807","nonce":2377656090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2377656090},{"type":"v1","addr":"172.21.15.28:6805","nonce":2377656090}]},"public_addr":"172.21.15.28:6801/2377656090","cluster_addr":"172.21.15.28:6803/2377656090","heartbeat_back_addr":"172.21.15.28:6807/2377656090","heartbeat_front_addr":"172.21.15.28:6805/2377656090","state":["exists","up"]},{"osd":1,"uuid":"52819ace-a070-4822-b33e-3c54466e61ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":1678150206},{"type":"v1","addr":"172.21.15.42:6801","nonce":1678150206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1678150206},{"type":"v1","addr":"172.21.15.42:6803","nonce":1678150206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1678150206},{"type":"v1","addr":"172.21.15.42:6807","nonce":1678150206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1678150206},{"type":"v1","addr":"172.21.15.42:6805","nonce":1678150206}]},"public_addr":"172.21.15.42:6801/1678150206","cluster_addr":"172.21.15.42:6803/1678150206","heartbeat_back_addr":"172.21.15.42:6807/1678150206","heartbeat_front_addr":"172.21.15.42:6805/1678150206","state":["exists","up"]},{"osd":2,"uuid":"858f8ac1-2edc-43d5-914c-c1914bdc0c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":714535415},{"type":"v1","addr":"172.21.15.66:6801","nonce":714535415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":714535415},{"type":"v1","addr":"172.21.15.66:6803","nonce":714535415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":714535415},{"type":"v1","addr":"172.21.15.66:6807","nonce":714535415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":714535415},{"type":"v1","addr":"172.21.15.66:6805","nonce":714535415}]},"public_addr":"172.21.15.66:6801/714535415","cluster_addr":"172.21.15.66:6803/714535415","heartbeat_back_addr":"172.21.15.66:6807/714535415","heartbeat_front_addr":"172.21.15.66:6805/714535415","state":["exists","up"]},{"osd":3,"uuid":"531a2e30-3351-4f84-804b-5fb5050b84ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2377983625},{"type":"v1","addr":"172.21.15.92:6811","nonce":2377983625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2377983625},{"type":"v1","addr":"172.21.15.92:6813","nonce":2377983625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2377983625},{"type":"v1","addr":"172.21.15.92:6817","nonce":2377983625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2377983625},{"type":"v1","addr":"172.21.15.92:6815","nonce":2377983625}]},"public_addr":"172.21.15.92:6811/2377983625","cluster_addr":"172.21.15.92:6813/2377983625","heartbeat_back_addr":"172.21.15.92:6817/2377983625","heartbeat_front_addr":"172.21.15.92:6815/2377983625","state":["exists","up"]},{"osd":4,"uuid":"6d38ffda-40eb-4fd7-9300-2a09f2673e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":183752455},{"type":"v1","addr":"172.21.15.28:6809","nonce":183752455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":183752455},{"type":"v1","addr":"172.21.15.28:6811","nonce":183752455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":183752455},{"type":"v1","addr":"172.21.15.28:6815","nonce":183752455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":183752455},{"type":"v1","addr":"172.21.15.28:6813","nonce":183752455}]},"public_addr":"172.21.15.28:6809/183752455","cluster_addr":"172.21.15.28:6811/183752455","heartbeat_back_addr":"172.21.15.28:6815/183752455","heartbeat_front_addr":"172.21.15.28:6813/183752455","state":["exists","up"]},{"osd":5,"uuid":"361aa40d-e53b-42af-afd4-81fbcac28725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2012261939},{"type":"v1","addr":"172.21.15.42:6809","nonce":2012261939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2012261939},{"type":"v1","addr":"172.21.15.42:6811","nonce":2012261939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2012261939},{"type":"v1","addr":"172.21.15.42:6815","nonce":2012261939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2012261939},{"type":"v1","addr":"172.21.15.42:6813","nonce":2012261939}]},"public_addr":"172.21.15.42:6809/2012261939","cluster_addr":"172.21.15.42:6811/2012261939","heartbeat_back_addr":"172.21.15.42:6815/2012261939","heartbeat_front_addr":"172.21.15.42:6813/2012261939","state":["exists","up"]},{"osd":6,"uuid":"f610ad3e-1fd8-42bd-af4b-0d79c3bb9cf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":3533460882},{"type":"v1","addr":"172.21.15.66:6809","nonce":3533460882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":3533460882},{"type":"v1","addr":"172.21.15.66:6811","nonce":3533460882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":3533460882},{"type":"v1","addr":"172.21.15.66:6815","nonce":3533460882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":3533460882},{"type":"v1","addr":"172.21.15.66:6813","nonce":3533460882}]},"public_addr":"172.21.15.66:6809/3533460882","cluster_addr":"172.21.15.66:6811/3533460882","heartbeat_back_addr":"172.21.15.66:6815/3533460882","heartbeat_front_addr":"172.21.15.66:6813/3533460882","state":["exists","up"]},{"osd":7,"uuid":"a4bae6fb-e087-4563-a129-6938e18eed8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":1760163933},{"type":"v1","addr":"172.21.15.92:6803","nonce":1760163933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":1760163933},{"type":"v1","addr":"172.21.15.92:6805","nonce":1760163933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1760163933},{"type":"v1","addr":"172.21.15.92:6809","nonce":1760163933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":1760163933},{"type":"v1","addr":"172.21.15.92:6807","nonce":1760163933}]},"public_addr":"172.21.15.92:6803/1760163933","cluster_addr":"172.21.15.92:6805/1760163933","heartbeat_back_addr":"172.21.15.92:6809/1760163933","heartbeat_front_addr":"172.21.15.92:6807/1760163933","state":["exists","up"]},{"osd":8,"uuid":"479f3885-452e-4231-bdc2-b79668384ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":4127602407},{"type":"v1","addr":"172.21.15.28:6817","nonce":4127602407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":4127602407},{"type":"v1","addr":"172.21.15.28:6819","nonce":4127602407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":4127602407},{"type":"v1","addr":"172.21.15.28:6823","nonce":4127602407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":4127602407},{"type":"v1","addr":"172.21.15.28:6821","nonce":4127602407}]},"public_addr":"172.21.15.28:6817/4127602407","cluster_addr":"172.21.15.28:6819/4127602407","heartbeat_back_addr":"172.21.15.28:6823/4127602407","heartbeat_front_addr":"172.21.15.28:6821/4127602407","state":["exists","up"]},{"osd":9,"uuid":"9a603a47-7c5a-47d4-a298-56b3ac991892","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":4168729845},{"type":"v1","addr":"172.21.15.42:6818","nonce":4168729845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":4168729845},{"type":"v1","addr":"172.21.15.42:6822","nonce":4168729845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":4168729845},{"type":"v1","addr":"172.21.15.42:6830","nonce":4168729845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":4168729845},{"type":"v1","addr":"172.21.15.42:6826","nonce":4168729845}]},"public_addr":"172.21.15.42:6818/4168729845","cluster_addr":"172.21.15.42:6822/4168729845","heartbeat_back_addr":"172.21.15.42:6830/4168729845","heartbeat_front_addr":"172.21.15.42:6826/4168729845","state":["exists","up"]},{"osd":10,"uuid":"1cd42945-6ee1-49f0-b17b-4093dd36b4ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4180572718},{"type":"v1","addr":"172.21.15.66:6817","nonce":4180572718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":4180572718},{"type":"v1","addr":"172.21.15.66:6819","nonce":4180572718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":4180572718},{"type":"v1","addr":"172.21.15.66:6823","nonce":4180572718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":4180572718},{"type":"v1","addr":"172.21.15.66:6821","nonce":4180572718}]},"public_addr":"172.21.15.66:6817/4180572718","cluster_addr":"172.21.15.66:6819/4180572718","heartbeat_back_addr":"172.21.15.66:6823/4180572718","heartbeat_front_addr":"172.21.15.66:6821/4180572718","state":["exists","up"]},{"osd":11,"uuid":"b41e8100-1d4f-4b7a-991a-7a27bc29feec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":295713815},{"type":"v1","addr":"172.21.15.92:6819","nonce":295713815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":295713815},{"type":"v1","addr":"172.21.15.92:6821","nonce":295713815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":295713815},{"type":"v1","addr":"172.21.15.92:6825","nonce":295713815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":295713815},{"type":"v1","addr":"172.21.15.92:6823","nonce":295713815}]},"public_addr":"172.21.15.92:6819/295713815","cluster_addr":"172.21.15.92:6821/295713815","heartbeat_back_addr":"172.21.15.92:6825/295713815","heartbeat_front_addr":"172.21.15.92:6823/295713815","state":["exists","up"]},{"osd":12,"uuid":"458cf150-32e1-449e-b533-ac6ab5226f85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":2081141953},{"type":"v1","addr":"172.21.15.28:6825","nonce":2081141953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2081141953},{"type":"v1","addr":"172.21.15.28:6827","nonce":2081141953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2081141953},{"type":"v1","addr":"172.21.15.28:6831","nonce":2081141953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":2081141953},{"type":"v1","addr":"172.21.15.28:6829","nonce":2081141953}]},"public_addr":"172.21.15.28:6825/2081141953","cluster_addr":"172.21.15.28:6827/2081141953","heartbeat_back_addr":"172.21.15.28:6831/2081141953","heartbeat_front_addr":"172.21.15.28:6829/2081141953","state":["exists","up"]},{"osd":13,"uuid":"f008462e-a735-4845-9ecf-dd7d4a6941af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6817","nonce":301360893},{"type":"v1","addr":"172.21.15.42:6819","nonce":301360893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6821","nonce":301360893},{"type":"v1","addr":"172.21.15.42:6823","nonce":301360893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6829","nonce":301360893},{"type":"v1","addr":"172.21.15.42:6831","nonce":301360893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6825","nonce":301360893},{"type":"v1","addr":"172.21.15.42:6827","nonce":301360893}]},"public_addr":"172.21.15.42:6819/301360893","cluster_addr":"172.21.15.42:6823/301360893","heartbeat_back_addr":"172.21.15.42:6831/301360893","heartbeat_front_addr":"172.21.15.42:6827/301360893","state":["exists","up"]},{"osd":14,"uuid":"6f66ae97-eb8a-48eb-9bcc-20f65fbf690b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":1534905602},{"type":"v1","addr":"172.21.15.66:6825","nonce":1534905602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1534905602},{"type":"v1","addr":"172.21.15.66:6827","nonce":1534905602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1534905602},{"type":"v1","addr":"172.21.15.66:6831","nonce":1534905602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1534905602},{"type":"v1","addr":"172.21.15.66:6829","nonce":1534905602}]},"public_addr":"172.21.15.66:6825/1534905602","cluster_addr":"172.21.15.66:6827/1534905602","heartbeat_back_addr":"172.21.15.66:6831/1534905602","heartbeat_front_addr":"172.21.15.66:6829/1534905602","state":["exists","up"]},{"osd":15,"uuid":"b856bfc8-253b-4112-be5c-f67d61d01768","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3982634323},{"type":"v1","addr":"172.21.15.92:6827","nonce":3982634323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3982634323},{"type":"v1","addr":"172.21.15.92:6829","nonce":3982634323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3982634323},{"type":"v1","addr":"172.21.15.92:6833","nonce":3982634323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3982634323},{"type":"v1","addr":"172.21.15.92:6831","nonce":3982634323}]},"public_addr":"172.21.15.92:6827/3982634323","cluster_addr":"172.21.15.92:6829/3982634323","heartbeat_back_addr":"172.21.15.92:6833/3982634323","heartbeat_front_addr":"172.21.15.92:6831/3982634323","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.265053+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.243502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.252440+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.213076+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.315801+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.329538+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.293303+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.334962+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.233032+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.305138+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.351964+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.241086+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.352515+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.357462+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T03:38:34.265 DEBUG:teuthology.misc:16 of 16 OSDs are up 2024-04-26T03:38:34.265 INFO:tasks.ceph:Creating RBD pool 2024-04-26T03:38:34.265 DEBUG:teuthology.orchestra.run.smithi028:> sudo ceph --cluster ceph osd pool create rbd 8 2024-04-26T03:38:35.323 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'rbd' created 2024-04-26T03:38:35.338 DEBUG:teuthology.orchestra.run.smithi028:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-04-26T03:38:36.413 INFO:teuthology.orchestra.run.smithi028.stderr:enabled application 'rbd' on pool 'rbd' 2024-04-26T03:38:36.429 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-26T03:38:36.430 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-26T03:38:36.430 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-26T03:38:36.430 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-26T03:38:36.431 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-26T03:38:36.784 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:38:36.793 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":5,"flags":0,"active_gid":4107,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3453371003},{"type":"v1","addr":"172.21.15.92:6801","nonce":3453371003}]},"active_addr":"172.21.15.92:6801/3453371003","active_change":"2024-04-26T03:38:24.043037+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":3233178977}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":331201604}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":1570818811}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":617401486}]}]} 2024-04-26T03:38:36.797 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-26T03:38:36.797 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-26T03:38:36.798 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T03:38:37.095 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:38:37.095 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":27,"fsid":"c5a13c99-5c9a-4b8b-a893-4f6f4211677a","created":"2024-04-26T03:38:19.896648+0000","modified":"2024-04-26T03:38:36.323522+0000","last_up_change":"2024-04-26T03:38:33.308561+0000","last_in_change":"2024-04-26T03:38:24.150336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T03:38:32.306696+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-26T03:38:34.592633+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":3.9900000095367432,"score_stable":3.9900000095367432,"optimal_score":0.68999999761581421,"raw_score_acting":2.75,"raw_score_stable":2.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"31b0e7a9-455d-4874-9c60-26efac8d0b1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2377656090},{"type":"v1","addr":"172.21.15.28:6801","nonce":2377656090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2377656090},{"type":"v1","addr":"172.21.15.28:6803","nonce":2377656090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2377656090},{"type":"v1","addr":"172.21.15.28:6807","nonce":2377656090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2377656090},{"type":"v1","addr":"172.21.15.28:6805","nonce":2377656090}]},"public_addr":"172.21.15.28:6801/2377656090","cluster_addr":"172.21.15.28:6803/2377656090","heartbeat_back_addr":"172.21.15.28:6807/2377656090","heartbeat_front_addr":"172.21.15.28:6805/2377656090","state":["exists","up"]},{"osd":1,"uuid":"52819ace-a070-4822-b33e-3c54466e61ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":1678150206},{"type":"v1","addr":"172.21.15.42:6801","nonce":1678150206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1678150206},{"type":"v1","addr":"172.21.15.42:6803","nonce":1678150206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1678150206},{"type":"v1","addr":"172.21.15.42:6807","nonce":1678150206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1678150206},{"type":"v1","addr":"172.21.15.42:6805","nonce":1678150206}]},"public_addr":"172.21.15.42:6801/1678150206","cluster_addr":"172.21.15.42:6803/1678150206","heartbeat_back_addr":"172.21.15.42:6807/1678150206","heartbeat_front_addr":"172.21.15.42:6805/1678150206","state":["exists","up"]},{"osd":2,"uuid":"858f8ac1-2edc-43d5-914c-c1914bdc0c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":714535415},{"type":"v1","addr":"172.21.15.66:6801","nonce":714535415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":714535415},{"type":"v1","addr":"172.21.15.66:6803","nonce":714535415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":714535415},{"type":"v1","addr":"172.21.15.66:6807","nonce":714535415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":714535415},{"type":"v1","addr":"172.21.15.66:6805","nonce":714535415}]},"public_addr":"172.21.15.66:6801/714535415","cluster_addr":"172.21.15.66:6803/714535415","heartbeat_back_addr":"172.21.15.66:6807/714535415","heartbeat_front_addr":"172.21.15.66:6805/714535415","state":["exists","up"]},{"osd":3,"uuid":"531a2e30-3351-4f84-804b-5fb5050b84ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2377983625},{"type":"v1","addr":"172.21.15.92:6811","nonce":2377983625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2377983625},{"type":"v1","addr":"172.21.15.92:6813","nonce":2377983625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2377983625},{"type":"v1","addr":"172.21.15.92:6817","nonce":2377983625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2377983625},{"type":"v1","addr":"172.21.15.92:6815","nonce":2377983625}]},"public_addr":"172.21.15.92:6811/2377983625","cluster_addr":"172.21.15.92:6813/2377983625","heartbeat_back_addr":"172.21.15.92:6817/2377983625","heartbeat_front_addr":"172.21.15.92:6815/2377983625","state":["exists","up"]},{"osd":4,"uuid":"6d38ffda-40eb-4fd7-9300-2a09f2673e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":183752455},{"type":"v1","addr":"172.21.15.28:6809","nonce":183752455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":183752455},{"type":"v1","addr":"172.21.15.28:6811","nonce":183752455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":183752455},{"type":"v1","addr":"172.21.15.28:6815","nonce":183752455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":183752455},{"type":"v1","addr":"172.21.15.28:6813","nonce":183752455}]},"public_addr":"172.21.15.28:6809/183752455","cluster_addr":"172.21.15.28:6811/183752455","heartbeat_back_addr":"172.21.15.28:6815/183752455","heartbeat_front_addr":"172.21.15.28:6813/183752455","state":["exists","up"]},{"osd":5,"uuid":"361aa40d-e53b-42af-afd4-81fbcac28725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2012261939},{"type":"v1","addr":"172.21.15.42:6809","nonce":2012261939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2012261939},{"type":"v1","addr":"172.21.15.42:6811","nonce":2012261939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2012261939},{"type":"v1","addr":"172.21.15.42:6815","nonce":2012261939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2012261939},{"type":"v1","addr":"172.21.15.42:6813","nonce":2012261939}]},"public_addr":"172.21.15.42:6809/2012261939","cluster_addr":"172.21.15.42:6811/2012261939","heartbeat_back_addr":"172.21.15.42:6815/2012261939","heartbeat_front_addr":"172.21.15.42:6813/2012261939","state":["exists","up"]},{"osd":6,"uuid":"f610ad3e-1fd8-42bd-af4b-0d79c3bb9cf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":3533460882},{"type":"v1","addr":"172.21.15.66:6809","nonce":3533460882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":3533460882},{"type":"v1","addr":"172.21.15.66:6811","nonce":3533460882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":3533460882},{"type":"v1","addr":"172.21.15.66:6815","nonce":3533460882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":3533460882},{"type":"v1","addr":"172.21.15.66:6813","nonce":3533460882}]},"public_addr":"172.21.15.66:6809/3533460882","cluster_addr":"172.21.15.66:6811/3533460882","heartbeat_back_addr":"172.21.15.66:6815/3533460882","heartbeat_front_addr":"172.21.15.66:6813/3533460882","state":["exists","up"]},{"osd":7,"uuid":"a4bae6fb-e087-4563-a129-6938e18eed8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":1760163933},{"type":"v1","addr":"172.21.15.92:6803","nonce":1760163933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":1760163933},{"type":"v1","addr":"172.21.15.92:6805","nonce":1760163933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1760163933},{"type":"v1","addr":"172.21.15.92:6809","nonce":1760163933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":1760163933},{"type":"v1","addr":"172.21.15.92:6807","nonce":1760163933}]},"public_addr":"172.21.15.92:6803/1760163933","cluster_addr":"172.21.15.92:6805/1760163933","heartbeat_back_addr":"172.21.15.92:6809/1760163933","heartbeat_front_addr":"172.21.15.92:6807/1760163933","state":["exists","up"]},{"osd":8,"uuid":"479f3885-452e-4231-bdc2-b79668384ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":4127602407},{"type":"v1","addr":"172.21.15.28:6817","nonce":4127602407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":4127602407},{"type":"v1","addr":"172.21.15.28:6819","nonce":4127602407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":4127602407},{"type":"v1","addr":"172.21.15.28:6823","nonce":4127602407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":4127602407},{"type":"v1","addr":"172.21.15.28:6821","nonce":4127602407}]},"public_addr":"172.21.15.28:6817/4127602407","cluster_addr":"172.21.15.28:6819/4127602407","heartbeat_back_addr":"172.21.15.28:6823/4127602407","heartbeat_front_addr":"172.21.15.28:6821/4127602407","state":["exists","up"]},{"osd":9,"uuid":"9a603a47-7c5a-47d4-a298-56b3ac991892","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":4168729845},{"type":"v1","addr":"172.21.15.42:6818","nonce":4168729845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":4168729845},{"type":"v1","addr":"172.21.15.42:6822","nonce":4168729845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":4168729845},{"type":"v1","addr":"172.21.15.42:6830","nonce":4168729845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":4168729845},{"type":"v1","addr":"172.21.15.42:6826","nonce":4168729845}]},"public_addr":"172.21.15.42:6818/4168729845","cluster_addr":"172.21.15.42:6822/4168729845","heartbeat_back_addr":"172.21.15.42:6830/4168729845","heartbeat_front_addr":"172.21.15.42:6826/4168729845","state":["exists","up"]},{"osd":10,"uuid":"1cd42945-6ee1-49f0-b17b-4093dd36b4ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4180572718},{"type":"v1","addr":"172.21.15.66:6817","nonce":4180572718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":4180572718},{"type":"v1","addr":"172.21.15.66:6819","nonce":4180572718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":4180572718},{"type":"v1","addr":"172.21.15.66:6823","nonce":4180572718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":4180572718},{"type":"v1","addr":"172.21.15.66:6821","nonce":4180572718}]},"public_addr":"172.21.15.66:6817/4180572718","cluster_addr":"172.21.15.66:6819/4180572718","heartbeat_back_addr":"172.21.15.66:6823/4180572718","heartbeat_front_addr":"172.21.15.66:6821/4180572718","state":["exists","up"]},{"osd":11,"uuid":"b41e8100-1d4f-4b7a-991a-7a27bc29feec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":295713815},{"type":"v1","addr":"172.21.15.92:6819","nonce":295713815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":295713815},{"type":"v1","addr":"172.21.15.92:6821","nonce":295713815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":295713815},{"type":"v1","addr":"172.21.15.92:6825","nonce":295713815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":295713815},{"type":"v1","addr":"172.21.15.92:6823","nonce":295713815}]},"public_addr":"172.21.15.92:6819/295713815","cluster_addr":"172.21.15.92:6821/295713815","heartbeat_back_addr":"172.21.15.92:6825/295713815","heartbeat_front_addr":"172.21.15.92:6823/295713815","state":["exists","up"]},{"osd":12,"uuid":"458cf150-32e1-449e-b533-ac6ab5226f85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":2081141953},{"type":"v1","addr":"172.21.15.28:6825","nonce":2081141953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2081141953},{"type":"v1","addr":"172.21.15.28:6827","nonce":2081141953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2081141953},{"type":"v1","addr":"172.21.15.28:6831","nonce":2081141953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":2081141953},{"type":"v1","addr":"172.21.15.28:6829","nonce":2081141953}]},"public_addr":"172.21.15.28:6825/2081141953","cluster_addr":"172.21.15.28:6827/2081141953","heartbeat_back_addr":"172.21.15.28:6831/2081141953","heartbeat_front_addr":"172.21.15.28:6829/2081141953","state":["exists","up"]},{"osd":13,"uuid":"f008462e-a735-4845-9ecf-dd7d4a6941af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6817","nonce":301360893},{"type":"v1","addr":"172.21.15.42:6819","nonce":301360893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6821","nonce":301360893},{"type":"v1","addr":"172.21.15.42:6823","nonce":301360893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6829","nonce":301360893},{"type":"v1","addr":"172.21.15.42:6831","nonce":301360893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6825","nonce":301360893},{"type":"v1","addr":"172.21.15.42:6827","nonce":301360893}]},"public_addr":"172.21.15.42:6819/301360893","cluster_addr":"172.21.15.42:6823/301360893","heartbeat_back_addr":"172.21.15.42:6831/301360893","heartbeat_front_addr":"172.21.15.42:6827/301360893","state":["exists","up"]},{"osd":14,"uuid":"6f66ae97-eb8a-48eb-9bcc-20f65fbf690b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":1534905602},{"type":"v1","addr":"172.21.15.66:6825","nonce":1534905602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1534905602},{"type":"v1","addr":"172.21.15.66:6827","nonce":1534905602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1534905602},{"type":"v1","addr":"172.21.15.66:6831","nonce":1534905602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1534905602},{"type":"v1","addr":"172.21.15.66:6829","nonce":1534905602}]},"public_addr":"172.21.15.66:6825/1534905602","cluster_addr":"172.21.15.66:6827/1534905602","heartbeat_back_addr":"172.21.15.66:6831/1534905602","heartbeat_front_addr":"172.21.15.66:6829/1534905602","state":["exists","up"]},{"osd":15,"uuid":"b856bfc8-253b-4112-be5c-f67d61d01768","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3982634323},{"type":"v1","addr":"172.21.15.92:6827","nonce":3982634323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3982634323},{"type":"v1","addr":"172.21.15.92:6829","nonce":3982634323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3982634323},{"type":"v1","addr":"172.21.15.92:6833","nonce":3982634323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3982634323},{"type":"v1","addr":"172.21.15.92:6831","nonce":3982634323}]},"public_addr":"172.21.15.92:6827/3982634323","cluster_addr":"172.21.15.92:6829/3982634323","heartbeat_back_addr":"172.21.15.92:6833/3982634323","heartbeat_front_addr":"172.21.15.92:6831/3982634323","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.265053+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.243502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.252440+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.295331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.213076+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.281193+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.315801+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.329538+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.293303+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.334962+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.233032+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.305138+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.351964+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.241086+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.352515+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.357462+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T03:38:37.107 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-26T03:38:37.108 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T03:38:37.410 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:38:37.410 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":27,"fsid":"c5a13c99-5c9a-4b8b-a893-4f6f4211677a","created":"2024-04-26T03:38:19.896648+0000","modified":"2024-04-26T03:38:36.323522+0000","last_up_change":"2024-04-26T03:38:33.308561+0000","last_in_change":"2024-04-26T03:38:24.150336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T03:38:32.306696+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-26T03:38:34.592633+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":3.9900000095367432,"score_stable":3.9900000095367432,"optimal_score":0.68999999761581421,"raw_score_acting":2.75,"raw_score_stable":2.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"31b0e7a9-455d-4874-9c60-26efac8d0b1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2377656090},{"type":"v1","addr":"172.21.15.28:6801","nonce":2377656090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2377656090},{"type":"v1","addr":"172.21.15.28:6803","nonce":2377656090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2377656090},{"type":"v1","addr":"172.21.15.28:6807","nonce":2377656090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2377656090},{"type":"v1","addr":"172.21.15.28:6805","nonce":2377656090}]},"public_addr":"172.21.15.28:6801/2377656090","cluster_addr":"172.21.15.28:6803/2377656090","heartbeat_back_addr":"172.21.15.28:6807/2377656090","heartbeat_front_addr":"172.21.15.28:6805/2377656090","state":["exists","up"]},{"osd":1,"uuid":"52819ace-a070-4822-b33e-3c54466e61ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":1678150206},{"type":"v1","addr":"172.21.15.42:6801","nonce":1678150206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1678150206},{"type":"v1","addr":"172.21.15.42:6803","nonce":1678150206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1678150206},{"type":"v1","addr":"172.21.15.42:6807","nonce":1678150206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1678150206},{"type":"v1","addr":"172.21.15.42:6805","nonce":1678150206}]},"public_addr":"172.21.15.42:6801/1678150206","cluster_addr":"172.21.15.42:6803/1678150206","heartbeat_back_addr":"172.21.15.42:6807/1678150206","heartbeat_front_addr":"172.21.15.42:6805/1678150206","state":["exists","up"]},{"osd":2,"uuid":"858f8ac1-2edc-43d5-914c-c1914bdc0c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":714535415},{"type":"v1","addr":"172.21.15.66:6801","nonce":714535415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":714535415},{"type":"v1","addr":"172.21.15.66:6803","nonce":714535415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":714535415},{"type":"v1","addr":"172.21.15.66:6807","nonce":714535415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":714535415},{"type":"v1","addr":"172.21.15.66:6805","nonce":714535415}]},"public_addr":"172.21.15.66:6801/714535415","cluster_addr":"172.21.15.66:6803/714535415","heartbeat_back_addr":"172.21.15.66:6807/714535415","heartbeat_front_addr":"172.21.15.66:6805/714535415","state":["exists","up"]},{"osd":3,"uuid":"531a2e30-3351-4f84-804b-5fb5050b84ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2377983625},{"type":"v1","addr":"172.21.15.92:6811","nonce":2377983625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2377983625},{"type":"v1","addr":"172.21.15.92:6813","nonce":2377983625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2377983625},{"type":"v1","addr":"172.21.15.92:6817","nonce":2377983625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2377983625},{"type":"v1","addr":"172.21.15.92:6815","nonce":2377983625}]},"public_addr":"172.21.15.92:6811/2377983625","cluster_addr":"172.21.15.92:6813/2377983625","heartbeat_back_addr":"172.21.15.92:6817/2377983625","heartbeat_front_addr":"172.21.15.92:6815/2377983625","state":["exists","up"]},{"osd":4,"uuid":"6d38ffda-40eb-4fd7-9300-2a09f2673e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":183752455},{"type":"v1","addr":"172.21.15.28:6809","nonce":183752455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":183752455},{"type":"v1","addr":"172.21.15.28:6811","nonce":183752455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":183752455},{"type":"v1","addr":"172.21.15.28:6815","nonce":183752455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":183752455},{"type":"v1","addr":"172.21.15.28:6813","nonce":183752455}]},"public_addr":"172.21.15.28:6809/183752455","cluster_addr":"172.21.15.28:6811/183752455","heartbeat_back_addr":"172.21.15.28:6815/183752455","heartbeat_front_addr":"172.21.15.28:6813/183752455","state":["exists","up"]},{"osd":5,"uuid":"361aa40d-e53b-42af-afd4-81fbcac28725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2012261939},{"type":"v1","addr":"172.21.15.42:6809","nonce":2012261939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2012261939},{"type":"v1","addr":"172.21.15.42:6811","nonce":2012261939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2012261939},{"type":"v1","addr":"172.21.15.42:6815","nonce":2012261939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2012261939},{"type":"v1","addr":"172.21.15.42:6813","nonce":2012261939}]},"public_addr":"172.21.15.42:6809/2012261939","cluster_addr":"172.21.15.42:6811/2012261939","heartbeat_back_addr":"172.21.15.42:6815/2012261939","heartbeat_front_addr":"172.21.15.42:6813/2012261939","state":["exists","up"]},{"osd":6,"uuid":"f610ad3e-1fd8-42bd-af4b-0d79c3bb9cf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":3533460882},{"type":"v1","addr":"172.21.15.66:6809","nonce":3533460882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":3533460882},{"type":"v1","addr":"172.21.15.66:6811","nonce":3533460882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":3533460882},{"type":"v1","addr":"172.21.15.66:6815","nonce":3533460882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":3533460882},{"type":"v1","addr":"172.21.15.66:6813","nonce":3533460882}]},"public_addr":"172.21.15.66:6809/3533460882","cluster_addr":"172.21.15.66:6811/3533460882","heartbeat_back_addr":"172.21.15.66:6815/3533460882","heartbeat_front_addr":"172.21.15.66:6813/3533460882","state":["exists","up"]},{"osd":7,"uuid":"a4bae6fb-e087-4563-a129-6938e18eed8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":1760163933},{"type":"v1","addr":"172.21.15.92:6803","nonce":1760163933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":1760163933},{"type":"v1","addr":"172.21.15.92:6805","nonce":1760163933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1760163933},{"type":"v1","addr":"172.21.15.92:6809","nonce":1760163933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":1760163933},{"type":"v1","addr":"172.21.15.92:6807","nonce":1760163933}]},"public_addr":"172.21.15.92:6803/1760163933","cluster_addr":"172.21.15.92:6805/1760163933","heartbeat_back_addr":"172.21.15.92:6809/1760163933","heartbeat_front_addr":"172.21.15.92:6807/1760163933","state":["exists","up"]},{"osd":8,"uuid":"479f3885-452e-4231-bdc2-b79668384ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":4127602407},{"type":"v1","addr":"172.21.15.28:6817","nonce":4127602407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":4127602407},{"type":"v1","addr":"172.21.15.28:6819","nonce":4127602407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":4127602407},{"type":"v1","addr":"172.21.15.28:6823","nonce":4127602407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":4127602407},{"type":"v1","addr":"172.21.15.28:6821","nonce":4127602407}]},"public_addr":"172.21.15.28:6817/4127602407","cluster_addr":"172.21.15.28:6819/4127602407","heartbeat_back_addr":"172.21.15.28:6823/4127602407","heartbeat_front_addr":"172.21.15.28:6821/4127602407","state":["exists","up"]},{"osd":9,"uuid":"9a603a47-7c5a-47d4-a298-56b3ac991892","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":4168729845},{"type":"v1","addr":"172.21.15.42:6818","nonce":4168729845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":4168729845},{"type":"v1","addr":"172.21.15.42:6822","nonce":4168729845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":4168729845},{"type":"v1","addr":"172.21.15.42:6830","nonce":4168729845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":4168729845},{"type":"v1","addr":"172.21.15.42:6826","nonce":4168729845}]},"public_addr":"172.21.15.42:6818/4168729845","cluster_addr":"172.21.15.42:6822/4168729845","heartbeat_back_addr":"172.21.15.42:6830/4168729845","heartbeat_front_addr":"172.21.15.42:6826/4168729845","state":["exists","up"]},{"osd":10,"uuid":"1cd42945-6ee1-49f0-b17b-4093dd36b4ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4180572718},{"type":"v1","addr":"172.21.15.66:6817","nonce":4180572718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":4180572718},{"type":"v1","addr":"172.21.15.66:6819","nonce":4180572718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":4180572718},{"type":"v1","addr":"172.21.15.66:6823","nonce":4180572718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":4180572718},{"type":"v1","addr":"172.21.15.66:6821","nonce":4180572718}]},"public_addr":"172.21.15.66:6817/4180572718","cluster_addr":"172.21.15.66:6819/4180572718","heartbeat_back_addr":"172.21.15.66:6823/4180572718","heartbeat_front_addr":"172.21.15.66:6821/4180572718","state":["exists","up"]},{"osd":11,"uuid":"b41e8100-1d4f-4b7a-991a-7a27bc29feec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":295713815},{"type":"v1","addr":"172.21.15.92:6819","nonce":295713815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":295713815},{"type":"v1","addr":"172.21.15.92:6821","nonce":295713815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":295713815},{"type":"v1","addr":"172.21.15.92:6825","nonce":295713815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":295713815},{"type":"v1","addr":"172.21.15.92:6823","nonce":295713815}]},"public_addr":"172.21.15.92:6819/295713815","cluster_addr":"172.21.15.92:6821/295713815","heartbeat_back_addr":"172.21.15.92:6825/295713815","heartbeat_front_addr":"172.21.15.92:6823/295713815","state":["exists","up"]},{"osd":12,"uuid":"458cf150-32e1-449e-b533-ac6ab5226f85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":2081141953},{"type":"v1","addr":"172.21.15.28:6825","nonce":2081141953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2081141953},{"type":"v1","addr":"172.21.15.28:6827","nonce":2081141953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2081141953},{"type":"v1","addr":"172.21.15.28:6831","nonce":2081141953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":2081141953},{"type":"v1","addr":"172.21.15.28:6829","nonce":2081141953}]},"public_addr":"172.21.15.28:6825/2081141953","cluster_addr":"172.21.15.28:6827/2081141953","heartbeat_back_addr":"172.21.15.28:6831/2081141953","heartbeat_front_addr":"172.21.15.28:6829/2081141953","state":["exists","up"]},{"osd":13,"uuid":"f008462e-a735-4845-9ecf-dd7d4a6941af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6817","nonce":301360893},{"type":"v1","addr":"172.21.15.42:6819","nonce":301360893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6821","nonce":301360893},{"type":"v1","addr":"172.21.15.42:6823","nonce":301360893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6829","nonce":301360893},{"type":"v1","addr":"172.21.15.42:6831","nonce":301360893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6825","nonce":301360893},{"type":"v1","addr":"172.21.15.42:6827","nonce":301360893}]},"public_addr":"172.21.15.42:6819/301360893","cluster_addr":"172.21.15.42:6823/301360893","heartbeat_back_addr":"172.21.15.42:6831/301360893","heartbeat_front_addr":"172.21.15.42:6827/301360893","state":["exists","up"]},{"osd":14,"uuid":"6f66ae97-eb8a-48eb-9bcc-20f65fbf690b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":1534905602},{"type":"v1","addr":"172.21.15.66:6825","nonce":1534905602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1534905602},{"type":"v1","addr":"172.21.15.66:6827","nonce":1534905602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1534905602},{"type":"v1","addr":"172.21.15.66:6831","nonce":1534905602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1534905602},{"type":"v1","addr":"172.21.15.66:6829","nonce":1534905602}]},"public_addr":"172.21.15.66:6825/1534905602","cluster_addr":"172.21.15.66:6827/1534905602","heartbeat_back_addr":"172.21.15.66:6831/1534905602","heartbeat_front_addr":"172.21.15.66:6829/1534905602","state":["exists","up"]},{"osd":15,"uuid":"b856bfc8-253b-4112-be5c-f67d61d01768","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3982634323},{"type":"v1","addr":"172.21.15.92:6827","nonce":3982634323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3982634323},{"type":"v1","addr":"172.21.15.92:6829","nonce":3982634323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3982634323},{"type":"v1","addr":"172.21.15.92:6833","nonce":3982634323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3982634323},{"type":"v1","addr":"172.21.15.92:6831","nonce":3982634323}]},"public_addr":"172.21.15.92:6827/3982634323","cluster_addr":"172.21.15.92:6829/3982634323","heartbeat_back_addr":"172.21.15.92:6833/3982634323","heartbeat_front_addr":"172.21.15.92:6831/3982634323","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.265053+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.243502+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.252440+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.295331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.213076+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.281193+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.315801+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.329538+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.293303+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.334962+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.233032+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.305138+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.351964+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.241086+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.352515+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.357462+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T03:38:37.424 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-26T03:38:37.425 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-26T03:38:37.425 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-26T03:38:37.425 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-26T03:38:37.426 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-26T03:38:37.426 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-26T03:38:37.426 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-26T03:38:37.427 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-26T03:38:37.427 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 flush_pg_stats 2024-04-26T03:38:37.428 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 flush_pg_stats 2024-04-26T03:38:37.428 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 flush_pg_stats 2024-04-26T03:38:37.429 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 flush_pg_stats 2024-04-26T03:38:37.429 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.12 flush_pg_stats 2024-04-26T03:38:37.429 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.13 flush_pg_stats 2024-04-26T03:38:37.430 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.14 flush_pg_stats 2024-04-26T03:38:37.430 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.15 flush_pg_stats 2024-04-26T03:38:37.869 INFO:teuthology.orchestra.run.smithi028.stdout:94489280515 2024-04-26T03:38:37.870 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.12 2024-04-26T03:38:37.878 INFO:teuthology.orchestra.run.smithi028.stdout:98784247814 2024-04-26T03:38:37.878 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-26T03:38:37.881 INFO:teuthology.orchestra.run.smithi028.stdout:98784247812 2024-04-26T03:38:37.881 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-26T03:38:37.884 INFO:teuthology.orchestra.run.smithi028.stdout:103079215109 2024-04-26T03:38:37.884 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-26T03:38:37.913 INFO:teuthology.orchestra.run.smithi028.stdout:94489280517 2024-04-26T03:38:37.915 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2024-04-26T03:38:37.917 INFO:teuthology.orchestra.run.smithi028.stdout:98784247814 2024-04-26T03:38:37.917 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-26T03:38:37.933 INFO:teuthology.orchestra.run.smithi028.stdout:94489280516 2024-04-26T03:38:37.934 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-26T03:38:37.935 INFO:teuthology.orchestra.run.smithi028.stdout:98784247812 2024-04-26T03:38:37.935 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-26T03:38:37.936 INFO:teuthology.orchestra.run.smithi028.stdout:94489280515 2024-04-26T03:38:37.936 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.15 2024-04-26T03:38:37.940 INFO:teuthology.orchestra.run.smithi028.stdout:94489280515 2024-04-26T03:38:37.941 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.13 2024-04-26T03:38:37.948 INFO:teuthology.orchestra.run.smithi028.stdout:94489280515 2024-04-26T03:38:37.948 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2024-04-26T03:38:37.957 INFO:teuthology.orchestra.run.smithi028.stdout:90194313221 2024-04-26T03:38:37.958 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2024-04-26T03:38:37.959 INFO:teuthology.orchestra.run.smithi028.stdout:94489280516 2024-04-26T03:38:37.959 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2024-04-26T03:38:37.979 INFO:teuthology.orchestra.run.smithi028.stdout:94489280515 2024-04-26T03:38:37.979 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.14 2024-04-26T03:38:37.983 INFO:teuthology.orchestra.run.smithi028.stdout:103079215109 2024-04-26T03:38:37.983 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-26T03:38:38.022 INFO:teuthology.orchestra.run.smithi028.stdout:94489280516 2024-04-26T03:38:38.022 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-26T03:38:38.851 INFO:teuthology.orchestra.run.smithi028.stdout:98784247812 2024-04-26T03:38:38.892 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247812 got 98784247812 for osd.2 2024-04-26T03:38:38.893 DEBUG:teuthology.parallel:result is None 2024-04-26T03:38:39.006 INFO:teuthology.orchestra.run.smithi028.stdout:94489280515 2024-04-26T03:38:39.012 INFO:teuthology.orchestra.run.smithi028.stdout:94489280515 2024-04-26T03:38:39.032 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280515 got 94489280515 for osd.13 2024-04-26T03:38:39.032 DEBUG:teuthology.parallel:result is None 2024-04-26T03:38:39.044 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280515 got 94489280515 for osd.12 2024-04-26T03:38:39.044 DEBUG:teuthology.parallel:result is None 2024-04-26T03:38:39.050 INFO:teuthology.orchestra.run.smithi028.stdout:98784247814 2024-04-26T03:38:39.070 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247814 got 98784247814 for osd.4 2024-04-26T03:38:39.071 DEBUG:teuthology.parallel:result is None 2024-04-26T03:38:39.116 INFO:teuthology.orchestra.run.smithi028.stdout:103079215109 2024-04-26T03:38:39.139 INFO:teuthology.orchestra.run.smithi028.stdout:98784247814 2024-04-26T03:38:39.158 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247814 got 98784247814 for osd.1 2024-04-26T03:38:39.158 DEBUG:teuthology.parallel:result is None 2024-04-26T03:38:39.170 INFO:tasks.ceph.ceph_manager.ceph:need seq 103079215109 got 103079215109 for osd.3 2024-04-26T03:38:39.170 DEBUG:teuthology.parallel:result is None 2024-04-26T03:38:39.181 INFO:teuthology.orchestra.run.smithi028.stdout:94489280516 2024-04-26T03:38:39.185 INFO:teuthology.orchestra.run.smithi028.stdout:94489280515 2024-04-26T03:38:39.189 INFO:teuthology.orchestra.run.smithi028.stdout:94489280517 2024-04-26T03:38:39.197 INFO:teuthology.orchestra.run.smithi028.stdout:103079215109 2024-04-26T03:38:39.204 INFO:teuthology.orchestra.run.smithi028.stdout:94489280515 2024-04-26T03:38:39.204 INFO:teuthology.orchestra.run.smithi028.stdout:94489280515 2024-04-26T03:38:39.206 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280515 got 94489280515 for osd.9 2024-04-26T03:38:39.206 DEBUG:teuthology.parallel:result is None 2024-04-26T03:38:39.219 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280517 got 94489280517 for osd.8 2024-04-26T03:38:39.219 INFO:teuthology.orchestra.run.smithi028.stdout:98784247812 2024-04-26T03:38:39.219 DEBUG:teuthology.parallel:result is None 2024-04-26T03:38:39.221 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280516 for osd.7 2024-04-26T03:38:39.221 DEBUG:teuthology.parallel:result is None 2024-04-26T03:38:39.222 INFO:tasks.ceph.ceph_manager.ceph:need seq 103079215109 got 103079215109 for osd.5 2024-04-26T03:38:39.222 DEBUG:teuthology.parallel:result is None 2024-04-26T03:38:39.225 INFO:teuthology.orchestra.run.smithi028.stdout:94489280516 2024-04-26T03:38:39.228 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280515 got 94489280515 for osd.15 2024-04-26T03:38:39.228 DEBUG:teuthology.parallel:result is None 2024-04-26T03:38:39.230 INFO:teuthology.orchestra.run.smithi028.stdout:90194313221 2024-04-26T03:38:39.236 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280515 got 94489280515 for osd.14 2024-04-26T03:38:39.236 INFO:teuthology.orchestra.run.smithi028.stdout:94489280516 2024-04-26T03:38:39.236 DEBUG:teuthology.parallel:result is None 2024-04-26T03:38:39.241 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247812 got 98784247812 for osd.0 2024-04-26T03:38:39.241 DEBUG:teuthology.parallel:result is None 2024-04-26T03:38:39.244 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280516 for osd.10 2024-04-26T03:38:39.244 DEBUG:teuthology.parallel:result is None 2024-04-26T03:38:39.247 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313221 got 90194313221 for osd.11 2024-04-26T03:38:39.248 DEBUG:teuthology.parallel:result is None 2024-04-26T03:38:39.251 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280516 for osd.6 2024-04-26T03:38:39.251 DEBUG:teuthology.parallel:result is None 2024-04-26T03:38:39.251 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-26T03:38:39.252 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T03:38:39.588 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:38:39.589 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-04-26T03:38:39.601 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-04-26T03:38:38.441520+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":12,"kb":1509949440,"kb_used":453128,"kb_used_data":4060,"kb_used_omap":24,"kb_used_meta":448743,"kb_avail":1509496312,"statfs":{"total":1546188226560,"available":1545724223488,"internally_reserved":0,"allocated":4157440,"data_stored":2776924,"data_compressed":131319,"data_compressed_allocated":323584,"data_compressed_original":1540096,"omap_allocated":25522,"internal_metadata":459512910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.023463"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-26T03:38:37.714711+0000","last_change":"2024-04-26T03:38:37.359252+0000","last_active":"2024-04-26T03:38:37.714711+0000","last_peered":"2024-04-26T03:38:37.714711+0000","last_clean":"2024-04-26T03:38:37.714711+0000","last_became_active":"2024-04-26T03:38:37.358910+0000","last_became_peered":"2024-04-26T03:38:37.358910+0000","last_unstale":"2024-04-26T03:38:37.714711+0000","last_undegraded":"2024-04-26T03:38:37.714711+0000","last_fullsized":"2024-04-26T03:38:37.714711+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:40.814654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-26T03:38:37.419477+0000","last_change":"2024-04-26T03:38:36.484782+0000","last_active":"2024-04-26T03:38:37.419477+0000","last_peered":"2024-04-26T03:38:37.419477+0000","last_clean":"2024-04-26T03:38:37.419477+0000","last_became_active":"2024-04-26T03:38:36.484388+0000","last_became_peered":"2024-04-26T03:38:36.484388+0000","last_unstale":"2024-04-26T03:38:37.419477+0000","last_undegraded":"2024-04-26T03:38:37.419477+0000","last_fullsized":"2024-04-26T03:38:37.419477+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:58.542524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,13],"acting":[8,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":4,"reported_epoch":28,"state":"creating+activating","last_fresh":"2024-04-26T03:38:37.423108+0000","last_change":"2024-04-26T03:38:37.423234+0000","last_active":"2024-04-26T03:38:35.319452+0000","last_peered":"2024-04-26T03:38:35.319452+0000","last_clean":"2024-04-26T03:38:35.319452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T03:38:37.423108+0000","last_undegraded":"2024-04-26T03:38:37.423108+0000","last_fullsized":"2024-04-26T03:38:37.423108+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4],"acting":[9,4],"avail_no_missing":["9","4"],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-26T03:38:38.184946+0000","last_change":"2024-04-26T03:38:37.255389+0000","last_active":"2024-04-26T03:38:38.184946+0000","last_peered":"2024-04-26T03:38:38.184946+0000","last_clean":"2024-04-26T03:38:38.184946+0000","last_became_active":"2024-04-26T03:38:37.255066+0000","last_became_peered":"2024-04-26T03:38:37.255066+0000","last_unstale":"2024-04-26T03:38:38.184946+0000","last_undegraded":"2024-04-26T03:38:38.184946+0000","last_fullsized":"2024-04-26T03:38:38.184946+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:54.145566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T03:38:36.447845+0000","last_change":"2024-04-26T03:38:36.447388+0000","last_active":"2024-04-26T03:38:36.447845+0000","last_peered":"2024-04-26T03:38:36.447845+0000","last_clean":"2024-04-26T03:38:36.447845+0000","last_became_active":"2024-04-26T03:38:36.447194+0000","last_became_peered":"2024-04-26T03:38:36.447194+0000","last_unstale":"2024-04-26T03:38:36.447845+0000","last_undegraded":"2024-04-26T03:38:36.447845+0000","last_fullsized":"2024-04-26T03:38:36.447845+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:43.420813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-26T03:38:37.715445+0000","last_change":"2024-04-26T03:38:37.255450+0000","last_active":"2024-04-26T03:38:37.715445+0000","last_peered":"2024-04-26T03:38:37.715445+0000","last_clean":"2024-04-26T03:38:37.715445+0000","last_became_active":"2024-04-26T03:38:37.255181+0000","last_became_peered":"2024-04-26T03:38:37.255181+0000","last_unstale":"2024-04-26T03:38:37.715445+0000","last_undegraded":"2024-04-26T03:38:37.715445+0000","last_fullsized":"2024-04-26T03:38:37.715445+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:40.592285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0],"acting":[14,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T03:38:36.431757+0000","last_change":"2024-04-26T03:38:36.431325+0000","last_active":"2024-04-26T03:38:36.431757+0000","last_peered":"2024-04-26T03:38:36.431757+0000","last_clean":"2024-04-26T03:38:36.431757+0000","last_became_active":"2024-04-26T03:38:36.431121+0000","last_became_peered":"2024-04-26T03:38:36.431121+0000","last_unstale":"2024-04-26T03:38:36.431757+0000","last_undegraded":"2024-04-26T03:38:36.431757+0000","last_fullsized":"2024-04-26T03:38:36.431757+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:49.506263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-26T03:38:37.715446+0000","last_change":"2024-04-26T03:38:36.431667+0000","last_active":"2024-04-26T03:38:37.715446+0000","last_peered":"2024-04-26T03:38:37.715446+0000","last_clean":"2024-04-26T03:38:37.715446+0000","last_became_active":"2024-04-26T03:38:36.431449+0000","last_became_peered":"2024-04-26T03:38:36.431449+0000","last_unstale":"2024-04-26T03:38:37.715446+0000","last_undegraded":"2024-04-26T03:38:37.715446+0000","last_fullsized":"2024-04-26T03:38:37.715446+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:54.705392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"27'157","reported_seq":211,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T03:38:36.433490+0000","last_change":"2024-04-26T03:38:34.390206+0000","last_active":"2024-04-26T03:38:36.433490+0000","last_peered":"2024-04-26T03:38:36.433490+0000","last_clean":"2024-04-26T03:38:36.433490+0000","last_became_active":"2024-04-26T03:38:34.389099+0000","last_became_peered":"2024-04-26T03:38:34.389099+0000","last_unstale":"2024-04-26T03:38:36.433490+0000","last_undegraded":"2024-04-26T03:38:36.433490+0000","last_fullsized":"2024-04-26T03:38:36.433490+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:32.306810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:32.306810+0000","last_clean_scrub_stamp":"2024-04-26T03:38:32.306810+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:40.216975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":11},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":13852,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":24,"seq":103079215109,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31508,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340332,"statfs":{"total":96636764160,"available":96604499968,"internally_reserved":0,"allocated":307200,"data_stored":637033,"data_compressed":14386,"data_compressed_allocated":49152,"data_compressed_original":544768,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[4,6,7,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215109,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":108105,"data_compressed":7460,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,4,5,7,8,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":23,"seq":98784247815,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31428,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340412,"statfs":{"total":96636764160,"available":96604581888,"internally_reserved":0,"allocated":253952,"data_stored":108105,"data_compressed":7460,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,6,7,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":307200,"data_stored":637033,"data_compressed":14386,"data_compressed_allocated":49152,"data_compressed_original":544768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,4,5,7,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27260,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":241664,"data_stored":98603,"data_compressed":5567,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[6,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":22,"seq":94489280517,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":108105,"data_compressed":7460,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[2,5,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":241664,"data_stored":98603,"data_compressed":5567,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,2,7,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":241664,"data_stored":98603,"data_compressed":5567,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[1,6,7,8,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27260,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":241664,"data_stored":98603,"data_compressed":5567,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,8,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":22,"seq":94489280515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":108105,"data_compressed":7460,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[6,7,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":108105,"data_compressed":7460,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[4,6,7,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247813,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":266240,"data_stored":117237,"data_compressed":9353,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,4,6,7,8,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":108105,"data_compressed":7460,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[3,6,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":23,"seq":98784247815,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31444,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":266240,"data_stored":117237,"data_compressed":9353,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,4,6,7,8,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":108105,"data_compressed":7460,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,4,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247813,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":117237,"data_compressed":9353,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,4,6,7,8,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T03:38:39.602 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T03:38:39.896 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:38:39.896 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-04-26T03:38:39.908 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-04-26T03:38:38.441520+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":12,"kb":1509949440,"kb_used":453128,"kb_used_data":4060,"kb_used_omap":24,"kb_used_meta":448743,"kb_avail":1509496312,"statfs":{"total":1546188226560,"available":1545724223488,"internally_reserved":0,"allocated":4157440,"data_stored":2776924,"data_compressed":131319,"data_compressed_allocated":323584,"data_compressed_original":1540096,"omap_allocated":25522,"internal_metadata":459512910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.023463"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-26T03:38:37.714711+0000","last_change":"2024-04-26T03:38:37.359252+0000","last_active":"2024-04-26T03:38:37.714711+0000","last_peered":"2024-04-26T03:38:37.714711+0000","last_clean":"2024-04-26T03:38:37.714711+0000","last_became_active":"2024-04-26T03:38:37.358910+0000","last_became_peered":"2024-04-26T03:38:37.358910+0000","last_unstale":"2024-04-26T03:38:37.714711+0000","last_undegraded":"2024-04-26T03:38:37.714711+0000","last_fullsized":"2024-04-26T03:38:37.714711+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:40.814654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-26T03:38:37.419477+0000","last_change":"2024-04-26T03:38:36.484782+0000","last_active":"2024-04-26T03:38:37.419477+0000","last_peered":"2024-04-26T03:38:37.419477+0000","last_clean":"2024-04-26T03:38:37.419477+0000","last_became_active":"2024-04-26T03:38:36.484388+0000","last_became_peered":"2024-04-26T03:38:36.484388+0000","last_unstale":"2024-04-26T03:38:37.419477+0000","last_undegraded":"2024-04-26T03:38:37.419477+0000","last_fullsized":"2024-04-26T03:38:37.419477+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:58.542524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,13],"acting":[8,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":4,"reported_epoch":28,"state":"creating+activating","last_fresh":"2024-04-26T03:38:37.423108+0000","last_change":"2024-04-26T03:38:37.423234+0000","last_active":"2024-04-26T03:38:35.319452+0000","last_peered":"2024-04-26T03:38:35.319452+0000","last_clean":"2024-04-26T03:38:35.319452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T03:38:37.423108+0000","last_undegraded":"2024-04-26T03:38:37.423108+0000","last_fullsized":"2024-04-26T03:38:37.423108+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4],"acting":[9,4],"avail_no_missing":["9","4"],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-26T03:38:38.184946+0000","last_change":"2024-04-26T03:38:37.255389+0000","last_active":"2024-04-26T03:38:38.184946+0000","last_peered":"2024-04-26T03:38:38.184946+0000","last_clean":"2024-04-26T03:38:38.184946+0000","last_became_active":"2024-04-26T03:38:37.255066+0000","last_became_peered":"2024-04-26T03:38:37.255066+0000","last_unstale":"2024-04-26T03:38:38.184946+0000","last_undegraded":"2024-04-26T03:38:38.184946+0000","last_fullsized":"2024-04-26T03:38:38.184946+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:54.145566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T03:38:36.447845+0000","last_change":"2024-04-26T03:38:36.447388+0000","last_active":"2024-04-26T03:38:36.447845+0000","last_peered":"2024-04-26T03:38:36.447845+0000","last_clean":"2024-04-26T03:38:36.447845+0000","last_became_active":"2024-04-26T03:38:36.447194+0000","last_became_peered":"2024-04-26T03:38:36.447194+0000","last_unstale":"2024-04-26T03:38:36.447845+0000","last_undegraded":"2024-04-26T03:38:36.447845+0000","last_fullsized":"2024-04-26T03:38:36.447845+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:43.420813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-26T03:38:37.715445+0000","last_change":"2024-04-26T03:38:37.255450+0000","last_active":"2024-04-26T03:38:37.715445+0000","last_peered":"2024-04-26T03:38:37.715445+0000","last_clean":"2024-04-26T03:38:37.715445+0000","last_became_active":"2024-04-26T03:38:37.255181+0000","last_became_peered":"2024-04-26T03:38:37.255181+0000","last_unstale":"2024-04-26T03:38:37.715445+0000","last_undegraded":"2024-04-26T03:38:37.715445+0000","last_fullsized":"2024-04-26T03:38:37.715445+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:40.592285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0],"acting":[14,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T03:38:36.431757+0000","last_change":"2024-04-26T03:38:36.431325+0000","last_active":"2024-04-26T03:38:36.431757+0000","last_peered":"2024-04-26T03:38:36.431757+0000","last_clean":"2024-04-26T03:38:36.431757+0000","last_became_active":"2024-04-26T03:38:36.431121+0000","last_became_peered":"2024-04-26T03:38:36.431121+0000","last_unstale":"2024-04-26T03:38:36.431757+0000","last_undegraded":"2024-04-26T03:38:36.431757+0000","last_fullsized":"2024-04-26T03:38:36.431757+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:49.506263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-26T03:38:37.715446+0000","last_change":"2024-04-26T03:38:36.431667+0000","last_active":"2024-04-26T03:38:37.715446+0000","last_peered":"2024-04-26T03:38:37.715446+0000","last_clean":"2024-04-26T03:38:37.715446+0000","last_became_active":"2024-04-26T03:38:36.431449+0000","last_became_peered":"2024-04-26T03:38:36.431449+0000","last_unstale":"2024-04-26T03:38:37.715446+0000","last_undegraded":"2024-04-26T03:38:37.715446+0000","last_fullsized":"2024-04-26T03:38:37.715446+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:54.705392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"27'157","reported_seq":211,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T03:38:36.433490+0000","last_change":"2024-04-26T03:38:34.390206+0000","last_active":"2024-04-26T03:38:36.433490+0000","last_peered":"2024-04-26T03:38:36.433490+0000","last_clean":"2024-04-26T03:38:36.433490+0000","last_became_active":"2024-04-26T03:38:34.389099+0000","last_became_peered":"2024-04-26T03:38:34.389099+0000","last_unstale":"2024-04-26T03:38:36.433490+0000","last_undegraded":"2024-04-26T03:38:36.433490+0000","last_fullsized":"2024-04-26T03:38:36.433490+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:32.306810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:32.306810+0000","last_clean_scrub_stamp":"2024-04-26T03:38:32.306810+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:40.216975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":11},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":13852,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":24,"seq":103079215109,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31508,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340332,"statfs":{"total":96636764160,"available":96604499968,"internally_reserved":0,"allocated":307200,"data_stored":637033,"data_compressed":14386,"data_compressed_allocated":49152,"data_compressed_original":544768,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[4,6,7,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215109,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":108105,"data_compressed":7460,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,4,5,7,8,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":23,"seq":98784247815,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31428,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340412,"statfs":{"total":96636764160,"available":96604581888,"internally_reserved":0,"allocated":253952,"data_stored":108105,"data_compressed":7460,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,6,7,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":307200,"data_stored":637033,"data_compressed":14386,"data_compressed_allocated":49152,"data_compressed_original":544768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,4,5,7,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27260,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":241664,"data_stored":98603,"data_compressed":5567,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[6,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":22,"seq":94489280517,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":108105,"data_compressed":7460,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[2,5,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":241664,"data_stored":98603,"data_compressed":5567,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,2,7,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":241664,"data_stored":98603,"data_compressed":5567,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[1,6,7,8,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27260,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":241664,"data_stored":98603,"data_compressed":5567,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,8,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":22,"seq":94489280515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":108105,"data_compressed":7460,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[6,7,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":108105,"data_compressed":7460,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[4,6,7,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247813,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":266240,"data_stored":117237,"data_compressed":9353,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,4,6,7,8,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":108105,"data_compressed":7460,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[3,6,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":23,"seq":98784247815,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31444,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":266240,"data_stored":117237,"data_compressed":9353,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,4,6,7,8,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":108105,"data_compressed":7460,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,4,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247813,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":117237,"data_compressed":9353,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,4,6,7,8,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T03:38:39.909 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T03:38:40.203 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:38:40.203 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-04-26T03:38:40.215 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-04-26T03:38:40.049645+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":12,"kb":1509949440,"kb_used":453152,"kb_used_data":4084,"kb_used_omap":24,"kb_used_meta":448743,"kb_avail":1509496288,"statfs":{"total":1546188226560,"available":1545724198912,"internally_reserved":0,"allocated":4182016,"data_stored":2795188,"data_compressed":135105,"data_compressed_allocated":331776,"data_compressed_original":1556480,"omap_allocated":25522,"internal_metadata":459512910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.631588"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-26T03:38:37.714711+0000","last_change":"2024-04-26T03:38:37.359252+0000","last_active":"2024-04-26T03:38:37.714711+0000","last_peered":"2024-04-26T03:38:37.714711+0000","last_clean":"2024-04-26T03:38:37.714711+0000","last_became_active":"2024-04-26T03:38:37.358910+0000","last_became_peered":"2024-04-26T03:38:37.358910+0000","last_unstale":"2024-04-26T03:38:37.714711+0000","last_undegraded":"2024-04-26T03:38:37.714711+0000","last_fullsized":"2024-04-26T03:38:37.714711+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:40.814654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-26T03:38:37.419477+0000","last_change":"2024-04-26T03:38:36.484782+0000","last_active":"2024-04-26T03:38:37.419477+0000","last_peered":"2024-04-26T03:38:37.419477+0000","last_clean":"2024-04-26T03:38:37.419477+0000","last_became_active":"2024-04-26T03:38:36.484388+0000","last_became_peered":"2024-04-26T03:38:36.484388+0000","last_unstale":"2024-04-26T03:38:37.419477+0000","last_undegraded":"2024-04-26T03:38:37.419477+0000","last_fullsized":"2024-04-26T03:38:37.419477+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:58.542524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,13],"acting":[8,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":4,"reported_epoch":28,"state":"creating+activating","last_fresh":"2024-04-26T03:38:37.423108+0000","last_change":"2024-04-26T03:38:37.423234+0000","last_active":"2024-04-26T03:38:35.319452+0000","last_peered":"2024-04-26T03:38:35.319452+0000","last_clean":"2024-04-26T03:38:35.319452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T03:38:37.423108+0000","last_undegraded":"2024-04-26T03:38:37.423108+0000","last_fullsized":"2024-04-26T03:38:37.423108+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4],"acting":[9,4],"avail_no_missing":["9","4"],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-26T03:38:38.184946+0000","last_change":"2024-04-26T03:38:37.255389+0000","last_active":"2024-04-26T03:38:38.184946+0000","last_peered":"2024-04-26T03:38:38.184946+0000","last_clean":"2024-04-26T03:38:38.184946+0000","last_became_active":"2024-04-26T03:38:37.255066+0000","last_became_peered":"2024-04-26T03:38:37.255066+0000","last_unstale":"2024-04-26T03:38:38.184946+0000","last_undegraded":"2024-04-26T03:38:38.184946+0000","last_fullsized":"2024-04-26T03:38:38.184946+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:54.145566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T03:38:38.834966+0000","last_change":"2024-04-26T03:38:36.447388+0000","last_active":"2024-04-26T03:38:38.834966+0000","last_peered":"2024-04-26T03:38:38.834966+0000","last_clean":"2024-04-26T03:38:38.834966+0000","last_became_active":"2024-04-26T03:38:36.447194+0000","last_became_peered":"2024-04-26T03:38:36.447194+0000","last_unstale":"2024-04-26T03:38:38.834966+0000","last_undegraded":"2024-04-26T03:38:38.834966+0000","last_fullsized":"2024-04-26T03:38:38.834966+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:42.465065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-26T03:38:37.715445+0000","last_change":"2024-04-26T03:38:37.255450+0000","last_active":"2024-04-26T03:38:37.715445+0000","last_peered":"2024-04-26T03:38:37.715445+0000","last_clean":"2024-04-26T03:38:37.715445+0000","last_became_active":"2024-04-26T03:38:37.255181+0000","last_became_peered":"2024-04-26T03:38:37.255181+0000","last_unstale":"2024-04-26T03:38:37.715445+0000","last_undegraded":"2024-04-26T03:38:37.715445+0000","last_fullsized":"2024-04-26T03:38:37.715445+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:40.592285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0],"acting":[14,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T03:38:38.834924+0000","last_change":"2024-04-26T03:38:36.431325+0000","last_active":"2024-04-26T03:38:38.834924+0000","last_peered":"2024-04-26T03:38:38.834924+0000","last_clean":"2024-04-26T03:38:38.834924+0000","last_became_active":"2024-04-26T03:38:36.431121+0000","last_became_peered":"2024-04-26T03:38:36.431121+0000","last_unstale":"2024-04-26T03:38:38.834924+0000","last_undegraded":"2024-04-26T03:38:38.834924+0000","last_fullsized":"2024-04-26T03:38:38.834924+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:53.294619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-26T03:38:37.715446+0000","last_change":"2024-04-26T03:38:36.431667+0000","last_active":"2024-04-26T03:38:37.715446+0000","last_peered":"2024-04-26T03:38:37.715446+0000","last_clean":"2024-04-26T03:38:37.715446+0000","last_became_active":"2024-04-26T03:38:36.431449+0000","last_became_peered":"2024-04-26T03:38:36.431449+0000","last_unstale":"2024-04-26T03:38:37.715446+0000","last_undegraded":"2024-04-26T03:38:37.715446+0000","last_fullsized":"2024-04-26T03:38:37.715446+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:54.705392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"27'157","reported_seq":211,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T03:38:36.433490+0000","last_change":"2024-04-26T03:38:34.390206+0000","last_active":"2024-04-26T03:38:36.433490+0000","last_peered":"2024-04-26T03:38:36.433490+0000","last_clean":"2024-04-26T03:38:36.433490+0000","last_became_active":"2024-04-26T03:38:34.389099+0000","last_became_peered":"2024-04-26T03:38:34.389099+0000","last_unstale":"2024-04-26T03:38:36.433490+0000","last_undegraded":"2024-04-26T03:38:36.433490+0000","last_fullsized":"2024-04-26T03:38:36.433490+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:32.306810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:32.306810+0000","last_clean_scrub_stamp":"2024-04-26T03:38:32.306810+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:40.216975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":11},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":13852,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":24,"seq":103079215110,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31524,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":319488,"data_stored":646165,"data_compressed":16279,"data_compressed_allocated":53248,"data_compressed_original":552960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[4,6,7,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215110,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":117237,"data_compressed":9353,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,4,5,7,8,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":23,"seq":98784247815,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31428,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340412,"statfs":{"total":96636764160,"available":96604581888,"internally_reserved":0,"allocated":253952,"data_stored":108105,"data_compressed":7460,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,6,7,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":307200,"data_stored":637033,"data_compressed":14386,"data_compressed_allocated":49152,"data_compressed_original":544768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,4,5,7,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27260,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":241664,"data_stored":98603,"data_compressed":5567,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[6,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":22,"seq":94489280517,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":108105,"data_compressed":7460,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[2,5,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":241664,"data_stored":98603,"data_compressed":5567,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,2,7,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":241664,"data_stored":98603,"data_compressed":5567,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[1,6,7,8,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27260,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":241664,"data_stored":98603,"data_compressed":5567,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,8,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":22,"seq":94489280515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":108105,"data_compressed":7460,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[6,7,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":108105,"data_compressed":7460,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[4,6,7,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247813,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":266240,"data_stored":117237,"data_compressed":9353,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,4,6,7,8,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":108105,"data_compressed":7460,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[3,6,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":23,"seq":98784247815,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31444,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":266240,"data_stored":117237,"data_compressed":9353,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,4,6,7,8,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":108105,"data_compressed":7460,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,4,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247813,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":117237,"data_compressed":9353,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,4,6,7,8,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T03:38:40.216 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2024-04-26T03:38:40.216 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 4, 'reported_epoch': 28, 'state': 'creating+activating', 'last_fresh': '2024-04-26T03:38:37.423108+0000', 'last_change': '2024-04-26T03:38:37.423234+0000', 'last_active': '2024-04-26T03:38:35.319452+0000', 'last_peered': '2024-04-26T03:38:35.319452+0000', 'last_clean': '2024-04-26T03:38:35.319452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T03:38:37.423108+0000', 'last_undegraded': '2024-04-26T03:38:37.423108+0000', 'last_fullsized': '2024-04-26T03:38:37.423108+0000', 'mapping_epoch': 26, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 26, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T03:38:35.319452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T03:38:35.319452+0000', 'last_clean_scrub_stamp': '2024-04-26T03:38:35.319452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [9, 4], 'acting': [9, 4], 'avail_no_missing': ['9', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 9, 'acting_primary': 9, 'purged_snaps': []} 2024-04-26T03:38:40.216 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T03:38:40.584 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:38:40.585 INFO:teuthology.orchestra.run.smithi028.stdout:{"fsid":"c5a13c99-5c9a-4b8b-a893-4f6f4211677a","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":20,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":29,"num_osds":16,"num_up_osds":16,"osd_up_since":1714102713,"num_in_osds":16,"osd_in_since":1714102704,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":8},{"state_name":"creating+activating","count":1}],"num_pgs":9,"num_pools":2,"num_objects":2,"data_bytes":590368,"bytes_used":464027648,"bytes_avail":1545724198912,"bytes_total":1546188226560,"inactive_pgs_ratio":0.1111111119389534},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-04-26T03:38:19.898356+0000","services":{}},"progress_events":{}} 2024-04-26T03:38:40.597 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-04-26T03:38:40.597 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T03:38:40.890 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:38:40.891 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-04-26T03:38:40.902 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-04-26T03:38:40.049645+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":12,"kb":1509949440,"kb_used":453152,"kb_used_data":4084,"kb_used_omap":24,"kb_used_meta":448743,"kb_avail":1509496288,"statfs":{"total":1546188226560,"available":1545724198912,"internally_reserved":0,"allocated":4182016,"data_stored":2795188,"data_compressed":135105,"data_compressed_allocated":331776,"data_compressed_original":1556480,"omap_allocated":25522,"internal_metadata":459512910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.631588"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-26T03:38:37.714711+0000","last_change":"2024-04-26T03:38:37.359252+0000","last_active":"2024-04-26T03:38:37.714711+0000","last_peered":"2024-04-26T03:38:37.714711+0000","last_clean":"2024-04-26T03:38:37.714711+0000","last_became_active":"2024-04-26T03:38:37.358910+0000","last_became_peered":"2024-04-26T03:38:37.358910+0000","last_unstale":"2024-04-26T03:38:37.714711+0000","last_undegraded":"2024-04-26T03:38:37.714711+0000","last_fullsized":"2024-04-26T03:38:37.714711+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:40.814654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-26T03:38:37.419477+0000","last_change":"2024-04-26T03:38:36.484782+0000","last_active":"2024-04-26T03:38:37.419477+0000","last_peered":"2024-04-26T03:38:37.419477+0000","last_clean":"2024-04-26T03:38:37.419477+0000","last_became_active":"2024-04-26T03:38:36.484388+0000","last_became_peered":"2024-04-26T03:38:36.484388+0000","last_unstale":"2024-04-26T03:38:37.419477+0000","last_undegraded":"2024-04-26T03:38:37.419477+0000","last_fullsized":"2024-04-26T03:38:37.419477+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:58.542524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,13],"acting":[8,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":4,"reported_epoch":28,"state":"creating+activating","last_fresh":"2024-04-26T03:38:37.423108+0000","last_change":"2024-04-26T03:38:37.423234+0000","last_active":"2024-04-26T03:38:35.319452+0000","last_peered":"2024-04-26T03:38:35.319452+0000","last_clean":"2024-04-26T03:38:35.319452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T03:38:37.423108+0000","last_undegraded":"2024-04-26T03:38:37.423108+0000","last_fullsized":"2024-04-26T03:38:37.423108+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4],"acting":[9,4],"avail_no_missing":["9","4"],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-26T03:38:38.184946+0000","last_change":"2024-04-26T03:38:37.255389+0000","last_active":"2024-04-26T03:38:38.184946+0000","last_peered":"2024-04-26T03:38:38.184946+0000","last_clean":"2024-04-26T03:38:38.184946+0000","last_became_active":"2024-04-26T03:38:37.255066+0000","last_became_peered":"2024-04-26T03:38:37.255066+0000","last_unstale":"2024-04-26T03:38:38.184946+0000","last_undegraded":"2024-04-26T03:38:38.184946+0000","last_fullsized":"2024-04-26T03:38:38.184946+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:54.145566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T03:38:38.834966+0000","last_change":"2024-04-26T03:38:36.447388+0000","last_active":"2024-04-26T03:38:38.834966+0000","last_peered":"2024-04-26T03:38:38.834966+0000","last_clean":"2024-04-26T03:38:38.834966+0000","last_became_active":"2024-04-26T03:38:36.447194+0000","last_became_peered":"2024-04-26T03:38:36.447194+0000","last_unstale":"2024-04-26T03:38:38.834966+0000","last_undegraded":"2024-04-26T03:38:38.834966+0000","last_fullsized":"2024-04-26T03:38:38.834966+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:42.465065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-26T03:38:37.715445+0000","last_change":"2024-04-26T03:38:37.255450+0000","last_active":"2024-04-26T03:38:37.715445+0000","last_peered":"2024-04-26T03:38:37.715445+0000","last_clean":"2024-04-26T03:38:37.715445+0000","last_became_active":"2024-04-26T03:38:37.255181+0000","last_became_peered":"2024-04-26T03:38:37.255181+0000","last_unstale":"2024-04-26T03:38:37.715445+0000","last_undegraded":"2024-04-26T03:38:37.715445+0000","last_fullsized":"2024-04-26T03:38:37.715445+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:40.592285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0],"acting":[14,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T03:38:38.834924+0000","last_change":"2024-04-26T03:38:36.431325+0000","last_active":"2024-04-26T03:38:38.834924+0000","last_peered":"2024-04-26T03:38:38.834924+0000","last_clean":"2024-04-26T03:38:38.834924+0000","last_became_active":"2024-04-26T03:38:36.431121+0000","last_became_peered":"2024-04-26T03:38:36.431121+0000","last_unstale":"2024-04-26T03:38:38.834924+0000","last_undegraded":"2024-04-26T03:38:38.834924+0000","last_fullsized":"2024-04-26T03:38:38.834924+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:53.294619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-04-26T03:38:37.715446+0000","last_change":"2024-04-26T03:38:36.431667+0000","last_active":"2024-04-26T03:38:37.715446+0000","last_peered":"2024-04-26T03:38:37.715446+0000","last_clean":"2024-04-26T03:38:37.715446+0000","last_became_active":"2024-04-26T03:38:36.431449+0000","last_became_peered":"2024-04-26T03:38:36.431449+0000","last_unstale":"2024-04-26T03:38:37.715446+0000","last_undegraded":"2024-04-26T03:38:37.715446+0000","last_fullsized":"2024-04-26T03:38:37.715446+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:54.705392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"27'157","reported_seq":211,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T03:38:36.433490+0000","last_change":"2024-04-26T03:38:34.390206+0000","last_active":"2024-04-26T03:38:36.433490+0000","last_peered":"2024-04-26T03:38:36.433490+0000","last_clean":"2024-04-26T03:38:36.433490+0000","last_became_active":"2024-04-26T03:38:34.389099+0000","last_became_peered":"2024-04-26T03:38:34.389099+0000","last_unstale":"2024-04-26T03:38:36.433490+0000","last_undegraded":"2024-04-26T03:38:36.433490+0000","last_fullsized":"2024-04-26T03:38:36.433490+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:32.306810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:32.306810+0000","last_clean_scrub_stamp":"2024-04-26T03:38:32.306810+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:40.216975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":11},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":13852,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":24,"seq":103079215110,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31524,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":319488,"data_stored":646165,"data_compressed":16279,"data_compressed_allocated":53248,"data_compressed_original":552960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[4,6,7,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215110,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":117237,"data_compressed":9353,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,4,5,7,8,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":23,"seq":98784247815,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31428,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340412,"statfs":{"total":96636764160,"available":96604581888,"internally_reserved":0,"allocated":253952,"data_stored":108105,"data_compressed":7460,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,6,7,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":307200,"data_stored":637033,"data_compressed":14386,"data_compressed_allocated":49152,"data_compressed_original":544768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,4,5,7,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27260,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":241664,"data_stored":98603,"data_compressed":5567,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[6,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":22,"seq":94489280517,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":108105,"data_compressed":7460,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[2,5,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":241664,"data_stored":98603,"data_compressed":5567,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,2,7,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":241664,"data_stored":98603,"data_compressed":5567,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[1,6,7,8,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27260,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":241664,"data_stored":98603,"data_compressed":5567,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,8,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":22,"seq":94489280515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":108105,"data_compressed":7460,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[6,7,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":108105,"data_compressed":7460,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[4,6,7,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247813,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":266240,"data_stored":117237,"data_compressed":9353,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,4,6,7,8,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":108105,"data_compressed":7460,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[3,6,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":23,"seq":98784247815,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31444,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":266240,"data_stored":117237,"data_compressed":9353,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,4,6,7,8,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":108105,"data_compressed":7460,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,4,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247813,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":117237,"data_compressed":9353,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,4,6,7,8,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T03:38:43.904 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T03:38:44.213 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:38:44.214 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-04-26T03:38:44.226 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-26T03:38:44.050705+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":12,"kb":1509949440,"kb_used":453496,"kb_used_data":4432,"kb_used_omap":24,"kb_used_meta":448743,"kb_avail":1509495944,"statfs":{"total":1546188226560,"available":1545723846656,"internally_reserved":0,"allocated":4538368,"data_stored":3066116,"data_compressed":189946,"data_compressed_allocated":450560,"data_compressed_original":1794048,"omap_allocated":25522,"internal_metadata":459512910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.632648"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T03:38:38.835823+0000","last_change":"2024-04-26T03:38:37.359252+0000","last_active":"2024-04-26T03:38:38.835823+0000","last_peered":"2024-04-26T03:38:38.835823+0000","last_clean":"2024-04-26T03:38:38.835823+0000","last_became_active":"2024-04-26T03:38:37.358910+0000","last_became_peered":"2024-04-26T03:38:37.358910+0000","last_unstale":"2024-04-26T03:38:38.835823+0000","last_undegraded":"2024-04-26T03:38:38.835823+0000","last_fullsized":"2024-04-26T03:38:38.835823+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:04.186001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T03:38:38.836268+0000","last_change":"2024-04-26T03:38:36.484782+0000","last_active":"2024-04-26T03:38:38.836268+0000","last_peered":"2024-04-26T03:38:38.836268+0000","last_clean":"2024-04-26T03:38:38.836268+0000","last_became_active":"2024-04-26T03:38:36.484388+0000","last_became_peered":"2024-04-26T03:38:36.484388+0000","last_unstale":"2024-04-26T03:38:38.836268+0000","last_undegraded":"2024-04-26T03:38:38.836268+0000","last_fullsized":"2024-04-26T03:38:38.836268+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:36.318287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,13],"acting":[8,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T03:38:38.835800+0000","last_change":"2024-04-26T03:38:38.187714+0000","last_active":"2024-04-26T03:38:38.835800+0000","last_peered":"2024-04-26T03:38:38.835800+0000","last_clean":"2024-04-26T03:38:38.835800+0000","last_became_active":"2024-04-26T03:38:38.187383+0000","last_became_peered":"2024-04-26T03:38:38.187383+0000","last_unstale":"2024-04-26T03:38:38.835800+0000","last_undegraded":"2024-04-26T03:38:38.835800+0000","last_fullsized":"2024-04-26T03:38:38.835800+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:59.062623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4],"acting":[9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T03:38:38.443172+0000","last_change":"2024-04-26T03:38:37.255389+0000","last_active":"2024-04-26T03:38:38.443172+0000","last_peered":"2024-04-26T03:38:38.443172+0000","last_clean":"2024-04-26T03:38:38.443172+0000","last_became_active":"2024-04-26T03:38:37.255066+0000","last_became_peered":"2024-04-26T03:38:37.255066+0000","last_unstale":"2024-04-26T03:38:38.443172+0000","last_undegraded":"2024-04-26T03:38:38.443172+0000","last_fullsized":"2024-04-26T03:38:38.443172+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:43.860480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T03:38:38.834966+0000","last_change":"2024-04-26T03:38:36.447388+0000","last_active":"2024-04-26T03:38:38.834966+0000","last_peered":"2024-04-26T03:38:38.834966+0000","last_clean":"2024-04-26T03:38:38.834966+0000","last_became_active":"2024-04-26T03:38:36.447194+0000","last_became_peered":"2024-04-26T03:38:36.447194+0000","last_unstale":"2024-04-26T03:38:38.834966+0000","last_undegraded":"2024-04-26T03:38:38.834966+0000","last_fullsized":"2024-04-26T03:38:38.834966+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:42.465065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T03:38:39.263216+0000","last_change":"2024-04-26T03:38:37.255450+0000","last_active":"2024-04-26T03:38:39.263216+0000","last_peered":"2024-04-26T03:38:39.263216+0000","last_clean":"2024-04-26T03:38:39.263216+0000","last_became_active":"2024-04-26T03:38:37.255181+0000","last_became_peered":"2024-04-26T03:38:37.255181+0000","last_unstale":"2024-04-26T03:38:39.263216+0000","last_undegraded":"2024-04-26T03:38:39.263216+0000","last_fullsized":"2024-04-26T03:38:39.263216+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:00.449388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0],"acting":[14,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T03:38:38.834924+0000","last_change":"2024-04-26T03:38:36.431325+0000","last_active":"2024-04-26T03:38:38.834924+0000","last_peered":"2024-04-26T03:38:38.834924+0000","last_clean":"2024-04-26T03:38:38.834924+0000","last_became_active":"2024-04-26T03:38:36.431121+0000","last_became_peered":"2024-04-26T03:38:36.431121+0000","last_unstale":"2024-04-26T03:38:38.834924+0000","last_undegraded":"2024-04-26T03:38:38.834924+0000","last_fullsized":"2024-04-26T03:38:38.834924+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:53.294619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T03:38:39.263219+0000","last_change":"2024-04-26T03:38:36.431667+0000","last_active":"2024-04-26T03:38:39.263219+0000","last_peered":"2024-04-26T03:38:39.263219+0000","last_clean":"2024-04-26T03:38:39.263219+0000","last_became_active":"2024-04-26T03:38:36.431449+0000","last_became_peered":"2024-04-26T03:38:36.431449+0000","last_unstale":"2024-04-26T03:38:39.263219+0000","last_undegraded":"2024-04-26T03:38:39.263219+0000","last_fullsized":"2024-04-26T03:38:39.263219+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:47.562648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"27'157","reported_seq":215,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T03:38:39.498056+0000","last_change":"2024-04-26T03:38:34.390206+0000","last_active":"2024-04-26T03:38:39.498056+0000","last_peered":"2024-04-26T03:38:39.498056+0000","last_clean":"2024-04-26T03:38:39.498056+0000","last_became_active":"2024-04-26T03:38:34.389099+0000","last_became_peered":"2024-04-26T03:38:34.389099+0000","last_unstale":"2024-04-26T03:38:39.498056+0000","last_undegraded":"2024-04-26T03:38:39.498056+0000","last_fullsized":"2024-04-26T03:38:39.498056+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:32.306810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:32.306810+0000","last_clean_scrub_stamp":"2024-04-26T03:38:32.306810+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:40.216975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":11},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":13852,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":24,"seq":103079215110,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31524,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":319488,"data_stored":646165,"data_compressed":16279,"data_compressed_allocated":53248,"data_compressed_original":552960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[4,6,7,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215110,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":117237,"data_compressed":9353,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,4,5,7,8,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":23,"seq":98784247816,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,7,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280517,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":331776,"data_stored":655627,"data_compressed":18168,"data_compressed_allocated":57344,"data_compressed_original":561152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,4,5,7,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[4,6,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":22,"seq":94489280518,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[2,5,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,2,4,7,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280517,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[3,6,7,8,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313222,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,8,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":22,"seq":94489280516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,7,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[4,6,7,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247814,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,4,6,7,8,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280516,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,3,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":23,"seq":98784247816,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31460,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,4,6,7,8,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280516,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,4,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247814,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T03:38:44.227 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-26T03:38:44.227 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-26T03:38:44.227 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-26T03:38:44.227 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T03:38:44.561 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:38:44.562 INFO:teuthology.orchestra.run.smithi028.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-26T03:38:44.575 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-26T03:38:44.575 INFO:teuthology.run_tasks:Running task thrashosds... 2024-04-26T03:38:44.586 INFO:tasks.thrashosds:config is {'aggressive_pg_num_changes': False, 'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'min_in': 8, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-04-26T03:38:44.586 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-yuri2-testing-2024-04-24-0914-squid'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 2500, 'ms tcp read timeout': 5}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED'], 'sha1': 'ddeaa91797339a19172c3036ff48cca58c12f448'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'ddeaa91797339a19172c3036ff48cca58c12f448'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri2-testing-2024-04-24-0914-squid', 'sha1': 'ddeaa91797339a19172c3036ff48cca58c12f448'}} 2024-04-26T03:38:44.586 INFO:tasks.thrashosds:config is {'aggressive_pg_num_changes': False, 'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'min_in': 8, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-04-26T03:38:44.586 INFO:tasks.thrashosds:Beginning thrashosds... 2024-04-26T03:38:44.587 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-26T03:38:44.587 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T03:38:44.879 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:38:44.879 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-04-26T03:38:44.891 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-26T03:38:44.050705+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":12,"kb":1509949440,"kb_used":453496,"kb_used_data":4432,"kb_used_omap":24,"kb_used_meta":448743,"kb_avail":1509495944,"statfs":{"total":1546188226560,"available":1545723846656,"internally_reserved":0,"allocated":4538368,"data_stored":3066116,"data_compressed":189946,"data_compressed_allocated":450560,"data_compressed_original":1794048,"omap_allocated":25522,"internal_metadata":459512910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.632648"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T03:38:38.835823+0000","last_change":"2024-04-26T03:38:37.359252+0000","last_active":"2024-04-26T03:38:38.835823+0000","last_peered":"2024-04-26T03:38:38.835823+0000","last_clean":"2024-04-26T03:38:38.835823+0000","last_became_active":"2024-04-26T03:38:37.358910+0000","last_became_peered":"2024-04-26T03:38:37.358910+0000","last_unstale":"2024-04-26T03:38:38.835823+0000","last_undegraded":"2024-04-26T03:38:38.835823+0000","last_fullsized":"2024-04-26T03:38:38.835823+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:04.186001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T03:38:38.836268+0000","last_change":"2024-04-26T03:38:36.484782+0000","last_active":"2024-04-26T03:38:38.836268+0000","last_peered":"2024-04-26T03:38:38.836268+0000","last_clean":"2024-04-26T03:38:38.836268+0000","last_became_active":"2024-04-26T03:38:36.484388+0000","last_became_peered":"2024-04-26T03:38:36.484388+0000","last_unstale":"2024-04-26T03:38:38.836268+0000","last_undegraded":"2024-04-26T03:38:38.836268+0000","last_fullsized":"2024-04-26T03:38:38.836268+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:36.318287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,13],"acting":[8,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T03:38:38.835800+0000","last_change":"2024-04-26T03:38:38.187714+0000","last_active":"2024-04-26T03:38:38.835800+0000","last_peered":"2024-04-26T03:38:38.835800+0000","last_clean":"2024-04-26T03:38:38.835800+0000","last_became_active":"2024-04-26T03:38:38.187383+0000","last_became_peered":"2024-04-26T03:38:38.187383+0000","last_unstale":"2024-04-26T03:38:38.835800+0000","last_undegraded":"2024-04-26T03:38:38.835800+0000","last_fullsized":"2024-04-26T03:38:38.835800+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:59.062623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4],"acting":[9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T03:38:38.443172+0000","last_change":"2024-04-26T03:38:37.255389+0000","last_active":"2024-04-26T03:38:38.443172+0000","last_peered":"2024-04-26T03:38:38.443172+0000","last_clean":"2024-04-26T03:38:38.443172+0000","last_became_active":"2024-04-26T03:38:37.255066+0000","last_became_peered":"2024-04-26T03:38:37.255066+0000","last_unstale":"2024-04-26T03:38:38.443172+0000","last_undegraded":"2024-04-26T03:38:38.443172+0000","last_fullsized":"2024-04-26T03:38:38.443172+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:43.860480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T03:38:38.834966+0000","last_change":"2024-04-26T03:38:36.447388+0000","last_active":"2024-04-26T03:38:38.834966+0000","last_peered":"2024-04-26T03:38:38.834966+0000","last_clean":"2024-04-26T03:38:38.834966+0000","last_became_active":"2024-04-26T03:38:36.447194+0000","last_became_peered":"2024-04-26T03:38:36.447194+0000","last_unstale":"2024-04-26T03:38:38.834966+0000","last_undegraded":"2024-04-26T03:38:38.834966+0000","last_fullsized":"2024-04-26T03:38:38.834966+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:42.465065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T03:38:39.263216+0000","last_change":"2024-04-26T03:38:37.255450+0000","last_active":"2024-04-26T03:38:39.263216+0000","last_peered":"2024-04-26T03:38:39.263216+0000","last_clean":"2024-04-26T03:38:39.263216+0000","last_became_active":"2024-04-26T03:38:37.255181+0000","last_became_peered":"2024-04-26T03:38:37.255181+0000","last_unstale":"2024-04-26T03:38:39.263216+0000","last_undegraded":"2024-04-26T03:38:39.263216+0000","last_fullsized":"2024-04-26T03:38:39.263216+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:00.449388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0],"acting":[14,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T03:38:38.834924+0000","last_change":"2024-04-26T03:38:36.431325+0000","last_active":"2024-04-26T03:38:38.834924+0000","last_peered":"2024-04-26T03:38:38.834924+0000","last_clean":"2024-04-26T03:38:38.834924+0000","last_became_active":"2024-04-26T03:38:36.431121+0000","last_became_peered":"2024-04-26T03:38:36.431121+0000","last_unstale":"2024-04-26T03:38:38.834924+0000","last_undegraded":"2024-04-26T03:38:38.834924+0000","last_fullsized":"2024-04-26T03:38:38.834924+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:53.294619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T03:38:39.263219+0000","last_change":"2024-04-26T03:38:36.431667+0000","last_active":"2024-04-26T03:38:39.263219+0000","last_peered":"2024-04-26T03:38:39.263219+0000","last_clean":"2024-04-26T03:38:39.263219+0000","last_became_active":"2024-04-26T03:38:36.431449+0000","last_became_peered":"2024-04-26T03:38:36.431449+0000","last_unstale":"2024-04-26T03:38:39.263219+0000","last_undegraded":"2024-04-26T03:38:39.263219+0000","last_fullsized":"2024-04-26T03:38:39.263219+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:47.562648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"27'157","reported_seq":215,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T03:38:39.498056+0000","last_change":"2024-04-26T03:38:34.390206+0000","last_active":"2024-04-26T03:38:39.498056+0000","last_peered":"2024-04-26T03:38:39.498056+0000","last_clean":"2024-04-26T03:38:39.498056+0000","last_became_active":"2024-04-26T03:38:34.389099+0000","last_became_peered":"2024-04-26T03:38:34.389099+0000","last_unstale":"2024-04-26T03:38:39.498056+0000","last_undegraded":"2024-04-26T03:38:39.498056+0000","last_fullsized":"2024-04-26T03:38:39.498056+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:32.306810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:32.306810+0000","last_clean_scrub_stamp":"2024-04-26T03:38:32.306810+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:40.216975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":11},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":13852,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":24,"seq":103079215110,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31524,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":319488,"data_stored":646165,"data_compressed":16279,"data_compressed_allocated":53248,"data_compressed_original":552960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[4,6,7,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215110,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":117237,"data_compressed":9353,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,4,5,7,8,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":23,"seq":98784247816,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,7,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280517,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":331776,"data_stored":655627,"data_compressed":18168,"data_compressed_allocated":57344,"data_compressed_original":561152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,4,5,7,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[4,6,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":22,"seq":94489280518,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[2,5,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,2,4,7,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280517,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[3,6,7,8,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313222,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,8,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":22,"seq":94489280516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,7,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[4,6,7,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247814,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,4,6,7,8,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280516,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,3,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":23,"seq":98784247816,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31460,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,4,6,7,8,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280516,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,4,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247814,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T03:38:44.892 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T03:38:45.186 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:38:45.187 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-04-26T03:38:45.198 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-26T03:38:44.050705+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":12,"kb":1509949440,"kb_used":453496,"kb_used_data":4432,"kb_used_omap":24,"kb_used_meta":448743,"kb_avail":1509495944,"statfs":{"total":1546188226560,"available":1545723846656,"internally_reserved":0,"allocated":4538368,"data_stored":3066116,"data_compressed":189946,"data_compressed_allocated":450560,"data_compressed_original":1794048,"omap_allocated":25522,"internal_metadata":459512910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.632648"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T03:38:38.835823+0000","last_change":"2024-04-26T03:38:37.359252+0000","last_active":"2024-04-26T03:38:38.835823+0000","last_peered":"2024-04-26T03:38:38.835823+0000","last_clean":"2024-04-26T03:38:38.835823+0000","last_became_active":"2024-04-26T03:38:37.358910+0000","last_became_peered":"2024-04-26T03:38:37.358910+0000","last_unstale":"2024-04-26T03:38:38.835823+0000","last_undegraded":"2024-04-26T03:38:38.835823+0000","last_fullsized":"2024-04-26T03:38:38.835823+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:04.186001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T03:38:38.836268+0000","last_change":"2024-04-26T03:38:36.484782+0000","last_active":"2024-04-26T03:38:38.836268+0000","last_peered":"2024-04-26T03:38:38.836268+0000","last_clean":"2024-04-26T03:38:38.836268+0000","last_became_active":"2024-04-26T03:38:36.484388+0000","last_became_peered":"2024-04-26T03:38:36.484388+0000","last_unstale":"2024-04-26T03:38:38.836268+0000","last_undegraded":"2024-04-26T03:38:38.836268+0000","last_fullsized":"2024-04-26T03:38:38.836268+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:36.318287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,13],"acting":[8,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T03:38:38.835800+0000","last_change":"2024-04-26T03:38:38.187714+0000","last_active":"2024-04-26T03:38:38.835800+0000","last_peered":"2024-04-26T03:38:38.835800+0000","last_clean":"2024-04-26T03:38:38.835800+0000","last_became_active":"2024-04-26T03:38:38.187383+0000","last_became_peered":"2024-04-26T03:38:38.187383+0000","last_unstale":"2024-04-26T03:38:38.835800+0000","last_undegraded":"2024-04-26T03:38:38.835800+0000","last_fullsized":"2024-04-26T03:38:38.835800+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:59.062623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4],"acting":[9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T03:38:38.443172+0000","last_change":"2024-04-26T03:38:37.255389+0000","last_active":"2024-04-26T03:38:38.443172+0000","last_peered":"2024-04-26T03:38:38.443172+0000","last_clean":"2024-04-26T03:38:38.443172+0000","last_became_active":"2024-04-26T03:38:37.255066+0000","last_became_peered":"2024-04-26T03:38:37.255066+0000","last_unstale":"2024-04-26T03:38:38.443172+0000","last_undegraded":"2024-04-26T03:38:38.443172+0000","last_fullsized":"2024-04-26T03:38:38.443172+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:43.860480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T03:38:38.834966+0000","last_change":"2024-04-26T03:38:36.447388+0000","last_active":"2024-04-26T03:38:38.834966+0000","last_peered":"2024-04-26T03:38:38.834966+0000","last_clean":"2024-04-26T03:38:38.834966+0000","last_became_active":"2024-04-26T03:38:36.447194+0000","last_became_peered":"2024-04-26T03:38:36.447194+0000","last_unstale":"2024-04-26T03:38:38.834966+0000","last_undegraded":"2024-04-26T03:38:38.834966+0000","last_fullsized":"2024-04-26T03:38:38.834966+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:42.465065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T03:38:39.263216+0000","last_change":"2024-04-26T03:38:37.255450+0000","last_active":"2024-04-26T03:38:39.263216+0000","last_peered":"2024-04-26T03:38:39.263216+0000","last_clean":"2024-04-26T03:38:39.263216+0000","last_became_active":"2024-04-26T03:38:37.255181+0000","last_became_peered":"2024-04-26T03:38:37.255181+0000","last_unstale":"2024-04-26T03:38:39.263216+0000","last_undegraded":"2024-04-26T03:38:39.263216+0000","last_fullsized":"2024-04-26T03:38:39.263216+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:00.449388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0],"acting":[14,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T03:38:38.834924+0000","last_change":"2024-04-26T03:38:36.431325+0000","last_active":"2024-04-26T03:38:38.834924+0000","last_peered":"2024-04-26T03:38:38.834924+0000","last_clean":"2024-04-26T03:38:38.834924+0000","last_became_active":"2024-04-26T03:38:36.431121+0000","last_became_peered":"2024-04-26T03:38:36.431121+0000","last_unstale":"2024-04-26T03:38:38.834924+0000","last_undegraded":"2024-04-26T03:38:38.834924+0000","last_fullsized":"2024-04-26T03:38:38.834924+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:53.294619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T03:38:39.263219+0000","last_change":"2024-04-26T03:38:36.431667+0000","last_active":"2024-04-26T03:38:39.263219+0000","last_peered":"2024-04-26T03:38:39.263219+0000","last_clean":"2024-04-26T03:38:39.263219+0000","last_became_active":"2024-04-26T03:38:36.431449+0000","last_became_peered":"2024-04-26T03:38:36.431449+0000","last_unstale":"2024-04-26T03:38:39.263219+0000","last_undegraded":"2024-04-26T03:38:39.263219+0000","last_fullsized":"2024-04-26T03:38:39.263219+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:47.562648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"27'157","reported_seq":215,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T03:38:39.498056+0000","last_change":"2024-04-26T03:38:34.390206+0000","last_active":"2024-04-26T03:38:39.498056+0000","last_peered":"2024-04-26T03:38:39.498056+0000","last_clean":"2024-04-26T03:38:39.498056+0000","last_became_active":"2024-04-26T03:38:34.389099+0000","last_became_peered":"2024-04-26T03:38:34.389099+0000","last_unstale":"2024-04-26T03:38:39.498056+0000","last_undegraded":"2024-04-26T03:38:39.498056+0000","last_fullsized":"2024-04-26T03:38:39.498056+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:32.306810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:32.306810+0000","last_clean_scrub_stamp":"2024-04-26T03:38:32.306810+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:40.216975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":11},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":13852,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":24,"seq":103079215110,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31524,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":319488,"data_stored":646165,"data_compressed":16279,"data_compressed_allocated":53248,"data_compressed_original":552960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[4,6,7,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215110,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":117237,"data_compressed":9353,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,4,5,7,8,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":23,"seq":98784247816,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,7,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280517,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":331776,"data_stored":655627,"data_compressed":18168,"data_compressed_allocated":57344,"data_compressed_original":561152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,4,5,7,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[4,6,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":22,"seq":94489280518,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[2,5,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,2,4,7,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280517,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[3,6,7,8,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313222,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,8,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":22,"seq":94489280516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,7,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[4,6,7,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247814,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,4,6,7,8,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280516,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,3,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":23,"seq":98784247816,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31460,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,4,6,7,8,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280516,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,4,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247814,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T03:38:45.199 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-26T03:38:45.199 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-04-26T03:38:45.497 INFO:teuthology.orchestra.run.smithi028.stdout:epoch 29 2024-04-26T03:38:45.497 INFO:teuthology.orchestra.run.smithi028.stdout:fsid c5a13c99-5c9a-4b8b-a893-4f6f4211677a 2024-04-26T03:38:45.497 INFO:teuthology.orchestra.run.smithi028.stdout:created 2024-04-26T03:38:19.896648+0000 2024-04-26T03:38:45.497 INFO:teuthology.orchestra.run.smithi028.stdout:modified 2024-04-26T03:38:38.431503+0000 2024-04-26T03:38:45.497 INFO:teuthology.orchestra.run.smithi028.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-26T03:38:45.497 INFO:teuthology.orchestra.run.smithi028.stdout:crush_version 5 2024-04-26T03:38:45.497 INFO:teuthology.orchestra.run.smithi028.stdout:full_ratio 0.95 2024-04-26T03:38:45.497 INFO:teuthology.orchestra.run.smithi028.stdout:backfillfull_ratio 0.9 2024-04-26T03:38:45.497 INFO:teuthology.orchestra.run.smithi028.stdout:nearfull_ratio 0.85 2024-04-26T03:38:45.498 INFO:teuthology.orchestra.run.smithi028.stdout:require_min_compat_client luminous 2024-04-26T03:38:45.498 INFO:teuthology.orchestra.run.smithi028.stdout:min_compat_client jewel 2024-04-26T03:38:45.498 INFO:teuthology.orchestra.run.smithi028.stdout:require_osd_release squid 2024-04-26T03:38:45.498 INFO:teuthology.orchestra.run.smithi028.stdout:stretch_mode_enabled false 2024-04-26T03:38:45.498 INFO:teuthology.orchestra.run.smithi028.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 26 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 15.38 2024-04-26T03:38:45.498 INFO:teuthology.orchestra.run.smithi028.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 29 flags hashpspool stripe_width 0 application rbd read_balance_score 3.99 2024-04-26T03:38:45.498 INFO:teuthology.orchestra.run.smithi028.stdout:max_osd 16 2024-04-26T03:38:45.498 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0 up in weight 1 up_from 23 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.28:6800/2377656090,v1:172.21.15.28:6801/2377656090] [v2:172.21.15.28:6802/2377656090,v1:172.21.15.28:6803/2377656090] exists,up 31b0e7a9-455d-4874-9c60-26efac8d0b1e 2024-04-26T03:38:45.498 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1 up in weight 1 up_from 23 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6800/1678150206,v1:172.21.15.42:6801/1678150206] [v2:172.21.15.42:6802/1678150206,v1:172.21.15.42:6803/1678150206] exists,up 52819ace-a070-4822-b33e-3c54466e61ff 2024-04-26T03:38:45.498 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2 up in weight 1 up_from 23 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6800/714535415,v1:172.21.15.66:6801/714535415] [v2:172.21.15.66:6802/714535415,v1:172.21.15.66:6803/714535415] exists,up 858f8ac1-2edc-43d5-914c-c1914bdc0c9a 2024-04-26T03:38:45.498 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3 up in weight 1 up_from 24 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6810/2377983625,v1:172.21.15.92:6811/2377983625] [v2:172.21.15.92:6812/2377983625,v1:172.21.15.92:6813/2377983625] exists,up 531a2e30-3351-4f84-804b-5fb5050b84ed 2024-04-26T03:38:45.498 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4 up in weight 1 up_from 23 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:172.21.15.28:6808/183752455,v1:172.21.15.28:6809/183752455] [v2:172.21.15.28:6810/183752455,v1:172.21.15.28:6811/183752455] exists,up 6d38ffda-40eb-4fd7-9300-2a09f2673e66 2024-04-26T03:38:45.499 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5 up in weight 1 up_from 24 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6808/2012261939,v1:172.21.15.42:6809/2012261939] [v2:172.21.15.42:6810/2012261939,v1:172.21.15.42:6811/2012261939] exists,up 361aa40d-e53b-42af-afd4-81fbcac28725 2024-04-26T03:38:45.499 INFO:teuthology.orchestra.run.smithi028.stdout:osd.6 up in weight 1 up_from 22 up_thru 24 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6808/3533460882,v1:172.21.15.66:6809/3533460882] [v2:172.21.15.66:6810/3533460882,v1:172.21.15.66:6811/3533460882] exists,up f610ad3e-1fd8-42bd-af4b-0d79c3bb9cf8 2024-04-26T03:38:45.499 INFO:teuthology.orchestra.run.smithi028.stdout:osd.7 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6802/1760163933,v1:172.21.15.92:6803/1760163933] [v2:172.21.15.92:6804/1760163933,v1:172.21.15.92:6805/1760163933] exists,up a4bae6fb-e087-4563-a129-6938e18eed8c 2024-04-26T03:38:45.499 INFO:teuthology.orchestra.run.smithi028.stdout:osd.8 up in weight 1 up_from 22 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:172.21.15.28:6816/4127602407,v1:172.21.15.28:6817/4127602407] [v2:172.21.15.28:6818/4127602407,v1:172.21.15.28:6819/4127602407] exists,up 479f3885-452e-4231-bdc2-b79668384ba6 2024-04-26T03:38:45.499 INFO:teuthology.orchestra.run.smithi028.stdout:osd.9 up in weight 1 up_from 22 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6816/4168729845,v1:172.21.15.42:6818/4168729845] [v2:172.21.15.42:6820/4168729845,v1:172.21.15.42:6822/4168729845] exists,up 9a603a47-7c5a-47d4-a298-56b3ac991892 2024-04-26T03:38:45.499 INFO:teuthology.orchestra.run.smithi028.stdout:osd.10 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6816/4180572718,v1:172.21.15.66:6817/4180572718] [v2:172.21.15.66:6818/4180572718,v1:172.21.15.66:6819/4180572718] exists,up 1cd42945-6ee1-49f0-b17b-4093dd36b4ec 2024-04-26T03:38:45.499 INFO:teuthology.orchestra.run.smithi028.stdout:osd.11 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6818/295713815,v1:172.21.15.92:6819/295713815] [v2:172.21.15.92:6820/295713815,v1:172.21.15.92:6821/295713815] exists,up b41e8100-1d4f-4b7a-991a-7a27bc29feec 2024-04-26T03:38:45.499 INFO:teuthology.orchestra.run.smithi028.stdout:osd.12 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.28:6824/2081141953,v1:172.21.15.28:6825/2081141953] [v2:172.21.15.28:6826/2081141953,v1:172.21.15.28:6827/2081141953] exists,up 458cf150-32e1-449e-b533-ac6ab5226f85 2024-04-26T03:38:45.499 INFO:teuthology.orchestra.run.smithi028.stdout:osd.13 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6817/301360893,v1:172.21.15.42:6819/301360893] [v2:172.21.15.42:6821/301360893,v1:172.21.15.42:6823/301360893] exists,up f008462e-a735-4845-9ecf-dd7d4a6941af 2024-04-26T03:38:45.499 INFO:teuthology.orchestra.run.smithi028.stdout:osd.14 up in weight 1 up_from 22 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6824/1534905602,v1:172.21.15.66:6825/1534905602] [v2:172.21.15.66:6826/1534905602,v1:172.21.15.66:6827/1534905602] exists,up 6f66ae97-eb8a-48eb-9bcc-20f65fbf690b 2024-04-26T03:38:45.499 INFO:teuthology.orchestra.run.smithi028.stdout:osd.15 up in weight 1 up_from 22 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6826/3982634323,v1:172.21.15.92:6827/3982634323] [v2:172.21.15.92:6828/3982634323,v1:172.21.15.92:6829/3982634323] exists,up b856bfc8-253b-4112-be5c-f67d61d01768 2024-04-26T03:38:45.509 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 23 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.28:6800/2377656090,v1:172.21.15.28:6801/2377656090] [v2:172.21.15.28:6802/2377656090,v1:172.21.15.28:6803/2377656090] exists,up 31b0e7a9-455d-4874-9c60-26efac8d0b1e', 'osd.1 up in weight 1 up_from 23 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6800/1678150206,v1:172.21.15.42:6801/1678150206] [v2:172.21.15.42:6802/1678150206,v1:172.21.15.42:6803/1678150206] exists,up 52819ace-a070-4822-b33e-3c54466e61ff', 'osd.2 up in weight 1 up_from 23 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6800/714535415,v1:172.21.15.66:6801/714535415] [v2:172.21.15.66:6802/714535415,v1:172.21.15.66:6803/714535415] exists,up 858f8ac1-2edc-43d5-914c-c1914bdc0c9a', 'osd.3 up in weight 1 up_from 24 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6810/2377983625,v1:172.21.15.92:6811/2377983625] [v2:172.21.15.92:6812/2377983625,v1:172.21.15.92:6813/2377983625] exists,up 531a2e30-3351-4f84-804b-5fb5050b84ed', 'osd.4 up in weight 1 up_from 23 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:172.21.15.28:6808/183752455,v1:172.21.15.28:6809/183752455] [v2:172.21.15.28:6810/183752455,v1:172.21.15.28:6811/183752455] exists,up 6d38ffda-40eb-4fd7-9300-2a09f2673e66', 'osd.5 up in weight 1 up_from 24 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6808/2012261939,v1:172.21.15.42:6809/2012261939] [v2:172.21.15.42:6810/2012261939,v1:172.21.15.42:6811/2012261939] exists,up 361aa40d-e53b-42af-afd4-81fbcac28725', 'osd.6 up in weight 1 up_from 22 up_thru 24 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6808/3533460882,v1:172.21.15.66:6809/3533460882] [v2:172.21.15.66:6810/3533460882,v1:172.21.15.66:6811/3533460882] exists,up f610ad3e-1fd8-42bd-af4b-0d79c3bb9cf8', 'osd.7 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6802/1760163933,v1:172.21.15.92:6803/1760163933] [v2:172.21.15.92:6804/1760163933,v1:172.21.15.92:6805/1760163933] exists,up a4bae6fb-e087-4563-a129-6938e18eed8c', 'osd.8 up in weight 1 up_from 22 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:172.21.15.28:6816/4127602407,v1:172.21.15.28:6817/4127602407] [v2:172.21.15.28:6818/4127602407,v1:172.21.15.28:6819/4127602407] exists,up 479f3885-452e-4231-bdc2-b79668384ba6', 'osd.9 up in weight 1 up_from 22 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6816/4168729845,v1:172.21.15.42:6818/4168729845] [v2:172.21.15.42:6820/4168729845,v1:172.21.15.42:6822/4168729845] exists,up 9a603a47-7c5a-47d4-a298-56b3ac991892', 'osd.10 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6816/4180572718,v1:172.21.15.66:6817/4180572718] [v2:172.21.15.66:6818/4180572718,v1:172.21.15.66:6819/4180572718] exists,up 1cd42945-6ee1-49f0-b17b-4093dd36b4ec', 'osd.11 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6818/295713815,v1:172.21.15.92:6819/295713815] [v2:172.21.15.92:6820/295713815,v1:172.21.15.92:6821/295713815] exists,up b41e8100-1d4f-4b7a-991a-7a27bc29feec', 'osd.12 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.28:6824/2081141953,v1:172.21.15.28:6825/2081141953] [v2:172.21.15.28:6826/2081141953,v1:172.21.15.28:6827/2081141953] exists,up 458cf150-32e1-449e-b533-ac6ab5226f85', 'osd.13 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6817/301360893,v1:172.21.15.42:6819/301360893] [v2:172.21.15.42:6821/301360893,v1:172.21.15.42:6823/301360893] exists,up f008462e-a735-4845-9ecf-dd7d4a6941af', 'osd.14 up in weight 1 up_from 22 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6824/1534905602,v1:172.21.15.66:6825/1534905602] [v2:172.21.15.66:6826/1534905602,v1:172.21.15.66:6827/1534905602] exists,up 6f66ae97-eb8a-48eb-9bcc-20f65fbf690b', 'osd.15 up in weight 1 up_from 22 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6826/3982634323,v1:172.21.15.92:6827/3982634323] [v2:172.21.15.92:6828/3982634323,v1:172.21.15.92:6829/3982634323] exists,up b856bfc8-253b-4112-be5c-f67d61d01768'] 2024-04-26T03:38:45.510 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-04-26T03:38:45.611 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:38:45.611 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "mon.a", 2024-04-26T03:38:45.611 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster": "ceph", 2024-04-26T03:38:45.611 INFO:teuthology.orchestra.run.smithi028.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-04-26T03:38:45.611 INFO:teuthology.orchestra.run.smithi028.stdout: "admin_socket_mode": "", 2024-04-26T03:38:45.612 INFO:teuthology.orchestra.run.smithi028.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-04-26T03:38:45.612 INFO:teuthology.orchestra.run.smithi028.stdout: "auth_client_required": "cephx, none", 2024-04-26T03:38:45.612 INFO:teuthology.orchestra.run.smithi028.stdout: "auth_cluster_required": "cephx", 2024-04-26T03:38:45.612 INFO:teuthology.orchestra.run.smithi028.stdout: "auth_debug": "true", 2024-04-26T03:38:45.612 INFO:teuthology.orchestra.run.smithi028.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-04-26T03:38:45.613 INFO:teuthology.orchestra.run.smithi028.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-04-26T03:38:45.613 INFO:teuthology.orchestra.run.smithi028.stdout: "auth_service_required": "cephx", 2024-04-26T03:38:45.614 INFO:teuthology.orchestra.run.smithi028.stdout: "auth_service_ticket_ttl": "240.000000", 2024-04-26T03:38:45.614 INFO:teuthology.orchestra.run.smithi028.stdout: "auth_supported": "cephx", 2024-04-26T03:38:45.614 INFO:teuthology.orchestra.run.smithi028.stdout: "bdev_aio": "true", 2024-04-26T03:38:45.614 INFO:teuthology.orchestra.run.smithi028.stdout: "bdev_aio_max_queue_depth": "1024", 2024-04-26T03:38:45.614 INFO:teuthology.orchestra.run.smithi028.stdout: "bdev_aio_poll_ms": "250", 2024-04-26T03:38:45.614 INFO:teuthology.orchestra.run.smithi028.stdout: "bdev_aio_reap_max": "16", 2024-04-26T03:38:45.614 INFO:teuthology.orchestra.run.smithi028.stdout: "bdev_async_discard": "false", 2024-04-26T03:38:45.614 INFO:teuthology.orchestra.run.smithi028.stdout: "bdev_block_size": "4096", 2024-04-26T03:38:45.614 INFO:teuthology.orchestra.run.smithi028.stdout: "bdev_debug_aio": "false", 2024-04-26T03:38:45.614 INFO:teuthology.orchestra.run.smithi028.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-04-26T03:38:45.614 INFO:teuthology.orchestra.run.smithi028.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-04-26T03:38:45.614 INFO:teuthology.orchestra.run.smithi028.stdout: "bdev_debug_inflight_ios": "false", 2024-04-26T03:38:45.615 INFO:teuthology.orchestra.run.smithi028.stdout: "bdev_enable_discard": "false", 2024-04-26T03:38:45.616 INFO:teuthology.orchestra.run.smithi028.stdout: "bdev_flock_retry": "3", 2024-04-26T03:38:45.616 INFO:teuthology.orchestra.run.smithi028.stdout: "bdev_flock_retry_interval": "0.100000", 2024-04-26T03:38:45.616 INFO:teuthology.orchestra.run.smithi028.stdout: "bdev_inject_crash": "0", 2024-04-26T03:38:45.616 INFO:teuthology.orchestra.run.smithi028.stdout: "bdev_inject_crash_flush_delay": "2", 2024-04-26T03:38:45.616 INFO:teuthology.orchestra.run.smithi028.stdout: "bdev_ioring": "false", 2024-04-26T03:38:45.616 INFO:teuthology.orchestra.run.smithi028.stdout: "bdev_ioring_hipri": "false", 2024-04-26T03:38:45.616 INFO:teuthology.orchestra.run.smithi028.stdout: "bdev_ioring_sqthread_poll": "false", 2024-04-26T03:38:45.616 INFO:teuthology.orchestra.run.smithi028.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-04-26T03:38:45.616 INFO:teuthology.orchestra.run.smithi028.stdout: "bdev_read_buffer_alignment": "4096", 2024-04-26T03:38:45.616 INFO:teuthology.orchestra.run.smithi028.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-04-26T03:38:45.616 INFO:teuthology.orchestra.run.smithi028.stdout: "bdev_type": "", 2024-04-26T03:38:45.617 INFO:teuthology.orchestra.run.smithi028.stdout: "bluefs_alloc_size": "1048576", 2024-04-26T03:38:45.617 INFO:teuthology.orchestra.run.smithi028.stdout: "bluefs_allocator": "hybrid", 2024-04-26T03:38:45.617 INFO:teuthology.orchestra.run.smithi028.stdout: "bluefs_buffered_io": "true", 2024-04-26T03:38:45.617 INFO:teuthology.orchestra.run.smithi028.stdout: "bluefs_check_for_zeros": "false", 2024-04-26T03:38:45.617 INFO:teuthology.orchestra.run.smithi028.stdout: "bluefs_check_volume_selector_often": "false", 2024-04-26T03:38:45.617 INFO:teuthology.orchestra.run.smithi028.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-04-26T03:38:45.617 INFO:teuthology.orchestra.run.smithi028.stdout: "bluefs_compact_log_sync": "false", 2024-04-26T03:38:45.617 INFO:teuthology.orchestra.run.smithi028.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-04-26T03:38:45.617 INFO:teuthology.orchestra.run.smithi028.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-04-26T03:38:45.617 INFO:teuthology.orchestra.run.smithi028.stdout: "bluefs_log_compact_min_size": "16777216", 2024-04-26T03:38:45.617 INFO:teuthology.orchestra.run.smithi028.stdout: "bluefs_log_replay_check_allocations": "true", 2024-04-26T03:38:45.617 INFO:teuthology.orchestra.run.smithi028.stdout: "bluefs_max_log_runway": "4194304", 2024-04-26T03:38:45.617 INFO:teuthology.orchestra.run.smithi028.stdout: "bluefs_max_prefetch": "1048576", 2024-04-26T03:38:45.618 INFO:teuthology.orchestra.run.smithi028.stdout: "bluefs_min_flush_size": "524288", 2024-04-26T03:38:45.618 INFO:teuthology.orchestra.run.smithi028.stdout: "bluefs_min_log_runway": "1048576", 2024-04-26T03:38:45.618 INFO:teuthology.orchestra.run.smithi028.stdout: "bluefs_replay_recovery": "false", 2024-04-26T03:38:45.618 INFO:teuthology.orchestra.run.smithi028.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-04-26T03:38:45.618 INFO:teuthology.orchestra.run.smithi028.stdout: "bluefs_shared_alloc_size": "65536", 2024-04-26T03:38:45.618 INFO:teuthology.orchestra.run.smithi028.stdout: "bluefs_sync_write": "false", 2024-04-26T03:38:45.618 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-04-26T03:38:45.618 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-04-26T03:38:45.618 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-04-26T03:38:45.618 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_allocation_from_file": "true", 2024-04-26T03:38:45.618 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_allocator": "hybrid", 2024-04-26T03:38:45.618 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-04-26T03:38:45.619 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-04-26T03:38:45.619 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-04-26T03:38:45.619 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-04-26T03:38:45.619 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-04-26T03:38:45.619 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-04-26T03:38:45.619 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_blobid_prealloc": "10240", 2024-04-26T03:38:45.619 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_block_create": "true", 2024-04-26T03:38:45.619 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_block_db_create": "false", 2024-04-26T03:38:45.619 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_block_db_path": "", 2024-04-26T03:38:45.619 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_block_db_size": "0", 2024-04-26T03:38:45.619 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_block_path": "", 2024-04-26T03:38:45.620 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_block_preallocate_file": "false", 2024-04-26T03:38:45.620 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_block_size": "107374182400", 2024-04-26T03:38:45.620 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_block_wal_create": "false", 2024-04-26T03:38:45.620 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_block_wal_path": "", 2024-04-26T03:38:45.620 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_block_wal_size": "100663296", 2024-04-26T03:38:45.620 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_bluefs": "true", 2024-04-26T03:38:45.620 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-04-26T03:38:45.620 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_bluefs_env_mirror": "false", 2024-04-26T03:38:45.620 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-04-26T03:38:45.620 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-04-26T03:38:45.620 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-04-26T03:38:45.620 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-04-26T03:38:45.620 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-04-26T03:38:45.621 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-04-26T03:38:45.621 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_autotune": "true", 2024-04-26T03:38:45.621 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-04-26T03:38:45.621 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-04-26T03:38:45.621 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-04-26T03:38:45.621 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-04-26T03:38:45.621 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_size": "0", 2024-04-26T03:38:45.621 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-04-26T03:38:45.621 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-04-26T03:38:45.621 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-04-26T03:38:45.621 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-04-26T03:38:45.622 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cache_type": "2q", 2024-04-26T03:38:45.622 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-04-26T03:38:45.622 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_clone_cow": "true", 2024-04-26T03:38:45.622 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_compression_algorithm": "snappy", 2024-04-26T03:38:45.622 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_compression_max_blob_size": "0", 2024-04-26T03:38:45.622 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-04-26T03:38:45.622 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-04-26T03:38:45.622 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_compression_min_blob_size": "0", 2024-04-26T03:38:45.622 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-04-26T03:38:45.622 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-04-26T03:38:45.622 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_compression_mode": "none", 2024-04-26T03:38:45.622 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-04-26T03:38:45.623 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_csum_type": "crc32c", 2024-04-26T03:38:45.623 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_debug_enforce_settings": "default", 2024-04-26T03:38:45.623 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_debug_freelist": "false", 2024-04-26T03:38:45.623 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_debug_fsck_abort": "false", 2024-04-26T03:38:45.623 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-04-26T03:38:45.623 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-04-26T03:38:45.623 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_debug_inject_read_err": "false", 2024-04-26T03:38:45.623 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_debug_legacy_omap": "false", 2024-04-26T03:38:45.623 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-04-26T03:38:45.623 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-04-26T03:38:45.623 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-04-26T03:38:45.624 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-04-26T03:38:45.624 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_debug_prefill": "0.000000", 2024-04-26T03:38:45.624 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-04-26T03:38:45.624 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-04-26T03:38:45.624 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-04-26T03:38:45.624 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_debug_small_allocations": "0", 2024-04-26T03:38:45.625 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-04-26T03:38:45.625 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_default_buffered_read": "true", 2024-04-26T03:38:45.625 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_default_buffered_write": "false", 2024-04-26T03:38:45.625 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_deferred_batch_ops": "0", 2024-04-26T03:38:45.625 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-04-26T03:38:45.625 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-04-26T03:38:45.625 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_elastic_shared_blobs": "true", 2024-04-26T03:38:45.625 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-04-26T03:38:45.625 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-04-26T03:38:45.625 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-04-26T03:38:45.625 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-04-26T03:38:45.625 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-04-26T03:38:45.625 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_fail_eio": "false", 2024-04-26T03:38:45.625 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-04-26T03:38:45.626 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-04-26T03:38:45.626 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-04-26T03:38:45.626 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-04-26T03:38:45.626 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_fsck_on_mkfs": "true", 2024-04-26T03:38:45.626 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-04-26T03:38:45.626 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_fsck_on_mount": "false", 2024-04-26T03:38:45.626 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-04-26T03:38:45.626 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_fsck_on_umount": "false", 2024-04-26T03:38:45.626 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-04-26T03:38:45.626 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-04-26T03:38:45.626 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-04-26T03:38:45.626 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-04-26T03:38:45.626 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-04-26T03:38:45.627 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-04-26T03:38:45.627 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-04-26T03:38:45.628 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-04-26T03:38:45.628 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_ignore_data_csum": "false", 2024-04-26T03:38:45.628 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-04-26T03:38:45.628 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_kvbackend": "rocksdb", 2024-04-26T03:38:45.628 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-04-26T03:38:45.629 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-04-26T03:38:45.629 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_log_op_age": "5.000000", 2024-04-26T03:38:45.629 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_max_alloc_size": "0", 2024-04-26T03:38:45.629 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_max_blob_size": "0", 2024-04-26T03:38:45.629 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-04-26T03:38:45.629 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-04-26T03:38:45.629 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_max_defer_interval": "3.000000", 2024-04-26T03:38:45.629 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_max_deferred_txc": "32", 2024-04-26T03:38:45.630 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_min_alloc_size": "0", 2024-04-26T03:38:45.630 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-04-26T03:38:45.630 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-04-26T03:38:45.630 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_nid_prealloc": "1024", 2024-04-26T03:38:45.630 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_prefer_deferred_size": "0", 2024-04-26T03:38:45.630 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-04-26T03:38:45.630 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-04-26T03:38:45.630 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_qfsck_on_mount": "true", 2024-04-26T03:38:45.630 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_retry_disk_reads": "3", 2024-04-26T03:38:45.630 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_rocksdb_cf": "true", 2024-04-26T03:38:45.630 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-04-26T03:38:45.630 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-04-26T03:38:45.631 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_rocksdb_options_annex": "", 2024-04-26T03:38:45.631 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_spdk_coremask": "0x1", 2024-04-26T03:38:45.631 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_spdk_io_sleep": "5", 2024-04-26T03:38:45.631 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_spdk_max_io_completion": "0", 2024-04-26T03:38:45.631 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_spdk_mem": "512", 2024-04-26T03:38:45.631 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_sync_submit_transaction": "false", 2024-04-26T03:38:45.631 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_throttle_bytes": "67108864", 2024-04-26T03:38:45.631 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_throttle_cost_per_io": "0", 2024-04-26T03:38:45.631 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-04-26T03:38:45.631 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-04-26T03:38:45.631 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-04-26T03:38:45.631 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-04-26T03:38:45.631 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_tracing": "false", 2024-04-26T03:38:45.631 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-04-26T03:38:45.632 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-04-26T03:38:45.632 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_volume_selection_reserved": "0", 2024-04-26T03:38:45.632 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-04-26T03:38:45.632 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-04-26T03:38:45.632 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-04-26T03:38:45.632 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-04-26T03:38:45.632 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-04-26T03:38:45.632 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-04-26T03:38:45.632 INFO:teuthology.orchestra.run.smithi028.stdout: "bluestore_zero_block_detection": "false", 2024-04-26T03:38:45.632 INFO:teuthology.orchestra.run.smithi028.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-04-26T03:38:45.632 INFO:teuthology.orchestra.run.smithi028.stdout: "cephfs_mirror_action_update_interval": "2", 2024-04-26T03:38:45.632 INFO:teuthology.orchestra.run.smithi028.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-04-26T03:38:45.633 INFO:teuthology.orchestra.run.smithi028.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-04-26T03:38:45.633 INFO:teuthology.orchestra.run.smithi028.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-04-26T03:38:45.633 INFO:teuthology.orchestra.run.smithi028.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-04-26T03:38:45.633 INFO:teuthology.orchestra.run.smithi028.stdout: "cephfs_mirror_mount_timeout": "10", 2024-04-26T03:38:45.633 INFO:teuthology.orchestra.run.smithi028.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-04-26T03:38:45.633 INFO:teuthology.orchestra.run.smithi028.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-04-26T03:38:45.633 INFO:teuthology.orchestra.run.smithi028.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-04-26T03:38:45.633 INFO:teuthology.orchestra.run.smithi028.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-04-26T03:38:45.633 INFO:teuthology.orchestra.run.smithi028.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-04-26T03:38:45.633 INFO:teuthology.orchestra.run.smithi028.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-04-26T03:38:45.633 INFO:teuthology.orchestra.run.smithi028.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-04-26T03:38:45.633 INFO:teuthology.orchestra.run.smithi028.stdout: "cephx_cluster_require_signatures": "false", 2024-04-26T03:38:45.634 INFO:teuthology.orchestra.run.smithi028.stdout: "cephx_cluster_require_version": "2", 2024-04-26T03:38:45.634 INFO:teuthology.orchestra.run.smithi028.stdout: "cephx_require_signatures": "false", 2024-04-26T03:38:45.634 INFO:teuthology.orchestra.run.smithi028.stdout: "cephx_require_version": "2", 2024-04-26T03:38:45.634 INFO:teuthology.orchestra.run.smithi028.stdout: "cephx_service_require_signatures": "false", 2024-04-26T03:38:45.634 INFO:teuthology.orchestra.run.smithi028.stdout: "cephx_service_require_version": "2", 2024-04-26T03:38:45.634 INFO:teuthology.orchestra.run.smithi028.stdout: "cephx_sign_messages": "true", 2024-04-26T03:38:45.634 INFO:teuthology.orchestra.run.smithi028.stdout: "chdir": "", 2024-04-26T03:38:45.634 INFO:teuthology.orchestra.run.smithi028.stdout: "client_acl_type": "", 2024-04-26T03:38:45.634 INFO:teuthology.orchestra.run.smithi028.stdout: "client_asio_thread_count": "2", 2024-04-26T03:38:45.634 INFO:teuthology.orchestra.run.smithi028.stdout: "client_cache_mid": "0.750000", 2024-04-26T03:38:45.634 INFO:teuthology.orchestra.run.smithi028.stdout: "client_cache_size": "16384", 2024-04-26T03:38:45.635 INFO:teuthology.orchestra.run.smithi028.stdout: "client_caps_release_delay": "5", 2024-04-26T03:38:45.635 INFO:teuthology.orchestra.run.smithi028.stdout: "client_check_pool_perm": "true", 2024-04-26T03:38:45.635 INFO:teuthology.orchestra.run.smithi028.stdout: "client_collect_and_send_global_metrics": "false", 2024-04-26T03:38:45.635 INFO:teuthology.orchestra.run.smithi028.stdout: "client_debug_force_sync_read": "false", 2024-04-26T03:38:45.635 INFO:teuthology.orchestra.run.smithi028.stdout: "client_debug_getattr_caps": "false", 2024-04-26T03:38:45.635 INFO:teuthology.orchestra.run.smithi028.stdout: "client_debug_inject_tick_delay": "0", 2024-04-26T03:38:45.635 INFO:teuthology.orchestra.run.smithi028.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-04-26T03:38:45.635 INFO:teuthology.orchestra.run.smithi028.stdout: "client_die_on_failed_remount": "false", 2024-04-26T03:38:45.635 INFO:teuthology.orchestra.run.smithi028.stdout: "client_dirsize_rbytes": "true", 2024-04-26T03:38:45.635 INFO:teuthology.orchestra.run.smithi028.stdout: "client_force_lazyio": "false", 2024-04-26T03:38:45.636 INFO:teuthology.orchestra.run.smithi028.stdout: "client_fs": "", 2024-04-26T03:38:45.636 INFO:teuthology.orchestra.run.smithi028.stdout: "client_inject_fixed_oldest_tid": "false", 2024-04-26T03:38:45.636 INFO:teuthology.orchestra.run.smithi028.stdout: "client_inject_release_failure": "false", 2024-04-26T03:38:45.636 INFO:teuthology.orchestra.run.smithi028.stdout: "client_max_inline_size": "4096", 2024-04-26T03:38:45.636 INFO:teuthology.orchestra.run.smithi028.stdout: "client_max_retries_on_remount_failure": "5", 2024-04-26T03:38:45.636 INFO:teuthology.orchestra.run.smithi028.stdout: "client_mds_namespace": "", 2024-04-26T03:38:45.636 INFO:teuthology.orchestra.run.smithi028.stdout: "client_metadata": "", 2024-04-26T03:38:45.636 INFO:teuthology.orchestra.run.smithi028.stdout: "client_mount_gid": "-1", 2024-04-26T03:38:45.636 INFO:teuthology.orchestra.run.smithi028.stdout: "client_mount_timeout": "300", 2024-04-26T03:38:45.636 INFO:teuthology.orchestra.run.smithi028.stdout: "client_mount_uid": "-1", 2024-04-26T03:38:45.636 INFO:teuthology.orchestra.run.smithi028.stdout: "client_mountpoint": "/", 2024-04-26T03:38:45.636 INFO:teuthology.orchestra.run.smithi028.stdout: "client_notify_timeout": "10", 2024-04-26T03:38:45.637 INFO:teuthology.orchestra.run.smithi028.stdout: "client_oc": "true", 2024-04-26T03:38:45.637 INFO:teuthology.orchestra.run.smithi028.stdout: "client_oc_max_dirty": "104857600", 2024-04-26T03:38:45.637 INFO:teuthology.orchestra.run.smithi028.stdout: "client_oc_max_dirty_age": "5.000000", 2024-04-26T03:38:45.637 INFO:teuthology.orchestra.run.smithi028.stdout: "client_oc_max_objects": "1000", 2024-04-26T03:38:45.637 INFO:teuthology.orchestra.run.smithi028.stdout: "client_oc_size": "209715200", 2024-04-26T03:38:45.637 INFO:teuthology.orchestra.run.smithi028.stdout: "client_oc_target_dirty": "8388608", 2024-04-26T03:38:45.637 INFO:teuthology.orchestra.run.smithi028.stdout: "client_permissions": "true", 2024-04-26T03:38:45.637 INFO:teuthology.orchestra.run.smithi028.stdout: "client_quota": "true", 2024-04-26T03:38:45.637 INFO:teuthology.orchestra.run.smithi028.stdout: "client_quota_df": "true", 2024-04-26T03:38:45.637 INFO:teuthology.orchestra.run.smithi028.stdout: "client_readahead_max_bytes": "0", 2024-04-26T03:38:45.637 INFO:teuthology.orchestra.run.smithi028.stdout: "client_readahead_max_periods": "4", 2024-04-26T03:38:45.637 INFO:teuthology.orchestra.run.smithi028.stdout: "client_readahead_min": "131072", 2024-04-26T03:38:45.637 INFO:teuthology.orchestra.run.smithi028.stdout: "client_reconnect_stale": "false", 2024-04-26T03:38:45.637 INFO:teuthology.orchestra.run.smithi028.stdout: "client_shutdown_timeout": "30", 2024-04-26T03:38:45.638 INFO:teuthology.orchestra.run.smithi028.stdout: "client_snapdir": ".snap", 2024-04-26T03:38:45.638 INFO:teuthology.orchestra.run.smithi028.stdout: "client_tick_interval": "1", 2024-04-26T03:38:45.638 INFO:teuthology.orchestra.run.smithi028.stdout: "client_trace": "", 2024-04-26T03:38:45.638 INFO:teuthology.orchestra.run.smithi028.stdout: "client_try_dentry_invalidate": "false", 2024-04-26T03:38:45.638 INFO:teuthology.orchestra.run.smithi028.stdout: "client_use_faked_inos": "false", 2024-04-26T03:38:45.638 INFO:teuthology.orchestra.run.smithi028.stdout: "client_use_random_mds": "false", 2024-04-26T03:38:45.638 INFO:teuthology.orchestra.run.smithi028.stdout: "clog_to_graylog": "false", 2024-04-26T03:38:45.638 INFO:teuthology.orchestra.run.smithi028.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-04-26T03:38:45.638 INFO:teuthology.orchestra.run.smithi028.stdout: "clog_to_graylog_port": "12201", 2024-04-26T03:38:45.638 INFO:teuthology.orchestra.run.smithi028.stdout: "clog_to_monitors": "default=true", 2024-04-26T03:38:45.638 INFO:teuthology.orchestra.run.smithi028.stdout: "clog_to_syslog": "false", 2024-04-26T03:38:45.638 INFO:teuthology.orchestra.run.smithi028.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-04-26T03:38:45.638 INFO:teuthology.orchestra.run.smithi028.stdout: "clog_to_syslog_level": "info", 2024-04-26T03:38:45.639 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "-", 2024-04-26T03:38:45.639 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_network": "", 2024-04-26T03:38:45.639 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_network_interface": "", 2024-04-26T03:38:45.639 INFO:teuthology.orchestra.run.smithi028.stdout: "compressor_zlib_isal": "false", 2024-04-26T03:38:45.639 INFO:teuthology.orchestra.run.smithi028.stdout: "compressor_zlib_level": "5", 2024-04-26T03:38:45.639 INFO:teuthology.orchestra.run.smithi028.stdout: "compressor_zlib_winsize": "-15", 2024-04-26T03:38:45.639 INFO:teuthology.orchestra.run.smithi028.stdout: "compressor_zstd_level": "1", 2024-04-26T03:38:45.639 INFO:teuthology.orchestra.run.smithi028.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-04-26T03:38:45.639 INFO:teuthology.orchestra.run.smithi028.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-04-26T03:38:45.639 INFO:teuthology.orchestra.run.smithi028.stdout: "crimson_alien_op_num_threads": "6", 2024-04-26T03:38:45.639 INFO:teuthology.orchestra.run.smithi028.stdout: "crimson_alien_thread_cpu_cores": "", 2024-04-26T03:38:45.639 INFO:teuthology.orchestra.run.smithi028.stdout: "crimson_osd_obc_lru_size": "512", 2024-04-26T03:38:45.639 INFO:teuthology.orchestra.run.smithi028.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-04-26T03:38:45.640 INFO:teuthology.orchestra.run.smithi028.stdout: "crimson_seastar_cpu_cores": "", 2024-04-26T03:38:45.640 INFO:teuthology.orchestra.run.smithi028.stdout: "crush_location": "", 2024-04-26T03:38:45.640 INFO:teuthology.orchestra.run.smithi028.stdout: "crush_location_hook": "", 2024-04-26T03:38:45.640 INFO:teuthology.orchestra.run.smithi028.stdout: "crush_location_hook_timeout": "10", 2024-04-26T03:38:45.640 INFO:teuthology.orchestra.run.smithi028.stdout: "daemonize": "false", 2024-04-26T03:38:45.640 INFO:teuthology.orchestra.run.smithi028.stdout: "daos_pool": "tank", 2024-04-26T03:38:45.640 INFO:teuthology.orchestra.run.smithi028.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-04-26T03:38:45.640 INFO:teuthology.orchestra.run.smithi028.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-04-26T03:38:45.640 INFO:teuthology.orchestra.run.smithi028.stdout: "dbstore_db_name_prefix": "dbstore", 2024-04-26T03:38:45.640 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_alienstore": "0/5", 2024-04-26T03:38:45.640 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_allow_any_pool_priority": "false", 2024-04-26T03:38:45.640 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_asok": "1/5", 2024-04-26T03:38:45.640 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_asok_assert_abort": "false", 2024-04-26T03:38:45.640 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_asserts_on_shutdown": "true", 2024-04-26T03:38:45.641 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_auth": "20/20", 2024-04-26T03:38:45.641 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_bdev": "1/3", 2024-04-26T03:38:45.641 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_bluefs": "1/5", 2024-04-26T03:38:45.641 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_bluestore": "1/5", 2024-04-26T03:38:45.641 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_buffer": "0/1", 2024-04-26T03:38:45.641 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_ceph_exporter": "1/5", 2024-04-26T03:38:45.641 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_cephfs_mirror": "0/5", 2024-04-26T03:38:45.641 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_cephsqlite": "0/5", 2024-04-26T03:38:45.641 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_client": "0/5", 2024-04-26T03:38:45.641 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_compressor": "1/5", 2024-04-26T03:38:45.641 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_context": "0/1", 2024-04-26T03:38:45.641 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_crimson_interrupt": "0/5", 2024-04-26T03:38:45.641 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_crush": "1/1", 2024-04-26T03:38:45.641 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_crypto": "1/5", 2024-04-26T03:38:45.642 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_cyanstore": "0/5", 2024-04-26T03:38:45.642 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_deliberately_leak_memory": "false", 2024-04-26T03:38:45.642 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_disable_randomized_ping": "false", 2024-04-26T03:38:45.642 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_dpdk": "1/5", 2024-04-26T03:38:45.642 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_eventtrace": "1/5", 2024-04-26T03:38:45.642 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_filer": "0/1", 2024-04-26T03:38:45.642 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_filestore": "1/3", 2024-04-26T03:38:45.642 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_finisher": "1/1", 2024-04-26T03:38:45.642 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_fuse": "1/5", 2024-04-26T03:38:45.642 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_heartbeat_testing_span": "0", 2024-04-26T03:38:45.642 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_heartbeatmap": "1/5", 2024-04-26T03:38:45.642 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_immutable_obj_cache": "0/5", 2024-04-26T03:38:45.642 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_javaclient": "1/5", 2024-04-26T03:38:45.642 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_journal": "1/3", 2024-04-26T03:38:45.643 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_journaler": "0/5", 2024-04-26T03:38:45.643 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_kstore": "1/5", 2024-04-26T03:38:45.643 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_lockdep": "0/1", 2024-04-26T03:38:45.643 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_mclock": "1/5", 2024-04-26T03:38:45.643 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_mds": "1/5", 2024-04-26T03:38:45.643 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_mds_balancer": "1/5", 2024-04-26T03:38:45.643 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_mds_locker": "1/5", 2024-04-26T03:38:45.643 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_mds_log": "1/5", 2024-04-26T03:38:45.643 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_mds_log_expire": "1/5", 2024-04-26T03:38:45.643 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_mds_migrator": "1/5", 2024-04-26T03:38:45.643 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_mds_quiesce": "3/5", 2024-04-26T03:38:45.643 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_memstore": "1/5", 2024-04-26T03:38:45.643 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_mgr": "2/5", 2024-04-26T03:38:45.643 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_mgrc": "1/5", 2024-04-26T03:38:45.643 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_mon": "20/20", 2024-04-26T03:38:45.644 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_monc": "0/10", 2024-04-26T03:38:45.644 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_ms": "1/1", 2024-04-26T03:38:45.644 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_none": "0/5", 2024-04-26T03:38:45.644 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_objclass": "0/5", 2024-04-26T03:38:45.644 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_objectcacher": "0/5", 2024-04-26T03:38:45.644 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_objecter": "0/1", 2024-04-26T03:38:45.644 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_optracker": "0/5", 2024-04-26T03:38:45.644 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_osd": "1/5", 2024-04-26T03:38:45.644 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_paxos": "20/20", 2024-04-26T03:38:45.644 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_perfcounter": "1/5", 2024-04-26T03:38:45.644 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_prioritycache": "1/5", 2024-04-26T03:38:45.644 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_rados": "0/5", 2024-04-26T03:38:45.644 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_rbd": "0/5", 2024-04-26T03:38:45.644 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_rbd_mirror": "0/5", 2024-04-26T03:38:45.645 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_rbd_pwl": "0/5", 2024-04-26T03:38:45.645 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_rbd_replay": "0/5", 2024-04-26T03:38:45.645 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_refs": "0/0", 2024-04-26T03:38:45.645 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_reserver": "1/1", 2024-04-26T03:38:45.645 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_rgw": "1/5", 2024-04-26T03:38:45.645 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_rgw_access": "1/5", 2024-04-26T03:38:45.645 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_rgw_datacache": "1/5", 2024-04-26T03:38:45.645 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_rgw_dbstore": "1/5", 2024-04-26T03:38:45.645 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_rgw_flight": "1/5", 2024-04-26T03:38:45.645 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_rgw_lifecycle": "1/5", 2024-04-26T03:38:45.645 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_rgw_sync": "1/5", 2024-04-26T03:38:45.645 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_rocksdb": "4/5", 2024-04-26T03:38:45.645 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_seastore": "0/5", 2024-04-26T03:38:45.645 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_seastore_backref": "0/5", 2024-04-26T03:38:45.645 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_seastore_cache": "0/5", 2024-04-26T03:38:45.646 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_seastore_cleaner": "0/5", 2024-04-26T03:38:45.646 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_seastore_device": "0/5", 2024-04-26T03:38:45.646 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_seastore_epm": "0/5", 2024-04-26T03:38:45.646 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-04-26T03:38:45.646 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_seastore_journal": "0/5", 2024-04-26T03:38:45.646 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_seastore_lba": "0/5", 2024-04-26T03:38:45.646 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_seastore_odata": "0/5", 2024-04-26T03:38:45.646 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_seastore_omap": "0/5", 2024-04-26T03:38:45.647 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_seastore_onode": "0/5", 2024-04-26T03:38:45.647 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_seastore_t": "0/5", 2024-04-26T03:38:45.647 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_seastore_tm": "0/5", 2024-04-26T03:38:45.647 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_striper": "0/1", 2024-04-26T03:38:45.647 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_test": "0/5", 2024-04-26T03:38:45.647 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_throttle": "1/1", 2024-04-26T03:38:45.647 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_timer": "0/1", 2024-04-26T03:38:45.647 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_tp": "0/5", 2024-04-26T03:38:45.647 INFO:teuthology.orchestra.run.smithi028.stdout: "debug_trace": "1/5", 2024-04-26T03:38:45.647 INFO:teuthology.orchestra.run.smithi028.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-04-26T03:38:45.647 INFO:teuthology.orchestra.run.smithi028.stdout: "device_failure_prediction_mode": "none", 2024-04-26T03:38:45.647 INFO:teuthology.orchestra.run.smithi028.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-04-26T03:38:45.647 INFO:teuthology.orchestra.run.smithi028.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-04-26T03:38:45.647 INFO:teuthology.orchestra.run.smithi028.stdout: "err_to_graylog": "false", 2024-04-26T03:38:45.648 INFO:teuthology.orchestra.run.smithi028.stdout: "err_to_journald": "false", 2024-04-26T03:38:45.648 INFO:teuthology.orchestra.run.smithi028.stdout: "err_to_stderr": "true", 2024-04-26T03:38:45.648 INFO:teuthology.orchestra.run.smithi028.stdout: "err_to_syslog": "false", 2024-04-26T03:38:45.648 INFO:teuthology.orchestra.run.smithi028.stdout: "event_tracing": "false", 2024-04-26T03:38:45.648 INFO:teuthology.orchestra.run.smithi028.stdout: "exporter_addr": "0.0.0.0", 2024-04-26T03:38:45.648 INFO:teuthology.orchestra.run.smithi028.stdout: "exporter_http_port": "9926", 2024-04-26T03:38:45.648 INFO:teuthology.orchestra.run.smithi028.stdout: "exporter_prio_limit": "5", 2024-04-26T03:38:45.648 INFO:teuthology.orchestra.run.smithi028.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-04-26T03:38:45.648 INFO:teuthology.orchestra.run.smithi028.stdout: "exporter_sort_metrics": "true", 2024-04-26T03:38:45.648 INFO:teuthology.orchestra.run.smithi028.stdout: "exporter_stats_period": "5", 2024-04-26T03:38:45.648 INFO:teuthology.orchestra.run.smithi028.stdout: "fake_statfs_for_testing": "0", 2024-04-26T03:38:45.648 INFO:teuthology.orchestra.run.smithi028.stdout: "fatal_signal_handlers": "true", 2024-04-26T03:38:45.648 INFO:teuthology.orchestra.run.smithi028.stdout: "filer_max_purge_ops": "10", 2024-04-26T03:38:45.648 INFO:teuthology.orchestra.run.smithi028.stdout: "filer_max_truncate_ops": "128", 2024-04-26T03:38:45.648 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_apply_finisher_threads": "1", 2024-04-26T03:38:45.649 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_blackhole": "false", 2024-04-26T03:38:45.649 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_btrfs_clone_range": "true", 2024-04-26T03:38:45.649 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_btrfs_snap": "true", 2024-04-26T03:38:45.649 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_caller_concurrency": "10", 2024-04-26T03:38:45.649 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_collect_device_partition_information": "true", 2024-04-26T03:38:45.649 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_commit_timeout": "600.000000", 2024-04-26T03:38:45.649 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_debug_inject_read_err": "false", 2024-04-26T03:38:45.649 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_debug_omap_check": "false", 2024-04-26T03:38:45.649 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_debug_random_read_err": "0.000000", 2024-04-26T03:38:45.649 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_debug_verify_split": "false", 2024-04-26T03:38:45.649 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_dump_file": "", 2024-04-26T03:38:45.649 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-04-26T03:38:45.649 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-04-26T03:38:45.649 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_fadvise": "true", 2024-04-26T03:38:45.650 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_fail_eio": "true", 2024-04-26T03:38:45.650 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_fd_cache_shards": "16", 2024-04-26T03:38:45.650 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_fd_cache_size": "128", 2024-04-26T03:38:45.650 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_fiemap": "false", 2024-04-26T03:38:45.650 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_fiemap_threshold": "4096", 2024-04-26T03:38:45.650 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-04-26T03:38:45.650 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_index_retry_probability": "0.000000", 2024-04-26T03:38:45.650 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_inject_stall": "0", 2024-04-26T03:38:45.650 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_journal_parallel": "false", 2024-04-26T03:38:45.651 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_journal_trailing": "false", 2024-04-26T03:38:45.651 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_journal_writeahead": "false", 2024-04-26T03:38:45.651 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_kill_at": "0", 2024-04-26T03:38:45.651 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-04-26T03:38:45.651 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_max_inline_xattr_size": "0", 2024-04-26T03:38:45.651 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-04-26T03:38:45.651 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-04-26T03:38:45.651 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-04-26T03:38:45.651 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_max_inline_xattrs": "0", 2024-04-26T03:38:45.651 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-04-26T03:38:45.651 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_max_inline_xattrs_other": "2", 2024-04-26T03:38:45.651 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-04-26T03:38:45.652 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_max_sync_interval": "5.000000", 2024-04-26T03:38:45.652 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_max_xattr_value_size": "0", 2024-04-26T03:38:45.652 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-04-26T03:38:45.652 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-04-26T03:38:45.652 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-04-26T03:38:45.652 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_merge_threshold": "-10", 2024-04-26T03:38:45.652 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_min_sync_interval": "0.010000", 2024-04-26T03:38:45.652 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_odsync_write": "false", 2024-04-26T03:38:45.652 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_omap_backend": "rocksdb", 2024-04-26T03:38:45.652 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_omap_backend_path": "", 2024-04-26T03:38:45.652 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_omap_header_cache_size": "1024", 2024-04-26T03:38:45.652 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_ondisk_finisher_threads": "1", 2024-04-26T03:38:45.653 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-04-26T03:38:45.653 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_op_thread_timeout": "60", 2024-04-26T03:38:45.653 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_op_threads": "2", 2024-04-26T03:38:45.653 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_punch_hole": "false", 2024-04-26T03:38:45.653 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-04-26T03:38:45.653 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-04-26T03:38:45.653 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-04-26T03:38:45.653 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-04-26T03:38:45.653 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-04-26T03:38:45.653 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_queue_max_bytes": "104857600", 2024-04-26T03:38:45.654 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-04-26T03:38:45.654 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-04-26T03:38:45.654 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-04-26T03:38:45.654 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_queue_max_ops": "50", 2024-04-26T03:38:45.654 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-04-26T03:38:45.654 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_seek_data_hole": "false", 2024-04-26T03:38:45.654 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_sloppy_crc": "false", 2024-04-26T03:38:45.654 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-04-26T03:38:45.654 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_splice": "false", 2024-04-26T03:38:45.654 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_split_multiple": "2", 2024-04-26T03:38:45.654 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_split_rand_factor": "20", 2024-04-26T03:38:45.655 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_update_to": "1000", 2024-04-26T03:38:45.655 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-04-26T03:38:45.655 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-04-26T03:38:45.655 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-04-26T03:38:45.655 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-04-26T03:38:45.655 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-04-26T03:38:45.655 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-04-26T03:38:45.655 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_wbthrottle_enable": "true", 2024-04-26T03:38:45.655 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-04-26T03:38:45.655 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-04-26T03:38:45.655 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-04-26T03:38:45.655 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-04-26T03:38:45.655 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-04-26T03:38:45.655 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-04-26T03:38:45.656 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_xfs_extsize": "false", 2024-04-26T03:38:45.656 INFO:teuthology.orchestra.run.smithi028.stdout: "filestore_zfs_snap": "false", 2024-04-26T03:38:45.656 INFO:teuthology.orchestra.run.smithi028.stdout: "fio_dir": "/tmp/fio", 2024-04-26T03:38:45.656 INFO:teuthology.orchestra.run.smithi028.stdout: "fsid": "c5a13c99-5c9a-4b8b-a893-4f6f4211677a", 2024-04-26T03:38:45.656 INFO:teuthology.orchestra.run.smithi028.stdout: "fuse_allow_other": "true", 2024-04-26T03:38:45.656 INFO:teuthology.orchestra.run.smithi028.stdout: "fuse_atomic_o_trunc": "true", 2024-04-26T03:38:45.656 INFO:teuthology.orchestra.run.smithi028.stdout: "fuse_big_writes": "true", 2024-04-26T03:38:45.656 INFO:teuthology.orchestra.run.smithi028.stdout: "fuse_debug": "false", 2024-04-26T03:38:45.656 INFO:teuthology.orchestra.run.smithi028.stdout: "fuse_default_permissions": "false", 2024-04-26T03:38:45.656 INFO:teuthology.orchestra.run.smithi028.stdout: "fuse_disable_pagecache": "false", 2024-04-26T03:38:45.656 INFO:teuthology.orchestra.run.smithi028.stdout: "fuse_max_write": "0", 2024-04-26T03:38:45.656 INFO:teuthology.orchestra.run.smithi028.stdout: "fuse_multithreaded": "true", 2024-04-26T03:38:45.656 INFO:teuthology.orchestra.run.smithi028.stdout: "fuse_require_active_mds": "true", 2024-04-26T03:38:45.657 INFO:teuthology.orchestra.run.smithi028.stdout: "fuse_set_user_groups": "true", 2024-04-26T03:38:45.657 INFO:teuthology.orchestra.run.smithi028.stdout: "fuse_splice_move": "true", 2024-04-26T03:38:45.657 INFO:teuthology.orchestra.run.smithi028.stdout: "fuse_splice_read": "true", 2024-04-26T03:38:45.657 INFO:teuthology.orchestra.run.smithi028.stdout: "fuse_splice_write": "true", 2024-04-26T03:38:45.657 INFO:teuthology.orchestra.run.smithi028.stdout: "fuse_syncfs_on_mksnap": "true", 2024-04-26T03:38:45.657 INFO:teuthology.orchestra.run.smithi028.stdout: "fuse_use_invalidate_cb": "true", 2024-04-26T03:38:45.657 INFO:teuthology.orchestra.run.smithi028.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-04-26T03:38:45.658 INFO:teuthology.orchestra.run.smithi028.stdout: "gss_target_name": "ceph", 2024-04-26T03:38:45.658 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_file": "", 2024-04-26T03:38:45.658 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_inject_failure": "0", 2024-04-26T03:38:45.658 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_interval": "5", 2024-04-26T03:38:45.658 INFO:teuthology.orchestra.run.smithi028.stdout: "host": "smithi028", 2024-04-26T03:38:45.658 INFO:teuthology.orchestra.run.smithi028.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-04-26T03:38:45.658 INFO:teuthology.orchestra.run.smithi028.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-04-26T03:38:45.658 INFO:teuthology.orchestra.run.smithi028.stdout: "immutable_object_cache_max_size": "1073741824", 2024-04-26T03:38:45.658 INFO:teuthology.orchestra.run.smithi028.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-04-26T03:38:45.658 INFO:teuthology.orchestra.run.smithi028.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-04-26T03:38:45.658 INFO:teuthology.orchestra.run.smithi028.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-04-26T03:38:45.658 INFO:teuthology.orchestra.run.smithi028.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-04-26T03:38:45.659 INFO:teuthology.orchestra.run.smithi028.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-04-26T03:38:45.659 INFO:teuthology.orchestra.run.smithi028.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-04-26T03:38:45.659 INFO:teuthology.orchestra.run.smithi028.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-04-26T03:38:45.659 INFO:teuthology.orchestra.run.smithi028.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-04-26T03:38:45.659 INFO:teuthology.orchestra.run.smithi028.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-04-26T03:38:45.659 INFO:teuthology.orchestra.run.smithi028.stdout: "immutable_object_cache_watermark": "0.900000", 2024-04-26T03:38:45.659 INFO:teuthology.orchestra.run.smithi028.stdout: "inject_early_sigterm": "false", 2024-04-26T03:38:45.659 INFO:teuthology.orchestra.run.smithi028.stdout: "jaeger_agent_port": "6799", 2024-04-26T03:38:45.659 INFO:teuthology.orchestra.run.smithi028.stdout: "jaeger_tracing_enable": "false", 2024-04-26T03:38:45.659 INFO:teuthology.orchestra.run.smithi028.stdout: "journal_aio": "true", 2024-04-26T03:38:45.659 INFO:teuthology.orchestra.run.smithi028.stdout: "journal_align_min_size": "65536", 2024-04-26T03:38:45.659 INFO:teuthology.orchestra.run.smithi028.stdout: "journal_block_align": "true", 2024-04-26T03:38:45.660 INFO:teuthology.orchestra.run.smithi028.stdout: "journal_block_size": "4096", 2024-04-26T03:38:45.660 INFO:teuthology.orchestra.run.smithi028.stdout: "journal_dio": "true", 2024-04-26T03:38:45.660 INFO:teuthology.orchestra.run.smithi028.stdout: "journal_discard": "false", 2024-04-26T03:38:45.660 INFO:teuthology.orchestra.run.smithi028.stdout: "journal_force_aio": "false", 2024-04-26T03:38:45.660 INFO:teuthology.orchestra.run.smithi028.stdout: "journal_ignore_corruption": "false", 2024-04-26T03:38:45.660 INFO:teuthology.orchestra.run.smithi028.stdout: "journal_max_write_bytes": "10485760", 2024-04-26T03:38:45.660 INFO:teuthology.orchestra.run.smithi028.stdout: "journal_max_write_entries": "100", 2024-04-26T03:38:45.660 INFO:teuthology.orchestra.run.smithi028.stdout: "journal_replay_from": "0", 2024-04-26T03:38:45.660 INFO:teuthology.orchestra.run.smithi028.stdout: "journal_throttle_high_multiple": "0.000000", 2024-04-26T03:38:45.660 INFO:teuthology.orchestra.run.smithi028.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-04-26T03:38:45.660 INFO:teuthology.orchestra.run.smithi028.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-04-26T03:38:45.660 INFO:teuthology.orchestra.run.smithi028.stdout: "journal_throttle_max_multiple": "0.000000", 2024-04-26T03:38:45.660 INFO:teuthology.orchestra.run.smithi028.stdout: "journal_write_header_frequency": "0", 2024-04-26T03:38:45.660 INFO:teuthology.orchestra.run.smithi028.stdout: "journal_zero_on_create": "false", 2024-04-26T03:38:45.660 INFO:teuthology.orchestra.run.smithi028.stdout: "journaler_prefetch_periods": "10", 2024-04-26T03:38:45.661 INFO:teuthology.orchestra.run.smithi028.stdout: "journaler_prezero_periods": "5", 2024-04-26T03:38:45.661 INFO:teuthology.orchestra.run.smithi028.stdout: "journaler_write_head_interval": "15", 2024-04-26T03:38:45.661 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "", 2024-04-26T03:38:45.661 INFO:teuthology.orchestra.run.smithi028.stdout: "keyfile": "", 2024-04-26T03:38:45.661 INFO:teuthology.orchestra.run.smithi028.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-04-26T03:38:45.661 INFO:teuthology.orchestra.run.smithi028.stdout: "kstore_backend": "rocksdb", 2024-04-26T03:38:45.661 INFO:teuthology.orchestra.run.smithi028.stdout: "kstore_default_stripe_size": "65536", 2024-04-26T03:38:45.661 INFO:teuthology.orchestra.run.smithi028.stdout: "kstore_fsck_on_mount": "false", 2024-04-26T03:38:45.661 INFO:teuthology.orchestra.run.smithi028.stdout: "kstore_fsck_on_mount_deep": "true", 2024-04-26T03:38:45.661 INFO:teuthology.orchestra.run.smithi028.stdout: "kstore_max_bytes": "67108864", 2024-04-26T03:38:45.661 INFO:teuthology.orchestra.run.smithi028.stdout: "kstore_max_ops": "512", 2024-04-26T03:38:45.661 INFO:teuthology.orchestra.run.smithi028.stdout: "kstore_nid_prealloc": "1024", 2024-04-26T03:38:45.661 INFO:teuthology.orchestra.run.smithi028.stdout: "kstore_onode_map_size": "1024", 2024-04-26T03:38:45.661 INFO:teuthology.orchestra.run.smithi028.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-04-26T03:38:45.662 INFO:teuthology.orchestra.run.smithi028.stdout: "kstore_sync_submit_transaction": "false", 2024-04-26T03:38:45.662 INFO:teuthology.orchestra.run.smithi028.stdout: "kstore_sync_transaction": "false", 2024-04-26T03:38:45.662 INFO:teuthology.orchestra.run.smithi028.stdout: "librados_thread_count": "2", 2024-04-26T03:38:45.662 INFO:teuthology.orchestra.run.smithi028.stdout: "lockdep": "false", 2024-04-26T03:38:45.662 INFO:teuthology.orchestra.run.smithi028.stdout: "lockdep_force_backtrace": "false", 2024-04-26T03:38:45.662 INFO:teuthology.orchestra.run.smithi028.stdout: "log_coarse_timestamps": "true", 2024-04-26T03:38:45.662 INFO:teuthology.orchestra.run.smithi028.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-04-26T03:38:45.662 INFO:teuthology.orchestra.run.smithi028.stdout: "log_flush_on_exit": "false", 2024-04-26T03:38:45.662 INFO:teuthology.orchestra.run.smithi028.stdout: "log_graylog_host": "127.0.0.1", 2024-04-26T03:38:45.662 INFO:teuthology.orchestra.run.smithi028.stdout: "log_graylog_port": "12201", 2024-04-26T03:38:45.662 INFO:teuthology.orchestra.run.smithi028.stdout: "log_max_new": "1000", 2024-04-26T03:38:45.662 INFO:teuthology.orchestra.run.smithi028.stdout: "log_max_recent": "10000", 2024-04-26T03:38:45.662 INFO:teuthology.orchestra.run.smithi028.stdout: "log_stderr_prefix": "", 2024-04-26T03:38:45.662 INFO:teuthology.orchestra.run.smithi028.stdout: "log_stop_at_utilization": "0.970000", 2024-04-26T03:38:45.663 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_file": "true", 2024-04-26T03:38:45.663 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_graylog": "false", 2024-04-26T03:38:45.663 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_journald": "false", 2024-04-26T03:38:45.663 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_stderr": "false", 2024-04-26T03:38:45.663 INFO:teuthology.orchestra.run.smithi028.stdout: "log_to_syslog": "false", 2024-04-26T03:38:45.663 INFO:teuthology.orchestra.run.smithi028.stdout: "max_rotating_auth_attempts": "10", 2024-04-26T03:38:45.663 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-04-26T03:38:45.663 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_action_on_write_error": "1", 2024-04-26T03:38:45.663 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_alternate_name_max": "8192", 2024-04-26T03:38:45.663 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_asio_thread_count": "2", 2024-04-26T03:38:45.663 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_bal_export_pin": "true", 2024-04-26T03:38:45.663 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_bal_fragment_dirs": "true", 2024-04-26T03:38:45.664 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-04-26T03:38:45.664 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_bal_fragment_interval": "5", 2024-04-26T03:38:45.664 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_bal_fragment_size_max": "100000", 2024-04-26T03:38:45.664 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_bal_idle_threshold": "0.000000", 2024-04-26T03:38:45.664 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_bal_interval": "10", 2024-04-26T03:38:45.664 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_bal_max": "-1", 2024-04-26T03:38:45.664 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_bal_max_until": "-1", 2024-04-26T03:38:45.664 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_bal_merge_size": "50", 2024-04-26T03:38:45.664 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_bal_midchunk": "0.300000", 2024-04-26T03:38:45.664 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_bal_min_rebalance": "0.100000", 2024-04-26T03:38:45.664 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_bal_min_start": "0.200000", 2024-04-26T03:38:45.664 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_bal_minchunk": "0.001000", 2024-04-26T03:38:45.664 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_bal_mode": "0", 2024-04-26T03:38:45.665 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_bal_need_max": "1.200000", 2024-04-26T03:38:45.665 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_bal_need_min": "0.800000", 2024-04-26T03:38:45.665 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_bal_overload_epochs": "2", 2024-04-26T03:38:45.665 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-04-26T03:38:45.665 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_bal_sample_interval": "3.000000", 2024-04-26T03:38:45.665 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_bal_split_bits": "3", 2024-04-26T03:38:45.665 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_bal_split_rd": "25000.000000", 2024-04-26T03:38:45.665 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_bal_split_size": "10000", 2024-04-26T03:38:45.665 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_bal_split_wr": "10000.000000", 2024-04-26T03:38:45.665 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_bal_target_decay": "10.000000", 2024-04-26T03:38:45.665 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-04-26T03:38:45.665 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_beacon_grace": "15.000000", 2024-04-26T03:38:45.665 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_beacon_interval": "4.000000", 2024-04-26T03:38:45.665 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_beacon_mon_down_grace": "60", 2024-04-26T03:38:45.665 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_cache_memory_limit": "4294967296", 2024-04-26T03:38:45.666 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_cache_mid": "0.700000", 2024-04-26T03:38:45.666 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-04-26T03:38:45.666 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_cache_quiesce_delay": "0", 2024-04-26T03:38:45.666 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_cache_quiesce_sleep": "200", 2024-04-26T03:38:45.666 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_cache_quiesce_splitauth": "true", 2024-04-26T03:38:45.666 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_cache_quiesce_threshold": "524288", 2024-04-26T03:38:45.666 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_cache_release_free_interval": "10", 2024-04-26T03:38:45.666 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_cache_reservation": "0.050000", 2024-04-26T03:38:45.666 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-04-26T03:38:45.666 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_cache_trim_interval": "1", 2024-04-26T03:38:45.667 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_cache_trim_threshold": "262144", 2024-04-26T03:38:45.667 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-04-26T03:38:45.667 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-04-26T03:38:45.667 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_client_delegate_inos_pct": "50", 2024-04-26T03:38:45.667 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_client_prealloc_inos": "1000", 2024-04-26T03:38:45.667 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-04-26T03:38:45.667 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_connect_bootstrapping": "false", 2024-04-26T03:38:45.667 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_damage_table_max_entries": "10000", 2024-04-26T03:38:45.668 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-04-26T03:38:45.668 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_debug_auth_pins": "false", 2024-04-26T03:38:45.668 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_debug_frag": "false", 2024-04-26T03:38:45.668 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_debug_scatterstat": "false", 2024-04-26T03:38:45.668 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_debug_subtrees": "false", 2024-04-26T03:38:45.668 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_decay_halflife": "5.000000", 2024-04-26T03:38:45.668 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_default_dir_hash": "2", 2024-04-26T03:38:45.668 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_defer_session_stale": "true", 2024-04-26T03:38:45.668 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_deny_all_reconnect": "false", 2024-04-26T03:38:45.669 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_dir_keys_per_op": "16384", 2024-04-26T03:38:45.669 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_dir_max_commit_size": "10", 2024-04-26T03:38:45.669 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_dir_max_entries": "0", 2024-04-26T03:38:45.669 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_dir_prefetch": "true", 2024-04-26T03:38:45.669 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_dirstat_min_interval": "1.000000", 2024-04-26T03:38:45.669 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_dump_cache_after_rejoin": "false", 2024-04-26T03:38:45.669 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_dump_cache_on_map": "false", 2024-04-26T03:38:45.669 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_dump_cache_threshold_file": "0", 2024-04-26T03:38:45.669 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-04-26T03:38:45.670 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_early_reply": "true", 2024-04-26T03:38:45.670 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_enable_op_tracker": "true", 2024-04-26T03:38:45.670 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_enforce_unique_name": "true", 2024-04-26T03:38:45.670 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_export_ephemeral_distributed": "true", 2024-04-26T03:38:45.670 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-04-26T03:38:45.670 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_export_ephemeral_random": "true", 2024-04-26T03:38:45.670 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-04-26T03:38:45.670 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-04-26T03:38:45.671 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_forward_all_requests_to_auth": "false", 2024-04-26T03:38:45.671 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-04-26T03:38:45.671 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-04-26T03:38:45.671 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-04-26T03:38:45.671 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-04-26T03:38:45.671 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_health_cache_threshold": "1.500000", 2024-04-26T03:38:45.671 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_health_summarize_threshold": "10", 2024-04-26T03:38:45.671 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_heartbeat_grace": "15.000000", 2024-04-26T03:38:45.672 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_heartbeat_reset_grace": "1000", 2024-04-26T03:38:45.672 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_inject_health_dummy": "false", 2024-04-26T03:38:45.672 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-04-26T03:38:45.672 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_inject_migrator_session_race": "false", 2024-04-26T03:38:45.672 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-04-26T03:38:45.672 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-04-26T03:38:45.672 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-04-26T03:38:45.672 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_join_fs": "", 2024-04-26T03:38:45.672 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_journal_format": "1", 2024-04-26T03:38:45.673 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_kill_create_at": "0", 2024-04-26T03:38:45.673 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_kill_export_at": "0", 2024-04-26T03:38:45.673 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_kill_import_at": "0", 2024-04-26T03:38:45.673 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_kill_journal_at": "0", 2024-04-26T03:38:45.673 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_kill_journal_expire_at": "0", 2024-04-26T03:38:45.673 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_kill_journal_replay_at": "0", 2024-04-26T03:38:45.673 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_kill_link_at": "0", 2024-04-26T03:38:45.673 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_kill_mdstable_at": "0", 2024-04-26T03:38:45.674 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_kill_openc_at": "0", 2024-04-26T03:38:45.674 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_kill_rename_at": "0", 2024-04-26T03:38:45.674 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_kill_shutdown_at": "0", 2024-04-26T03:38:45.674 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_kill_skip_replaying_inotable": "false", 2024-04-26T03:38:45.674 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_log_event_large_threshold": "524288", 2024-04-26T03:38:45.674 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_log_events_per_segment": "1024", 2024-04-26T03:38:45.674 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_log_major_segment_event_ratio": "12", 2024-04-26T03:38:45.674 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_log_max_events": "-1", 2024-04-26T03:38:45.674 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_log_max_segments": "128", 2024-04-26T03:38:45.675 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_log_pause": "false", 2024-04-26T03:38:45.675 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_log_segment_size": "0", 2024-04-26T03:38:45.675 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_log_skip_corrupt_events": "false", 2024-04-26T03:38:45.675 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_log_skip_unbounded_events": "false", 2024-04-26T03:38:45.675 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-04-26T03:38:45.675 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_log_trim_threshold": "128", 2024-04-26T03:38:45.675 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-04-26T03:38:45.676 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_log_warn_factor": "2.000000", 2024-04-26T03:38:45.676 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_max_caps_per_client": "1048576", 2024-04-26T03:38:45.676 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_max_completed_flushes": "100000", 2024-04-26T03:38:45.676 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_max_completed_requests": "100000", 2024-04-26T03:38:45.676 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_max_export_size": "20971520", 2024-04-26T03:38:45.676 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_max_file_recover": "32", 2024-04-26T03:38:45.676 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_max_purge_files": "64", 2024-04-26T03:38:45.676 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_max_purge_ops": "8192", 2024-04-26T03:38:45.677 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-04-26T03:38:45.677 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-04-26T03:38:45.677 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_max_snaps_per_dir": "100", 2024-04-26T03:38:45.677 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_metrics_update_interval": "2", 2024-04-26T03:38:45.677 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_min_caps_per_client": "100", 2024-04-26T03:38:45.677 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_min_caps_working_set": "10000", 2024-04-26T03:38:45.677 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-04-26T03:38:45.678 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_numa_node": "-1", 2024-04-26T03:38:45.678 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-04-26T03:38:45.678 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_op_complaint_time": "30.000000", 2024-04-26T03:38:45.678 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_op_history_duration": "600", 2024-04-26T03:38:45.678 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_op_history_size": "20", 2024-04-26T03:38:45.678 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_op_history_slow_op_size": "20", 2024-04-26T03:38:45.678 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_op_history_slow_op_threshold": "10", 2024-04-26T03:38:45.679 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_op_log_threshold": "5", 2024-04-26T03:38:45.679 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_ping_grace": "15", 2024-04-26T03:38:45.679 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_ping_interval": "5", 2024-04-26T03:38:45.679 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-04-26T03:38:45.679 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-04-26T03:38:45.679 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_recall_max_caps": "30000", 2024-04-26T03:38:45.679 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-04-26T03:38:45.679 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_recall_max_decay_threshold": "131072", 2024-04-26T03:38:45.680 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-04-26T03:38:45.680 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_recall_warning_threshold": "262144", 2024-04-26T03:38:45.680 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_reconnect_timeout": "45.000000", 2024-04-26T03:38:45.680 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_replay_interval": "1.000000", 2024-04-26T03:38:45.680 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-04-26T03:38:45.680 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-04-26T03:38:45.680 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_root_ino_gid": "0", 2024-04-26T03:38:45.681 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_root_ino_uid": "0", 2024-04-26T03:38:45.681 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-04-26T03:38:45.681 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_session_blocklist_on_evict": "true", 2024-04-26T03:38:45.681 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_session_blocklist_on_timeout": "true", 2024-04-26T03:38:45.681 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-04-26T03:38:45.681 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-04-26T03:38:45.681 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-04-26T03:38:45.682 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-04-26T03:38:45.682 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-04-26T03:38:45.682 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_session_metadata_threshold": "16777216", 2024-04-26T03:38:45.682 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-04-26T03:38:45.682 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_shutdown_check": "0", 2024-04-26T03:38:45.682 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_skip_ino": "0", 2024-04-26T03:38:45.682 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_sleep_rank_change": "0.000000", 2024-04-26T03:38:45.682 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_snap_max_uid": "4294967294", 2024-04-26T03:38:45.683 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_snap_min_uid": "0", 2024-04-26T03:38:45.683 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_snap_rstat": "false", 2024-04-26T03:38:45.683 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_standby_replay_damaged": "false", 2024-04-26T03:38:45.683 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_symlink_recovery": "true", 2024-04-26T03:38:45.683 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_task_status_update_interval": "2.000000", 2024-04-26T03:38:45.683 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_thrash_exports": "0", 2024-04-26T03:38:45.683 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_thrash_fragments": "0", 2024-04-26T03:38:45.684 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_tick_interval": "5.000000", 2024-04-26T03:38:45.684 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_valgrind_exit": "false", 2024-04-26T03:38:45.684 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_verify_backtrace": "1", 2024-04-26T03:38:45.684 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_verify_scatter": "false", 2024-04-26T03:38:45.684 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_wipe_ino_prealloc": "false", 2024-04-26T03:38:45.684 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_wipe_sessions": "false", 2024-04-26T03:38:45.684 INFO:teuthology.orchestra.run.smithi028.stdout: "mempool_debug": "false", 2024-04-26T03:38:45.684 INFO:teuthology.orchestra.run.smithi028.stdout: "memstore_debug_omit_block_device_write": "false", 2024-04-26T03:38:45.685 INFO:teuthology.orchestra.run.smithi028.stdout: "memstore_device_bytes": "1073741824", 2024-04-26T03:38:45.685 INFO:teuthology.orchestra.run.smithi028.stdout: "memstore_page_set": "false", 2024-04-26T03:38:45.685 INFO:teuthology.orchestra.run.smithi028.stdout: "memstore_page_size": "65536", 2024-04-26T03:38:45.685 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr_client_bytes": "134217728", 2024-04-26T03:38:45.685 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr_client_messages": "512", 2024-04-26T03:38:45.685 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-04-26T03:38:45.685 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr_connect_retry_interval": "1.000000", 2024-04-26T03:38:45.686 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-04-26T03:38:45.686 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-04-26T03:38:45.686 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr_disabled_modules": "", 2024-04-26T03:38:45.686 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-04-26T03:38:45.686 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr_max_pg_num_change": "128", 2024-04-26T03:38:45.686 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr_mds_bytes": "134217728", 2024-04-26T03:38:45.686 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr_mds_messages": "128", 2024-04-26T03:38:45.687 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-04-26T03:38:45.687 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr_mon_bytes": "134217728", 2024-04-26T03:38:45.687 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr_mon_messages": "128", 2024-04-26T03:38:45.687 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr_osd_bytes": "536870912", 2024-04-26T03:38:45.687 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr_osd_messages": "8192", 2024-04-26T03:38:45.687 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr_pool": "true", 2024-04-26T03:38:45.687 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr_service_beacon_grace": "60.000000", 2024-04-26T03:38:45.687 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr_standby_modules": "true", 2024-04-26T03:38:45.688 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr_stats_period": "5", 2024-04-26T03:38:45.688 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr_stats_threshold": "5", 2024-04-26T03:38:45.688 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr_tick_period": "2", 2024-04-26T03:38:45.688 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-04-26T03:38:45.688 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_accept_timeout_factor": "2.000000", 2024-04-26T03:38:45.688 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_allow_pool_delete": "true", 2024-04-26T03:38:45.688 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_allow_pool_size_one": "true", 2024-04-26T03:38:45.688 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_auth_validate_all_caps": "true", 2024-04-26T03:38:45.689 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-04-26T03:38:45.689 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-04-26T03:38:45.689 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_client_bytes": "104857600", 2024-04-26T03:38:45.689 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_client_directed_command_retry": "5", 2024-04-26T03:38:45.689 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_client_hunt_interval": "3.000000", 2024-04-26T03:38:45.689 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-04-26T03:38:45.689 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-04-26T03:38:45.690 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-04-26T03:38:45.690 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_client_hunt_parallel": "3", 2024-04-26T03:38:45.690 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_client_log_interval": "1.000000", 2024-04-26T03:38:45.690 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-04-26T03:38:45.690 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_client_ping_interval": "10.000000", 2024-04-26T03:38:45.690 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_client_ping_timeout": "30.000000", 2024-04-26T03:38:45.690 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_client_target_rank": "-1", 2024-04-26T03:38:45.690 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_clock_drift_allowed": "1.000000", 2024-04-26T03:38:45.691 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-04-26T03:38:45.691 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-04-26T03:38:45.691 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_cluster_log_file_level": "debug", 2024-04-26T03:38:45.691 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_cluster_log_to_file": "true", 2024-04-26T03:38:45.691 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_cluster_log_to_graylog": "false", 2024-04-26T03:38:45.691 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-04-26T03:38:45.691 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-04-26T03:38:45.692 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_cluster_log_to_journald": "false", 2024-04-26T03:38:45.692 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_cluster_log_to_stderr": "false", 2024-04-26T03:38:45.692 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-04-26T03:38:45.692 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-04-26T03:38:45.692 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-04-26T03:38:45.692 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_compact_on_bootstrap": "false", 2024-04-26T03:38:45.692 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_compact_on_start": "false", 2024-04-26T03:38:45.693 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_compact_on_trim": "true", 2024-04-26T03:38:45.693 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_con_tracker_persist_interval": "10", 2024-04-26T03:38:45.693 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_con_tracker_score_halflife": "43200", 2024-04-26T03:38:45.693 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_config_key_max_entry_size": "65536", 2024-04-26T03:38:45.693 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_cpu_threads": "4", 2024-04-26T03:38:45.693 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_crush_min_required_version": "hammer", 2024-04-26T03:38:45.693 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_daemon_bytes": "419430400", 2024-04-26T03:38:45.693 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-04-26T03:38:45.694 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_data_avail_crit": "5", 2024-04-26T03:38:45.694 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_data_avail_warn": "5", 2024-04-26T03:38:45.694 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_data_size_warn": "16106127360", 2024-04-26T03:38:45.694 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_debug_block_osdmap_trim": "false", 2024-04-26T03:38:45.694 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-04-26T03:38:45.694 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_debug_dump_json": "false", 2024-04-26T03:38:45.694 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-04-26T03:38:45.695 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_debug_dump_transactions": "false", 2024-04-26T03:38:45.695 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_debug_extra_checks": "false", 2024-04-26T03:38:45.695 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-04-26T03:38:45.695 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-04-26T03:38:45.695 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_debug_no_require_reef": "false", 2024-04-26T03:38:45.695 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_debug_no_require_squid": "false", 2024-04-26T03:38:45.695 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-04-26T03:38:45.696 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_delta_reset_interval": "10.000000", 2024-04-26T03:38:45.696 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_dns_srv_name": "ceph-mon", 2024-04-26T03:38:45.696 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_down_mkfs_grace": "120", 2024-04-26T03:38:45.696 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_down_uptime_grace": "60", 2024-04-26T03:38:45.696 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_election_default_strategy": "1", 2024-04-26T03:38:45.696 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_election_timeout": "5.000000", 2024-04-26T03:38:45.696 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-04-26T03:38:45.696 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_elector_ping_divisor": "2", 2024-04-26T03:38:45.697 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_elector_ping_timeout": "2.000000", 2024-04-26T03:38:45.697 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_enable_op_tracker": "true", 2024-04-26T03:38:45.697 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_fake_pool_delete": "false", 2024-04-26T03:38:45.697 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_force_quorum_join": "false", 2024-04-26T03:38:45.697 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_globalid_prealloc": "10000", 2024-04-26T03:38:45.697 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_health_detail_to_clog": "false", 2024-04-26T03:38:45.697 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_health_log_update_period": "5", 2024-04-26T03:38:45.698 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_health_max_detail": "50", 2024-04-26T03:38:45.698 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_health_to_clog": "true", 2024-04-26T03:38:45.698 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_health_to_clog_interval": "600", 2024-04-26T03:38:45.698 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-04-26T03:38:45.698 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_host": "172.21.15.28,172.21.15.42,172.21.15.66", 2024-04-26T03:38:45.698 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_host_override": "", 2024-04-26T03:38:45.698 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_initial_members": "", 2024-04-26T03:38:45.699 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-04-26T03:38:45.699 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-04-26T03:38:45.699 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-04-26T03:38:45.699 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-04-26T03:38:45.699 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_keyvaluedb": "rocksdb", 2024-04-26T03:38:45.699 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_lease": "5.000000", 2024-04-26T03:38:45.699 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-04-26T03:38:45.699 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-04-26T03:38:45.700 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_log_full_interval": "50", 2024-04-26T03:38:45.700 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_log_max": "10000", 2024-04-26T03:38:45.700 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_log_max_summary": "50", 2024-04-26T03:38:45.700 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_max_log_entries_per_event": "4096", 2024-04-26T03:38:45.700 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_max_log_epochs": "500", 2024-04-26T03:38:45.700 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_max_mdsmap_epochs": "500", 2024-04-26T03:38:45.700 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_max_mgrmap_epochs": "500", 2024-04-26T03:38:45.701 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_max_osd": "10000", 2024-04-26T03:38:45.701 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_max_pg_per_osd": "10000", 2024-04-26T03:38:45.701 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_max_pool_pg_num": "65536", 2024-04-26T03:38:45.701 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-04-26T03:38:45.701 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-04-26T03:38:45.701 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_mds_force_trim_to": "0", 2024-04-26T03:38:45.701 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_mds_skip_sanity": "false", 2024-04-26T03:38:45.702 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_memory_autotune": "true", 2024-04-26T03:38:45.702 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_memory_target": "2147483648", 2024-04-26T03:38:45.702 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_mgr_beacon_grace": "30", 2024-04-26T03:38:45.702 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-04-26T03:38:45.702 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_mgr_digest_period": "5", 2024-04-26T03:38:45.702 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_mgr_inactive_grace": "60", 2024-04-26T03:38:45.702 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_mgr_mkfs_grace": "240", 2024-04-26T03:38:45.703 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-04-26T03:38:45.703 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_min_osdmap_epochs": "500", 2024-04-26T03:38:45.703 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_op_complaint_time": "30", 2024-04-26T03:38:45.703 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_op_history_duration": "600", 2024-04-26T03:38:45.703 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_op_history_size": "20", 2024-04-26T03:38:45.703 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_op_history_slow_op_size": "20", 2024-04-26T03:38:45.703 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_op_history_slow_op_threshold": "10", 2024-04-26T03:38:45.703 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_op_log_threshold": "5", 2024-04-26T03:38:45.704 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-04-26T03:38:45.704 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-04-26T03:38:45.704 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-04-26T03:38:45.704 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_osd_auto_mark_in": "false", 2024-04-26T03:38:45.704 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_osd_auto_mark_new_in": "true", 2024-04-26T03:38:45.704 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-04-26T03:38:45.704 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-04-26T03:38:45.705 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_osd_cache_size": "500", 2024-04-26T03:38:45.705 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_osd_cache_size_min": "134217728", 2024-04-26T03:38:45.705 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_osd_crush_smoke_test": "true", 2024-04-26T03:38:45.705 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_osd_destroyed_out_interval": "600", 2024-04-26T03:38:45.705 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_osd_down_out_interval": "600", 2024-04-26T03:38:45.705 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-04-26T03:38:45.705 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-04-26T03:38:45.705 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_osd_force_trim_to": "0", 2024-04-26T03:38:45.706 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_osd_full_ratio": "0.950000", 2024-04-26T03:38:45.706 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-04-26T03:38:45.706 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_osd_laggy_halflife": "3600", 2024-04-26T03:38:45.706 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_osd_laggy_max_interval": "300", 2024-04-26T03:38:45.706 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_osd_laggy_weight": "0.300000", 2024-04-26T03:38:45.706 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-04-26T03:38:45.706 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_osd_max_creating_pgs": "1024", 2024-04-26T03:38:45.707 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_osd_max_initial_pgs": "1024", 2024-04-26T03:38:45.707 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_osd_min_down_reporters": "2", 2024-04-26T03:38:45.707 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-04-26T03:38:45.707 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-04-26T03:38:45.707 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-04-26T03:38:45.707 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_osd_prime_pg_temp": "true", 2024-04-26T03:38:45.707 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-04-26T03:38:45.708 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-04-26T03:38:45.708 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_osd_report_timeout": "900", 2024-04-26T03:38:45.708 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-04-26T03:38:45.708 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-04-26T03:38:45.708 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_osd_warn_num_repaired": "10", 2024-04-26T03:38:45.708 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_osd_warn_op_age": "32.000000", 2024-04-26T03:38:45.708 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-04-26T03:38:45.709 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_osdmap_full_prune_interval": "10", 2024-04-26T03:38:45.709 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_osdmap_full_prune_min": "10000", 2024-04-26T03:38:45.709 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-04-26T03:38:45.709 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-04-26T03:38:45.709 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_pg_stuck_threshold": "60", 2024-04-26T03:38:45.709 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-04-26T03:38:45.709 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_pg_warn_min_objects": "10000", 2024-04-26T03:38:45.709 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_pg_warn_min_per_osd": "0", 2024-04-26T03:38:45.710 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-04-26T03:38:45.710 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_pool_quota_crit_threshold": "0", 2024-04-26T03:38:45.710 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_pool_quota_warn_threshold": "0", 2024-04-26T03:38:45.710 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_probe_timeout": "2.000000", 2024-04-26T03:38:45.710 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_reweight_max_change": "0.050000", 2024-04-26T03:38:45.710 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_reweight_max_osds": "4", 2024-04-26T03:38:45.710 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-04-26T03:38:45.711 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-04-26T03:38:45.711 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-04-26T03:38:45.711 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-04-26T03:38:45.711 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-04-26T03:38:45.711 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_scrub_interval": "300", 2024-04-26T03:38:45.711 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_scrub_max_keys": "100", 2024-04-26T03:38:45.711 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_scrub_timeout": "300", 2024-04-26T03:38:45.712 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_session_timeout": "300", 2024-04-26T03:38:45.712 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_smart_report_timeout": "5", 2024-04-26T03:38:45.712 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_stat_smooth_intervals": "6", 2024-04-26T03:38:45.712 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-04-26T03:38:45.712 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_stretch_pool_min_size": "2", 2024-04-26T03:38:45.712 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_stretch_pool_size": "4", 2024-04-26T03:38:45.712 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-04-26T03:38:45.713 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_subscribe_interval": "86400.000000", 2024-04-26T03:38:45.713 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_sync_debug": "false", 2024-04-26T03:38:45.713 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_sync_max_payload_keys": "2000", 2024-04-26T03:38:45.713 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_sync_max_payload_size": "1048576", 2024-04-26T03:38:45.713 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_sync_provider_kill_at": "0", 2024-04-26T03:38:45.713 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_sync_requester_kill_at": "0", 2024-04-26T03:38:45.713 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_sync_timeout": "60.000000", 2024-04-26T03:38:45.713 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_target_pg_per_osd": "100", 2024-04-26T03:38:45.714 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_tick_interval": "5", 2024-04-26T03:38:45.714 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_timecheck_interval": "300.000000", 2024-04-26T03:38:45.714 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-04-26T03:38:45.714 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_use_min_delay_socket": "false", 2024-04-26T03:38:45.714 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_warn_older_version_delay": "604800", 2024-04-26T03:38:45.714 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-04-26T03:38:45.714 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-04-26T03:38:45.714 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-04-26T03:38:45.714 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_warn_on_filestore_osds": "false", 2024-04-26T03:38:45.714 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-04-26T03:38:45.714 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-04-26T03:38:45.715 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-04-26T03:38:45.715 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_warn_on_misplaced": "false", 2024-04-26T03:38:45.715 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-04-26T03:38:45.715 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_warn_on_older_version": "true", 2024-04-26T03:38:45.715 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-04-26T03:38:45.715 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_warn_on_pool_no_app": "true", 2024-04-26T03:38:45.715 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-04-26T03:38:45.715 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-04-26T03:38:45.715 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-04-26T03:38:45.715 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-04-26T03:38:45.715 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_warn_on_too_few_osds": "false", 2024-04-26T03:38:45.715 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-04-26T03:38:45.715 INFO:teuthology.orchestra.run.smithi028.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-04-26T03:38:45.715 INFO:teuthology.orchestra.run.smithi028.stdout: "monmap": "", 2024-04-26T03:38:45.716 INFO:teuthology.orchestra.run.smithi028.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-04-26T03:38:45.716 INFO:teuthology.orchestra.run.smithi028.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-04-26T03:38:45.716 INFO:teuthology.orchestra.run.smithi028.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-04-26T03:38:45.716 INFO:teuthology.orchestra.run.smithi028.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-04-26T03:38:45.716 INFO:teuthology.orchestra.run.smithi028.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-04-26T03:38:45.716 INFO:teuthology.orchestra.run.smithi028.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-04-26T03:38:45.716 INFO:teuthology.orchestra.run.smithi028.stdout: "motr_tracing_enabled": "false", 2024-04-26T03:38:45.716 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_async_op_threads": "3", 2024-04-26T03:38:45.716 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_async_rdma_buffer_size": "131072", 2024-04-26T03:38:45.716 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_async_rdma_cm": "false", 2024-04-26T03:38:45.716 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_async_rdma_device_name": "", 2024-04-26T03:38:45.716 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_async_rdma_dscp": "96", 2024-04-26T03:38:45.716 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-04-26T03:38:45.716 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_async_rdma_gid_idx": "0", 2024-04-26T03:38:45.716 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_async_rdma_local_gid": "", 2024-04-26T03:38:45.717 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_async_rdma_polling_us": "1000", 2024-04-26T03:38:45.717 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_async_rdma_port_num": "1", 2024-04-26T03:38:45.717 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-04-26T03:38:45.717 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-04-26T03:38:45.717 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_async_rdma_roce_ver": "1", 2024-04-26T03:38:45.717 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_async_rdma_send_buffers": "1024", 2024-04-26T03:38:45.717 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_async_rdma_sl": "3", 2024-04-26T03:38:45.717 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_async_rdma_support_srq": "true", 2024-04-26T03:38:45.717 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_async_rdma_type": "ib", 2024-04-26T03:38:45.717 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_async_reap_threshold": "5", 2024-04-26T03:38:45.717 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_bind_before_connect": "false", 2024-04-26T03:38:45.717 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_bind_ipv4": "true", 2024-04-26T03:38:45.717 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_bind_ipv6": "false", 2024-04-26T03:38:45.717 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_bind_msgr1": "true", 2024-04-26T03:38:45.717 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_bind_msgr2": "true", 2024-04-26T03:38:45.718 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_bind_port_max": "7568", 2024-04-26T03:38:45.718 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_bind_port_min": "6800", 2024-04-26T03:38:45.718 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_bind_prefer_ipv4": "false", 2024-04-26T03:38:45.718 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_bind_retry_count": "3", 2024-04-26T03:38:45.718 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_bind_retry_delay": "5", 2024-04-26T03:38:45.718 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_blackhole_client": "false", 2024-04-26T03:38:45.718 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_blackhole_mds": "false", 2024-04-26T03:38:45.718 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_blackhole_mgr": "false", 2024-04-26T03:38:45.718 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_blackhole_mon": "false", 2024-04-26T03:38:45.718 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_blackhole_osd": "false", 2024-04-26T03:38:45.719 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_client_mode": "crc secure", 2024-04-26T03:38:45.719 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_cluster_mode": "crc secure", 2024-04-26T03:38:45.719 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_cluster_type": "", 2024-04-26T03:38:45.719 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_compress_secure": "false", 2024-04-26T03:38:45.719 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_connection_idle_timeout": "900", 2024-04-26T03:38:45.719 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_connection_ready_timeout": "10", 2024-04-26T03:38:45.719 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_crc_data": "true", 2024-04-26T03:38:45.719 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_crc_header": "true", 2024-04-26T03:38:45.719 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_die_on_bad_msg": "false", 2024-04-26T03:38:45.719 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_die_on_bug": "true", 2024-04-26T03:38:45.720 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_die_on_old_message": "true", 2024-04-26T03:38:45.720 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_die_on_skipped_message": "false", 2024-04-26T03:38:45.720 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_die_on_unhandled_msg": "false", 2024-04-26T03:38:45.720 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-04-26T03:38:45.720 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_dpdk_coremask": "0xF", 2024-04-26T03:38:45.720 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-04-26T03:38:45.720 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_dpdk_devs_allowlist": "", 2024-04-26T03:38:45.720 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_dpdk_enable_tso": "true", 2024-04-26T03:38:45.720 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-04-26T03:38:45.720 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-04-26T03:38:45.720 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_dpdk_hugepages": "", 2024-04-26T03:38:45.720 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_dpdk_hw_flow_control": "true", 2024-04-26T03:38:45.720 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-04-26T03:38:45.721 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_dpdk_lro": "true", 2024-04-26T03:38:45.721 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_dpdk_memory_channel": "4", 2024-04-26T03:38:45.721 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-04-26T03:38:45.721 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_dpdk_pmd": "", 2024-04-26T03:38:45.721 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_dpdk_port_id": "0", 2024-04-26T03:38:45.721 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-04-26T03:38:45.721 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_dump_corrupt_message_level": "1", 2024-04-26T03:38:45.721 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_dump_on_send": "false", 2024-04-26T03:38:45.721 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_initial_backoff": "0.200000", 2024-04-26T03:38:45.721 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_inject_delay_max": "1.000000", 2024-04-26T03:38:45.721 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_inject_delay_probability": "0.000000", 2024-04-26T03:38:45.721 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_inject_delay_type": "", 2024-04-26T03:38:45.722 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_inject_internal_delays": "0.000000", 2024-04-26T03:38:45.722 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_inject_network_congestion": "0", 2024-04-26T03:38:45.722 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_inject_socket_failures": "2500", 2024-04-26T03:38:45.722 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_learn_addr_from_peer": "true", 2024-04-26T03:38:45.722 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_max_accept_failures": "4", 2024-04-26T03:38:45.722 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_max_backoff": "15.000000", 2024-04-26T03:38:45.722 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_mon_client_mode": "secure crc", 2024-04-26T03:38:45.722 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_mon_cluster_mode": "secure crc", 2024-04-26T03:38:45.722 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_mon_service_mode": "secure crc", 2024-04-26T03:38:45.722 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_osd_compress_min_size": "1024", 2024-04-26T03:38:45.723 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_osd_compress_mode": "none", 2024-04-26T03:38:45.723 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_osd_compression_algorithm": "snappy", 2024-04-26T03:38:45.723 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-04-26T03:38:45.723 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_pq_min_cost": "65536", 2024-04-26T03:38:45.723 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_public_type": "", 2024-04-26T03:38:45.723 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_service_mode": "crc secure", 2024-04-26T03:38:45.723 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_tcp_listen_backlog": "512", 2024-04-26T03:38:45.723 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_tcp_nodelay": "true", 2024-04-26T03:38:45.723 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-04-26T03:38:45.723 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_tcp_rcvbuf": "0", 2024-04-26T03:38:45.723 INFO:teuthology.orchestra.run.smithi028.stdout: "ms_type": "async+posix", 2024-04-26T03:38:45.723 INFO:teuthology.orchestra.run.smithi028.stdout: "no_config_file": "false", 2024-04-26T03:38:45.723 INFO:teuthology.orchestra.run.smithi028.stdout: "objecter_completion_locks_per_session": "32", 2024-04-26T03:38:45.723 INFO:teuthology.orchestra.run.smithi028.stdout: "objecter_debug_inject_relock_delay": "false", 2024-04-26T03:38:45.723 INFO:teuthology.orchestra.run.smithi028.stdout: "objecter_inflight_op_bytes": "104857600", 2024-04-26T03:38:45.724 INFO:teuthology.orchestra.run.smithi028.stdout: "objecter_inflight_ops": "1024", 2024-04-26T03:38:45.724 INFO:teuthology.orchestra.run.smithi028.stdout: "objecter_inject_no_watch_ping": "false", 2024-04-26T03:38:45.724 INFO:teuthology.orchestra.run.smithi028.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-04-26T03:38:45.724 INFO:teuthology.orchestra.run.smithi028.stdout: "objecter_tick_interval": "5.000000", 2024-04-26T03:38:45.724 INFO:teuthology.orchestra.run.smithi028.stdout: "objecter_timeout": "10.000000", 2024-04-26T03:38:45.724 INFO:teuthology.orchestra.run.smithi028.stdout: "objectstore_blackhole": "false", 2024-04-26T03:38:45.724 INFO:teuthology.orchestra.run.smithi028.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-04-26T03:38:45.724 INFO:teuthology.orchestra.run.smithi028.stdout: "openssl_engine_opts": "", 2024-04-26T03:38:45.724 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_agent_delay_time": "5.000000", 2024-04-26T03:38:45.724 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_agent_hist_halflife": "1000", 2024-04-26T03:38:45.724 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_agent_max_low_ops": "2", 2024-04-26T03:38:45.724 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_agent_max_ops": "4", 2024-04-26T03:38:45.724 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-04-26T03:38:45.724 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_agent_quantize_effort": "0.100000", 2024-04-26T03:38:45.725 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_agent_slop": "0.020000", 2024-04-26T03:38:45.725 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-04-26T03:38:45.725 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_allow_recovery_below_min_size": "true", 2024-04-26T03:38:45.725 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_asio_thread_count": "2", 2024-04-26T03:38:45.725 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_async_recovery_min_cost": "100", 2024-04-26T03:38:45.725 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_auto_mark_unfound_lost": "false", 2024-04-26T03:38:45.725 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_backfill_retry_interval": "30.000000", 2024-04-26T03:38:45.725 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_backfill_scan_max": "512", 2024-04-26T03:38:45.725 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_backfill_scan_min": "64", 2024-04-26T03:38:45.725 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_backoff_on_degraded": "false", 2024-04-26T03:38:45.725 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_backoff_on_peering": "false", 2024-04-26T03:38:45.725 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_backoff_on_unfound": "true", 2024-04-26T03:38:45.725 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_beacon_report_interval": "300", 2024-04-26T03:38:45.725 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_bench_duration": "30", 2024-04-26T03:38:45.726 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-04-26T03:38:45.726 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_bench_max_block_size": "67108864", 2024-04-26T03:38:45.726 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_bench_small_size_max_iops": "100", 2024-04-26T03:38:45.726 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_blkin_trace_all": "false", 2024-04-26T03:38:45.726 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_blocked_scrub_grace_period": "120", 2024-04-26T03:38:45.726 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-04-26T03:38:45.726 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-04-26T03:38:45.726 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-04-26T03:38:45.726 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_check_for_log_corruption": "false", 2024-04-26T03:38:45.726 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-04-26T03:38:45.726 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-04-26T03:38:45.726 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-04-26T03:38:45.726 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-04-26T03:38:45.726 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_class_update_on_start": "true", 2024-04-26T03:38:45.727 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_client_message_cap": "256", 2024-04-26T03:38:45.727 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_client_message_size_cap": "524288000", 2024-04-26T03:38:45.727 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_client_op_priority": "63", 2024-04-26T03:38:45.727 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_client_watch_timeout": "30", 2024-04-26T03:38:45.727 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_command_max_records": "256", 2024-04-26T03:38:45.727 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_compact_on_start": "false", 2024-04-26T03:38:45.727 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-04-26T03:38:45.727 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_crush_chooseleaf_type": "0", 2024-04-26T03:38:45.727 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_crush_initial_weight": "-1.000000", 2024-04-26T03:38:45.727 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_crush_update_on_start": "true", 2024-04-26T03:38:45.727 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_crush_update_weight_set": "true", 2024-04-26T03:38:45.727 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-04-26T03:38:45.727 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-04-26T03:38:45.727 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-04-26T03:38:45.728 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_debug_drop_ping_duration": "0", 2024-04-26T03:38:45.728 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-04-26T03:38:45.728 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_debug_feed_pullee": "-1", 2024-04-26T03:38:45.728 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-04-26T03:38:45.728 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-04-26T03:38:45.728 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-04-26T03:38:45.728 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_debug_misdirected_ops": "false", 2024-04-26T03:38:45.728 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_debug_no_acting_change": "false", 2024-04-26T03:38:45.728 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_debug_no_purge_strays": "false", 2024-04-26T03:38:45.728 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_debug_op_order": "false", 2024-04-26T03:38:45.728 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_debug_pg_log_writeout": "false", 2024-04-26T03:38:45.728 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_debug_pretend_recovery_active": "false", 2024-04-26T03:38:45.728 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-04-26T03:38:45.728 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-04-26T03:38:45.729 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_debug_shutdown": "false", 2024-04-26T03:38:45.729 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-04-26T03:38:45.729 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-04-26T03:38:45.729 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_debug_trim_objects": "false", 2024-04-26T03:38:45.729 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_debug_verify_cached_snaps": "false", 2024-04-26T03:38:45.729 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_debug_verify_missing_on_start": "false", 2024-04-26T03:38:45.729 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_debug_verify_snaps": "false", 2024-04-26T03:38:45.729 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-04-26T03:38:45.729 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-04-26T03:38:45.729 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_deep_scrub_keys": "1024", 2024-04-26T03:38:45.729 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-04-26T03:38:45.729 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-04-26T03:38:45.729 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-04-26T03:38:45.729 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_deep_scrub_stride": "524288", 2024-04-26T03:38:45.730 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-04-26T03:38:45.730 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_default_data_pool_replay_window": "5", 2024-04-26T03:38:45.730 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_default_notify_timeout": "30", 2024-04-26T03:38:45.730 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_delete_sleep": "0.000000", 2024-04-26T03:38:45.730 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-04-26T03:38:45.730 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-04-26T03:38:45.730 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-04-26T03:38:45.730 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_discard_disconnected_ops": "true", 2024-04-26T03:38:45.730 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_enable_op_tracker": "true", 2024-04-26T03:38:45.730 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-04-26T03:38:45.730 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_extblkdev_plugins": "vdo", 2024-04-26T03:38:45.730 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-04-26T03:38:45.731 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-04-26T03:38:45.731 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_fast_info": "true", 2024-04-26T03:38:45.731 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_fast_shutdown": "true", 2024-04-26T03:38:45.731 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-04-26T03:38:45.731 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_fast_shutdown_timeout": "15", 2024-04-26T03:38:45.731 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-04-26T03:38:45.731 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-04-26T03:38:45.731 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-04-26T03:38:45.732 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_function_tracing": "false", 2024-04-26T03:38:45.732 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_heartbeat_grace": "20", 2024-04-26T03:38:45.732 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_heartbeat_interval": "6", 2024-04-26T03:38:45.732 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-04-26T03:38:45.732 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_heartbeat_min_peers": "10", 2024-04-26T03:38:45.732 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_heartbeat_min_size": "2000", 2024-04-26T03:38:45.732 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_heartbeat_stale": "600", 2024-04-26T03:38:45.733 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-04-26T03:38:45.733 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_hit_set_max_size": "100000", 2024-04-26T03:38:45.733 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_hit_set_min_size": "1000", 2024-04-26T03:38:45.733 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-04-26T03:38:45.733 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-04-26T03:38:45.733 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-04-26T03:38:45.733 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-04-26T03:38:45.734 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-04-26T03:38:45.734 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_journal_flush_on_shutdown": "true", 2024-04-26T03:38:45.734 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_journal_size": "5120", 2024-04-26T03:38:45.734 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_kill_backfill_at": "0", 2024-04-26T03:38:45.734 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_loop_before_reset_tphandle": "64", 2024-04-26T03:38:45.735 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_map_cache_size": "50", 2024-04-26T03:38:45.735 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_map_dedup": "true", 2024-04-26T03:38:45.735 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_map_message_max": "40", 2024-04-26T03:38:45.735 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_map_message_max_bytes": "10485760", 2024-04-26T03:38:45.735 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_map_share_max_epochs": "40", 2024-04-26T03:38:45.735 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_max_attr_name_len": "100", 2024-04-26T03:38:45.735 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_max_attr_size": "0", 2024-04-26T03:38:45.736 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_max_backfills": "1", 2024-04-26T03:38:45.736 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_max_markdown_count": "5", 2024-04-26T03:38:45.736 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_max_markdown_period": "600", 2024-04-26T03:38:45.736 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_max_object_name_len": "2048", 2024-04-26T03:38:45.736 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_max_object_namespace_len": "256", 2024-04-26T03:38:45.736 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_max_object_size": "134217728", 2024-04-26T03:38:45.737 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-04-26T03:38:45.737 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_max_omap_entries_per_request": "1024", 2024-04-26T03:38:45.737 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_max_pg_blocked_by": "16", 2024-04-26T03:38:45.737 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_max_pg_log_entries": "10000", 2024-04-26T03:38:45.737 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-04-26T03:38:45.737 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_max_pgls": "1024", 2024-04-26T03:38:45.737 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_max_push_cost": "8388608", 2024-04-26T03:38:45.738 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_max_push_objects": "10", 2024-04-26T03:38:45.738 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_max_scrubs": "3", 2024-04-26T03:38:45.738 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-04-26T03:38:45.738 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_max_trimming_pgs": "2", 2024-04-26T03:38:45.738 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_max_write_op_reply_len": "64", 2024-04-26T03:38:45.738 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_max_write_size": "90", 2024-04-26T03:38:45.739 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-04-26T03:38:45.739 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-04-26T03:38:45.739 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-04-26T03:38:45.739 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-04-26T03:38:45.739 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-04-26T03:38:45.739 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-04-26T03:38:45.740 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-04-26T03:38:45.740 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_mclock_override_recovery_settings": "false", 2024-04-26T03:38:45.740 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_mclock_profile": "balanced", 2024-04-26T03:38:45.740 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-04-26T03:38:45.740 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-04-26T03:38:45.740 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-04-26T03:38:45.740 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-04-26T03:38:45.741 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-04-26T03:38:45.741 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-04-26T03:38:45.741 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-04-26T03:38:45.741 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-04-26T03:38:45.741 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-04-26T03:38:45.741 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-04-26T03:38:45.741 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_mclock_skip_benchmark": "false", 2024-04-26T03:38:45.741 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_memory_base": "805306368", 2024-04-26T03:38:45.741 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_memory_cache_min": "134217728", 2024-04-26T03:38:45.741 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-04-26T03:38:45.741 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-04-26T03:38:45.741 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_memory_target": "4294967296", 2024-04-26T03:38:45.741 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_memory_target_autotune": "false", 2024-04-26T03:38:45.741 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-04-26T03:38:45.742 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_min_pg_log_entries": "250", 2024-04-26T03:38:45.742 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_min_recovery_priority": "0", 2024-04-26T03:38:45.742 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_mon_heartbeat_interval": "30", 2024-04-26T03:38:45.742 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-04-26T03:38:45.742 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_mon_report_interval": "5", 2024-04-26T03:38:45.742 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_mon_report_max_in_flight": "2", 2024-04-26T03:38:45.742 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-04-26T03:38:45.742 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_num_cache_shards": "32", 2024-04-26T03:38:45.742 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_num_op_tracker_shard": "32", 2024-04-26T03:38:45.742 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_numa_auto_affinity": "true", 2024-04-26T03:38:45.742 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_numa_node": "-1", 2024-04-26T03:38:45.742 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_numa_prefer_iface": "true", 2024-04-26T03:38:45.742 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-04-26T03:38:45.742 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_objecter_finishers": "1", 2024-04-26T03:38:45.743 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_objectstore": "bluestore", 2024-04-26T03:38:45.743 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_objectstore_fuse": "false", 2024-04-26T03:38:45.743 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_objectstore_tracing": "false", 2024-04-26T03:38:45.743 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_op_complaint_time": "30.000000", 2024-04-26T03:38:45.743 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_op_history_duration": "600", 2024-04-26T03:38:45.743 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_op_history_size": "20", 2024-04-26T03:38:45.743 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_op_history_slow_op_size": "20", 2024-04-26T03:38:45.743 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-04-26T03:38:45.743 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_op_log_threshold": "5", 2024-04-26T03:38:45.743 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_op_num_shards": "0", 2024-04-26T03:38:45.743 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_op_num_shards_hdd": "5", 2024-04-26T03:38:45.743 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_op_num_shards_ssd": "8", 2024-04-26T03:38:45.743 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_op_num_threads_per_shard": "0", 2024-04-26T03:38:45.743 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-04-26T03:38:45.743 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-04-26T03:38:45.744 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-04-26T03:38:45.744 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_op_pq_min_cost": "65536", 2024-04-26T03:38:45.744 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_op_queue": "mclock_scheduler", 2024-04-26T03:38:45.744 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_op_queue_cut_off": "high", 2024-04-26T03:38:45.744 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_op_thread_suicide_timeout": "150", 2024-04-26T03:38:45.744 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_op_thread_timeout": "15", 2024-04-26T03:38:45.744 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_open_classes_on_start": "true", 2024-04-26T03:38:45.744 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_os_flags": "0", 2024-04-26T03:38:45.744 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_peering_op_priority": "255", 2024-04-26T03:38:45.744 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_pg_delete_cost": "1048576", 2024-04-26T03:38:45.744 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_pg_delete_priority": "5", 2024-04-26T03:38:45.744 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-04-26T03:38:45.744 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-04-26T03:38:45.744 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_pg_log_dups_tracked": "3000", 2024-04-26T03:38:45.745 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_pg_log_trim_max": "10000", 2024-04-26T03:38:45.745 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_pg_log_trim_min": "100", 2024-04-26T03:38:45.745 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-04-26T03:38:45.745 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_pg_object_context_cache_count": "64", 2024-04-26T03:38:45.745 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-04-26T03:38:45.745 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-04-26T03:38:45.745 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-04-26T03:38:45.745 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-04-26T03:38:45.745 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-04-26T03:38:45.745 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-04-26T03:38:45.745 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-04-26T03:38:45.745 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-04-26T03:38:45.745 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_pool_default_crimson": "false", 2024-04-26T03:38:45.745 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_pool_default_crush_rule": "-1", 2024-04-26T03:38:45.745 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_pool_default_ec_fast_read": "false", 2024-04-26T03:38:45.746 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-04-26T03:38:45.746 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_pool_default_flag_bulk": "false", 2024-04-26T03:38:45.746 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-04-26T03:38:45.746 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_pool_default_flag_nodelete": "false", 2024-04-26T03:38:45.746 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-04-26T03:38:45.746 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-04-26T03:38:45.746 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_pool_default_flags": "0", 2024-04-26T03:38:45.746 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-04-26T03:38:45.746 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_pool_default_min_size": "0", 2024-04-26T03:38:45.746 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-04-26T03:38:45.746 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_pool_default_pg_num": "32", 2024-04-26T03:38:45.746 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_pool_default_pgp_num": "0", 2024-04-26T03:38:45.746 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-04-26T03:38:45.746 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_pool_default_read_ratio": "70", 2024-04-26T03:38:45.747 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_pool_default_size": "2", 2024-04-26T03:38:45.747 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_pool_default_type": "replicated", 2024-04-26T03:38:45.747 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-04-26T03:38:45.747 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_pool_use_gmt_hitset": "true", 2024-04-26T03:38:45.747 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_push_per_object_cost": "1000", 2024-04-26T03:38:45.747 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_read_ec_check_for_errors": "false", 2024-04-26T03:38:45.747 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_recover_clone_overlap": "true", 2024-04-26T03:38:45.747 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_recover_clone_overlap_limit": "10", 2024-04-26T03:38:45.747 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_recovery_cost": "20971520", 2024-04-26T03:38:45.747 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_recovery_delay_start": "0.000000", 2024-04-26T03:38:45.747 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_recovery_max_active": "0", 2024-04-26T03:38:45.747 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_recovery_max_active_hdd": "3", 2024-04-26T03:38:45.747 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_recovery_max_active_ssd": "10", 2024-04-26T03:38:45.747 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_recovery_max_chunk": "8388608", 2024-04-26T03:38:45.748 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-04-26T03:38:45.748 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_recovery_max_single_start": "1", 2024-04-26T03:38:45.748 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_recovery_op_priority": "3", 2024-04-26T03:38:45.748 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_recovery_op_warn_multiple": "16", 2024-04-26T03:38:45.748 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_recovery_priority": "5", 2024-04-26T03:38:45.748 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_recovery_retry_interval": "30.000000", 2024-04-26T03:38:45.748 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_recovery_sleep": "0.000000", 2024-04-26T03:38:45.748 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-04-26T03:38:45.748 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-04-26T03:38:45.748 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-04-26T03:38:45.748 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_repair_during_recovery": "false", 2024-04-26T03:38:45.748 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_requested_scrub_priority": "120", 2024-04-26T03:38:45.748 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-04-26T03:38:45.748 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_rollback_to_cluster_snap": "", 2024-04-26T03:38:45.748 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_scrub_auto_repair": "false", 2024-04-26T03:38:45.749 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-04-26T03:38:45.749 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-04-26T03:38:45.749 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_scrub_begin_hour": "0", 2024-04-26T03:38:45.749 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_scrub_begin_week_day": "0", 2024-04-26T03:38:45.749 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_scrub_chunk_max": "25", 2024-04-26T03:38:45.749 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_scrub_chunk_min": "5", 2024-04-26T03:38:45.749 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_scrub_cost": "52428800", 2024-04-26T03:38:45.749 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_scrub_disable_reservation_queuing": "true", 2024-04-26T03:38:45.749 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_scrub_during_recovery": "false", 2024-04-26T03:38:45.749 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_scrub_end_hour": "0", 2024-04-26T03:38:45.749 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_scrub_end_week_day": "0", 2024-04-26T03:38:45.749 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_scrub_event_cost": "4096", 2024-04-26T03:38:45.749 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-04-26T03:38:45.749 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-04-26T03:38:45.749 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_scrub_invalid_stats": "true", 2024-04-26T03:38:45.750 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_scrub_load_threshold": "0.500000", 2024-04-26T03:38:45.750 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_scrub_max_interval": "604800.000000", 2024-04-26T03:38:45.750 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_scrub_max_preemptions": "5", 2024-04-26T03:38:45.750 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_scrub_min_interval": "86400.000000", 2024-04-26T03:38:45.750 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_scrub_priority": "5", 2024-04-26T03:38:45.750 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_scrub_sleep": "0.000000", 2024-04-26T03:38:45.750 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-04-26T03:38:45.750 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-04-26T03:38:45.750 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_shutdown_pgref_assert": "false", 2024-04-26T03:38:45.750 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_skip_data_digest": "false", 2024-04-26T03:38:45.750 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_smart_report_timeout": "5", 2024-04-26T03:38:45.750 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_snap_trim_cost": "1048576", 2024-04-26T03:38:45.750 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_snap_trim_priority": "5", 2024-04-26T03:38:45.750 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_snap_trim_sleep": "0.000000", 2024-04-26T03:38:45.751 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-04-26T03:38:45.751 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-04-26T03:38:45.751 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-04-26T03:38:45.751 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-04-26T03:38:45.751 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_stats_update_period_scrubbing": "15", 2024-04-26T03:38:45.751 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-04-26T03:38:45.751 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_target_transaction_size": "30", 2024-04-26T03:38:45.751 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-04-26T03:38:45.751 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-04-26T03:38:45.751 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-04-26T03:38:45.751 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-04-26T03:38:45.751 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-04-26T03:38:45.751 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-04-26T03:38:45.751 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-04-26T03:38:45.751 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_tier_default_cache_mode": "writeback", 2024-04-26T03:38:45.752 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-04-26T03:38:45.752 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-04-26T03:38:45.752 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_tracing": "false", 2024-04-26T03:38:45.752 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_use_stale_snap": "false", 2024-04-26T03:38:45.752 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-04-26T03:38:45.752 INFO:teuthology.orchestra.run.smithi028.stdout: "osdc_blkin_trace_all": "false", 2024-04-26T03:38:45.752 INFO:teuthology.orchestra.run.smithi028.stdout: "paxos_kill_at": "0", 2024-04-26T03:38:45.752 INFO:teuthology.orchestra.run.smithi028.stdout: "paxos_max_join_drift": "10", 2024-04-26T03:38:45.752 INFO:teuthology.orchestra.run.smithi028.stdout: "paxos_min": "500", 2024-04-26T03:38:45.752 INFO:teuthology.orchestra.run.smithi028.stdout: "paxos_min_wait": "0.050000", 2024-04-26T03:38:45.752 INFO:teuthology.orchestra.run.smithi028.stdout: "paxos_propose_interval": "1.000000", 2024-04-26T03:38:45.752 INFO:teuthology.orchestra.run.smithi028.stdout: "paxos_service_trim_max": "500", 2024-04-26T03:38:45.752 INFO:teuthology.orchestra.run.smithi028.stdout: "paxos_service_trim_max_multiplier": "20", 2024-04-26T03:38:45.752 INFO:teuthology.orchestra.run.smithi028.stdout: "paxos_service_trim_min": "250", 2024-04-26T03:38:45.753 INFO:teuthology.orchestra.run.smithi028.stdout: "paxos_stash_full_interval": "25", 2024-04-26T03:38:45.753 INFO:teuthology.orchestra.run.smithi028.stdout: "paxos_trim_max": "500", 2024-04-26T03:38:45.753 INFO:teuthology.orchestra.run.smithi028.stdout: "paxos_trim_min": "250", 2024-04-26T03:38:45.753 INFO:teuthology.orchestra.run.smithi028.stdout: "perf": "true", 2024-04-26T03:38:45.753 INFO:teuthology.orchestra.run.smithi028.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-04-26T03:38:45.753 INFO:teuthology.orchestra.run.smithi028.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-04-26T03:38:45.753 INFO:teuthology.orchestra.run.smithi028.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-04-26T03:38:45.753 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "-", 2024-04-26T03:38:45.753 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrv": "", 2024-04-26T03:38:45.753 INFO:teuthology.orchestra.run.smithi028.stdout: "public_bind_addr": "-", 2024-04-26T03:38:45.753 INFO:teuthology.orchestra.run.smithi028.stdout: "public_network": "", 2024-04-26T03:38:45.753 INFO:teuthology.orchestra.run.smithi028.stdout: "public_network_interface": "", 2024-04-26T03:38:45.753 INFO:teuthology.orchestra.run.smithi028.stdout: "qat_compressor_enabled": "false", 2024-04-26T03:38:45.753 INFO:teuthology.orchestra.run.smithi028.stdout: "qat_compressor_session_max_number": "256", 2024-04-26T03:38:45.754 INFO:teuthology.orchestra.run.smithi028.stdout: "rados_mon_op_timeout": "0", 2024-04-26T03:38:45.754 INFO:teuthology.orchestra.run.smithi028.stdout: "rados_osd_op_timeout": "0", 2024-04-26T03:38:45.754 INFO:teuthology.orchestra.run.smithi028.stdout: "rados_tracing": "false", 2024-04-26T03:38:45.754 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_atime_update_interval": "60", 2024-04-26T03:38:45.754 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-04-26T03:38:45.754 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_balance_parent_reads": "false", 2024-04-26T03:38:45.754 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_balance_snap_reads": "false", 2024-04-26T03:38:45.754 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_blkin_trace_all": "false", 2024-04-26T03:38:45.754 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_blocklist_expire_seconds": "0", 2024-04-26T03:38:45.754 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_blocklist_on_break_lock": "true", 2024-04-26T03:38:45.754 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_cache": "true", 2024-04-26T03:38:45.754 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_cache_block_writes_upfront": "false", 2024-04-26T03:38:45.754 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_cache_max_dirty": "25165824", 2024-04-26T03:38:45.754 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-04-26T03:38:45.754 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_cache_max_dirty_object": "0", 2024-04-26T03:38:45.755 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_cache_policy": "writearound", 2024-04-26T03:38:45.755 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_cache_size": "33554432", 2024-04-26T03:38:45.755 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_cache_target_dirty": "16777216", 2024-04-26T03:38:45.755 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-04-26T03:38:45.755 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_clone_copy_on_read": "false", 2024-04-26T03:38:45.755 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_compression_hint": "none", 2024-04-26T03:38:45.755 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_concurrent_management_ops": "10", 2024-04-26T03:38:45.755 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-04-26T03:38:45.755 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_default_clone_format": "auto", 2024-04-26T03:38:45.755 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_default_data_pool": "", 2024-04-26T03:38:45.755 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_default_features": "61", 2024-04-26T03:38:45.755 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_default_format": "2", 2024-04-26T03:38:45.755 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_default_map_options": "", 2024-04-26T03:38:45.755 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_default_order": "22", 2024-04-26T03:38:45.756 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_default_pool": "rbd", 2024-04-26T03:38:45.756 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-04-26T03:38:45.756 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_default_stripe_count": "0", 2024-04-26T03:38:45.756 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_default_stripe_unit": "0", 2024-04-26T03:38:45.756 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_disable_zero_copy_writes": "true", 2024-04-26T03:38:45.756 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_discard_granularity_bytes": "65536", 2024-04-26T03:38:45.756 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-04-26T03:38:45.756 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_enable_alloc_hint": "true", 2024-04-26T03:38:45.756 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-04-26T03:38:45.756 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_io_scheduler": "simple", 2024-04-26T03:38:45.756 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-04-26T03:38:45.756 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_journal_commit_age": "5.000000", 2024-04-26T03:38:45.756 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-04-26T03:38:45.756 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-04-26T03:38:45.756 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-04-26T03:38:45.757 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-04-26T03:38:45.757 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_journal_object_flush_interval": "0", 2024-04-26T03:38:45.757 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-04-26T03:38:45.757 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-04-26T03:38:45.757 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_journal_order": "24", 2024-04-26T03:38:45.757 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_journal_pool": "", 2024-04-26T03:38:45.757 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_journal_splay_width": "4", 2024-04-26T03:38:45.757 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_localize_parent_reads": "false", 2024-04-26T03:38:45.757 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_localize_snap_reads": "false", 2024-04-26T03:38:45.757 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-04-26T03:38:45.757 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-04-26T03:38:45.757 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-04-26T03:38:45.757 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-04-26T03:38:45.757 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-04-26T03:38:45.757 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-04-26T03:38:45.758 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_mirror_image_policy_type": "simple", 2024-04-26T03:38:45.758 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-04-26T03:38:45.758 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-04-26T03:38:45.758 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-04-26T03:38:45.758 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-04-26T03:38:45.758 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-04-26T03:38:45.758 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-04-26T03:38:45.758 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-04-26T03:38:45.758 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_mirror_memory_autotune": "true", 2024-04-26T03:38:45.758 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_mirror_memory_base": "805306368", 2024-04-26T03:38:45.758 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-04-26T03:38:45.758 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-04-26T03:38:45.758 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-04-26T03:38:45.758 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-04-26T03:38:45.759 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_mirror_memory_target": "4294967296", 2024-04-26T03:38:45.759 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-04-26T03:38:45.759 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-04-26T03:38:45.759 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-04-26T03:38:45.759 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_mirroring_delete_delay": "0", 2024-04-26T03:38:45.759 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-04-26T03:38:45.759 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_mirroring_replay_delay": "0", 2024-04-26T03:38:45.759 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-04-26T03:38:45.759 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-04-26T03:38:45.759 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_move_to_trash_on_remove": "false", 2024-04-26T03:38:45.759 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-04-26T03:38:45.759 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_mtime_update_interval": "60", 2024-04-26T03:38:45.759 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_non_blocking_aio": "true", 2024-04-26T03:38:45.759 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_op_thread_timeout": "60", 2024-04-26T03:38:45.759 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_op_threads": "1", 2024-04-26T03:38:45.760 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_parent_cache_enabled": "false", 2024-04-26T03:38:45.760 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_persistent_cache_mode": "disabled", 2024-04-26T03:38:45.760 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_persistent_cache_path": "/tmp", 2024-04-26T03:38:45.760 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_persistent_cache_size": "1073741824", 2024-04-26T03:38:45.760 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_plugins": "", 2024-04-26T03:38:45.760 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_qos_bps_burst": "0", 2024-04-26T03:38:45.760 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-04-26T03:38:45.760 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_qos_bps_limit": "0", 2024-04-26T03:38:45.760 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_qos_exclude_ops": "0", 2024-04-26T03:38:45.760 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_qos_iops_burst": "0", 2024-04-26T03:38:45.760 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-04-26T03:38:45.760 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_qos_iops_limit": "0", 2024-04-26T03:38:45.760 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_qos_read_bps_burst": "0", 2024-04-26T03:38:45.760 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-04-26T03:38:45.761 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_qos_read_bps_limit": "0", 2024-04-26T03:38:45.761 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_qos_read_iops_burst": "0", 2024-04-26T03:38:45.761 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-04-26T03:38:45.761 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_qos_read_iops_limit": "0", 2024-04-26T03:38:45.761 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_qos_schedule_tick_min": "50", 2024-04-26T03:38:45.761 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_qos_write_bps_burst": "0", 2024-04-26T03:38:45.761 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-04-26T03:38:45.761 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_qos_write_bps_limit": "0", 2024-04-26T03:38:45.761 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_qos_write_iops_burst": "0", 2024-04-26T03:38:45.761 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-04-26T03:38:45.761 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_qos_write_iops_limit": "0", 2024-04-26T03:38:45.761 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_quiesce_notification_attempts": "10", 2024-04-26T03:38:45.761 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_read_from_replica_policy": "default", 2024-04-26T03:38:45.761 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-04-26T03:38:45.761 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_readahead_max_bytes": "524288", 2024-04-26T03:38:45.762 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_readahead_trigger_requests": "10", 2024-04-26T03:38:45.762 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_request_timed_out_seconds": "30", 2024-04-26T03:38:45.762 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_skip_partial_discard": "true", 2024-04-26T03:38:45.762 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-04-26T03:38:45.762 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_tracing": "false", 2024-04-26T03:38:45.762 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_validate_names": "true", 2024-04-26T03:38:45.762 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd_validate_pool": "true", 2024-04-26T03:38:45.762 INFO:teuthology.orchestra.run.smithi028.stdout: "restapi_base_url": "", 2024-04-26T03:38:45.762 INFO:teuthology.orchestra.run.smithi028.stdout: "restapi_log_level": "", 2024-04-26T03:38:45.762 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_acl_grants_max_num": "100", 2024-04-26T03:38:45.762 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_admin_entry": "admin", 2024-04-26T03:38:45.762 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-04-26T03:38:45.762 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_backend_store": "rados", 2024-04-26T03:38:45.762 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_barbican_url": "", 2024-04-26T03:38:45.763 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_beast_enable_async": "true", 2024-04-26T03:38:45.763 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_bucket_counters_cache": "false", 2024-04-26T03:38:45.763 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-04-26T03:38:45.763 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-04-26T03:38:45.763 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-04-26T03:38:45.763 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_bucket_index_max_aio": "128", 2024-04-26T03:38:45.763 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-04-26T03:38:45.763 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-04-26T03:38:45.763 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_bucket_quota_ttl": "600", 2024-04-26T03:38:45.763 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-04-26T03:38:45.763 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_cache_enabled": "true", 2024-04-26T03:38:45.763 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_cache_expiry_interval": "900", 2024-04-26T03:38:45.763 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_cache_lru_size": "10000", 2024-04-26T03:38:45.763 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_config_store": "rados", 2024-04-26T03:38:45.763 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_content_length_compat": "false", 2024-04-26T03:38:45.764 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_copy_obj_progress": "true", 2024-04-26T03:38:45.764 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-04-26T03:38:45.764 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_cors_rules_max_num": "100", 2024-04-26T03:38:45.764 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_cross_domain_policy": "", 2024-04-26T03:38:45.764 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_crypt_default_encryption_key": "", 2024-04-26T03:38:45.764 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_crypt_kmip_addr": "", 2024-04-26T03:38:45.764 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_crypt_kmip_ca_path": "", 2024-04-26T03:38:45.764 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_crypt_kmip_client_cert": "", 2024-04-26T03:38:45.764 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_crypt_kmip_client_key": "", 2024-04-26T03:38:45.764 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-04-26T03:38:45.764 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_crypt_kmip_password": "", 2024-04-26T03:38:45.764 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-04-26T03:38:45.764 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_crypt_kmip_username": "", 2024-04-26T03:38:45.764 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_crypt_require_ssl": "true", 2024-04-26T03:38:45.765 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-04-26T03:38:45.765 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-04-26T03:38:45.765 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-04-26T03:38:45.765 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-04-26T03:38:45.765 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-04-26T03:38:45.765 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-04-26T03:38:45.765 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-04-26T03:38:45.765 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-04-26T03:38:45.765 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-04-26T03:38:45.765 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-04-26T03:38:45.765 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-04-26T03:38:45.765 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-04-26T03:38:45.765 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-04-26T03:38:45.765 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-04-26T03:38:45.765 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_crypt_suppress_logs": "true", 2024-04-26T03:38:45.766 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_crypt_vault_addr": "", 2024-04-26T03:38:45.766 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_crypt_vault_auth": "token", 2024-04-26T03:38:45.766 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_crypt_vault_namespace": "", 2024-04-26T03:38:45.766 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_crypt_vault_prefix": "", 2024-04-26T03:38:45.766 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-04-26T03:38:45.766 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-04-26T03:38:45.766 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-04-26T03:38:45.766 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-04-26T03:38:45.766 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_crypt_vault_token_file": "", 2024-04-26T03:38:45.766 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-04-26T03:38:45.766 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_curl_buffersize": "524288", 2024-04-26T03:38:45.766 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_curl_low_speed_limit": "1024", 2024-04-26T03:38:45.766 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_curl_low_speed_time": "300", 2024-04-26T03:38:45.766 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_curl_tcp_keepalive": "0", 2024-04-26T03:38:45.767 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-04-26T03:38:45.767 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-04-26T03:38:45.767 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-04-26T03:38:45.767 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-04-26T03:38:45.767 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-04-26T03:38:45.767 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_d3n_l1_fadvise": "4", 2024-04-26T03:38:45.767 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-04-26T03:38:45.767 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-04-26T03:38:45.767 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-04-26T03:38:45.767 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_d4n_host": "127.0.0.1", 2024-04-26T03:38:45.767 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_d4n_port": "6379", 2024-04-26T03:38:45.767 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-04-26T03:38:45.767 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_data_log_changes_size": "1000", 2024-04-26T03:38:45.767 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_data_log_num_shards": "128", 2024-04-26T03:38:45.767 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-04-26T03:38:45.768 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_data_log_window": "30", 2024-04-26T03:38:45.768 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_data_notify_interval_msec": "0", 2024-04-26T03:38:45.768 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_data_sync_poll_interval": "20", 2024-04-26T03:38:45.768 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_data_sync_spawn_window": "20", 2024-04-26T03:38:45.768 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-04-26T03:38:45.768 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-04-26T03:38:45.768 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-04-26T03:38:45.768 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_default_data_log_backing": "fifo", 2024-04-26T03:38:45.768 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-04-26T03:38:45.768 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_default_region_info_oid": "default.region", 2024-04-26T03:38:45.768 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-04-26T03:38:45.768 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-04-26T03:38:45.768 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_defer_to_bucket_acls": "", 2024-04-26T03:38:45.768 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-04-26T03:38:45.768 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-04-26T03:38:45.769 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-04-26T03:38:45.769 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-04-26T03:38:45.769 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-04-26T03:38:45.769 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-04-26T03:38:45.769 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-04-26T03:38:45.769 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-04-26T03:38:45.769 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_dmclock_data_res": "500.000000", 2024-04-26T03:38:45.769 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-04-26T03:38:45.769 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-04-26T03:38:45.769 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-04-26T03:38:45.769 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-04-26T03:38:45.769 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_dns_name": "", 2024-04-26T03:38:45.769 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_dns_s3website_name": "", 2024-04-26T03:38:45.769 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_dynamic_resharding": "true", 2024-04-26T03:38:45.769 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-04-26T03:38:45.770 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_enable_gc_threads": "true", 2024-04-26T03:38:45.770 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_enable_lc_threads": "true", 2024-04-26T03:38:45.770 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_enable_ops_log": "false", 2024-04-26T03:38:45.770 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_enable_quota_threads": "true", 2024-04-26T03:38:45.770 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_enable_static_website": "false", 2024-04-26T03:38:45.770 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_enable_usage_log": "false", 2024-04-26T03:38:45.770 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_enforce_swift_acls": "true", 2024-04-26T03:38:45.770 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_exit_timeout_secs": "120", 2024-04-26T03:38:45.770 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_expose_bucket": "false", 2024-04-26T03:38:45.770 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_extended_http_attrs": "", 2024-04-26T03:38:45.770 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_filter": "none", 2024-04-26T03:38:45.770 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-04-26T03:38:45.770 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_frontends": "beast port=7480", 2024-04-26T03:38:45.770 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_gc_max_concurrent_io": "10", 2024-04-26T03:38:45.771 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_gc_max_deferred": "50", 2024-04-26T03:38:45.771 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-04-26T03:38:45.771 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_gc_max_objs": "32", 2024-04-26T03:38:45.771 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_gc_max_queue_size": "134213632", 2024-04-26T03:38:45.771 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_gc_max_trim_chunk": "16", 2024-04-26T03:38:45.771 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_gc_obj_min_wait": "7200", 2024-04-26T03:38:45.771 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_gc_processor_max_time": "3600", 2024-04-26T03:38:45.771 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_gc_processor_period": "3600", 2024-04-26T03:38:45.771 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-04-26T03:38:45.771 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_get_obj_window_size": "16777216", 2024-04-26T03:38:45.771 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_healthcheck_disabling_path": "", 2024-04-26T03:38:45.771 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_ignore_get_invalid_range": "false", 2024-04-26T03:38:45.771 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_init_timeout": "300", 2024-04-26T03:38:45.771 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-04-26T03:38:45.771 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-04-26T03:38:45.772 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_kafka_connection_idle": "300", 2024-04-26T03:38:45.772 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_kafka_message_timeout": "5000", 2024-04-26T03:38:45.772 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_kafka_sleep_timeout": "10", 2024-04-26T03:38:45.772 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-04-26T03:38:45.772 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-04-26T03:38:45.772 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-04-26T03:38:45.772 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_keystone_admin_domain": "", 2024-04-26T03:38:45.772 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_keystone_admin_password": "", 2024-04-26T03:38:45.772 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_keystone_admin_password_path": "", 2024-04-26T03:38:45.772 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_keystone_admin_project": "", 2024-04-26T03:38:45.772 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_keystone_admin_tenant": "", 2024-04-26T03:38:45.772 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_keystone_admin_token": "", 2024-04-26T03:38:45.772 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_keystone_admin_token_path": "", 2024-04-26T03:38:45.772 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_keystone_admin_user": "", 2024-04-26T03:38:45.772 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_keystone_api_version": "2", 2024-04-26T03:38:45.773 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_keystone_barbican_domain": "", 2024-04-26T03:38:45.773 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_keystone_barbican_password": "", 2024-04-26T03:38:45.773 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_keystone_barbican_project": "", 2024-04-26T03:38:45.773 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_keystone_barbican_tenant": "", 2024-04-26T03:38:45.773 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_keystone_barbican_user": "", 2024-04-26T03:38:45.773 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-04-26T03:38:45.773 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_keystone_implicit_tenants": "false", 2024-04-26T03:38:45.773 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-04-26T03:38:45.773 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_keystone_service_token_enabled": "false", 2024-04-26T03:38:45.773 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_keystone_token_cache_size": "10000", 2024-04-26T03:38:45.773 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_keystone_url": "", 2024-04-26T03:38:45.773 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_keystone_verify_ssl": "true", 2024-04-26T03:38:45.773 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_lc_debug_interval": "-1", 2024-04-26T03:38:45.773 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_lc_lock_max_time": "90", 2024-04-26T03:38:45.773 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_lc_max_objs": "32", 2024-04-26T03:38:45.774 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_lc_max_rules": "1000", 2024-04-26T03:38:45.774 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_lc_max_worker": "3", 2024-04-26T03:38:45.774 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_lc_max_wp_worker": "3", 2024-04-26T03:38:45.774 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_lc_thread_delay": "0", 2024-04-26T03:38:45.774 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-04-26T03:38:45.774 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_ldap_dnattr": "uid", 2024-04-26T03:38:45.774 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-04-26T03:38:45.774 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_ldap_searchfilter": "", 2024-04-26T03:38:45.774 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-04-26T03:38:45.774 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_ldap_uri": "", 2024-04-26T03:38:45.774 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-04-26T03:38:45.774 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-04-26T03:38:45.774 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-04-26T03:38:45.774 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_log_http_headers": "", 2024-04-26T03:38:45.775 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_log_nonexistent_bucket": "false", 2024-04-26T03:38:45.775 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-04-26T03:38:45.775 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_log_object_name_utc": "false", 2024-04-26T03:38:45.775 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-04-26T03:38:45.775 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-04-26T03:38:45.775 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_max_attr_name_len": "0", 2024-04-26T03:38:45.775 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_max_attr_size": "0", 2024-04-26T03:38:45.775 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_max_attrs_num_in_req": "0", 2024-04-26T03:38:45.775 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_max_chunk_size": "4194304", 2024-04-26T03:38:45.775 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_max_concurrent_requests": "1024", 2024-04-26T03:38:45.775 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-04-26T03:38:45.775 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_max_dynamic_shards": "1999", 2024-04-26T03:38:45.775 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_max_listing_results": "1000", 2024-04-26T03:38:45.775 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_max_notify_retries": "10", 2024-04-26T03:38:45.775 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_max_objs_per_shard": "100000", 2024-04-26T03:38:45.776 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_max_put_param_size": "1048576", 2024-04-26T03:38:45.776 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_max_put_size": "5368709120", 2024-04-26T03:38:45.776 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_max_slo_entries": "1000", 2024-04-26T03:38:45.776 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_md_log_max_shards": "64", 2024-04-26T03:38:45.776 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_md_notify_interval_msec": "200", 2024-04-26T03:38:45.776 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_meta_sync_poll_interval": "20", 2024-04-26T03:38:45.776 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_meta_sync_spawn_window": "20", 2024-04-26T03:38:45.776 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-04-26T03:38:45.776 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_mp_lock_max_time": "600", 2024-04-26T03:38:45.776 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-04-26T03:38:45.776 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_multipart_min_part_size": "5242880", 2024-04-26T03:38:45.776 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-04-26T03:38:45.776 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-04-26T03:38:45.776 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_nfs_fhcache_size": "2017", 2024-04-26T03:38:45.777 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-04-26T03:38:45.777 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-04-26T03:38:45.777 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_nfs_lru_lanes": "5", 2024-04-26T03:38:45.777 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_nfs_max_gc": "300", 2024-04-26T03:38:45.777 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-04-26T03:38:45.777 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_nfs_run_gc_threads": "false", 2024-04-26T03:38:45.777 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_nfs_run_lc_threads": "false", 2024-04-26T03:38:45.777 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_nfs_run_quota_threads": "false", 2024-04-26T03:38:45.777 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_nfs_run_sync_thread": "false", 2024-04-26T03:38:45.777 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-04-26T03:38:45.777 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-04-26T03:38:45.777 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_num_async_rados_threads": "32", 2024-04-26T03:38:45.777 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_num_control_oids": "8", 2024-04-26T03:38:45.777 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_numa_node": "-1", 2024-04-26T03:38:45.777 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_obj_stripe_size": "4194304", 2024-04-26T03:38:45.778 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-04-26T03:38:45.778 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_objexp_chunk_size": "100", 2024-04-26T03:38:45.778 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_objexp_gc_interval": "600", 2024-04-26T03:38:45.778 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_objexp_hints_num_shards": "127", 2024-04-26T03:38:45.778 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-04-26T03:38:45.778 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-04-26T03:38:45.778 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_op_thread_timeout": "600", 2024-04-26T03:38:45.778 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_op_tracing": "false", 2024-04-26T03:38:45.778 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_opa_token": "", 2024-04-26T03:38:45.778 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_opa_url": "", 2024-04-26T03:38:45.778 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_opa_verify_ssl": "true", 2024-04-26T03:38:45.778 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-04-26T03:38:45.778 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-04-26T03:38:45.779 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_ops_log_rados": "false", 2024-04-26T03:38:45.779 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_ops_log_socket_path": "", 2024-04-26T03:38:45.779 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-04-26T03:38:45.779 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-04-26T03:38:45.779 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-04-26T03:38:45.779 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_period_push_interval": "2.000000", 2024-04-26T03:38:45.779 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_period_push_interval_max": "30.000000", 2024-04-26T03:38:45.779 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_period_root_pool": ".rgw.root", 2024-04-26T03:38:45.779 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-04-26T03:38:45.779 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-04-26T03:38:45.779 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_posix_cache_lanes": "3", 2024-04-26T03:38:45.779 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-04-26T03:38:45.779 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_posix_cache_max_buckets": "100", 2024-04-26T03:38:45.779 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_posix_cache_partitions": "3", 2024-04-26T03:38:45.779 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-04-26T03:38:45.780 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_print_continue": "true", 2024-04-26T03:38:45.780 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_print_prohibited_content_length": "false", 2024-04-26T03:38:45.780 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-04-26T03:38:45.780 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-04-26T03:38:45.780 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-04-26T03:38:45.780 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-04-26T03:38:45.780 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_rados_tracing": "false", 2024-04-26T03:38:45.780 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_realm": "", 2024-04-26T03:38:45.780 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_realm_id": "", 2024-04-26T03:38:45.780 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-04-26T03:38:45.780 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_region": "", 2024-04-26T03:38:45.780 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_region_root_pool": ".rgw.root", 2024-04-26T03:38:45.780 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_relaxed_region_enforcement": "false", 2024-04-26T03:38:45.780 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-04-26T03:38:45.780 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-04-26T03:38:45.781 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_request_uri": "", 2024-04-26T03:38:45.781 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_reshard_batch_size": "64", 2024-04-26T03:38:45.781 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-04-26T03:38:45.781 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_reshard_max_aio": "128", 2024-04-26T03:38:45.781 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_reshard_num_logs": "16", 2024-04-26T03:38:45.781 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_reshard_thread_interval": "600", 2024-04-26T03:38:45.781 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_resolve_cname": "false", 2024-04-26T03:38:45.781 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_rest_getusage_op_compat": "false", 2024-04-26T03:38:45.781 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_run_sync_thread": "true", 2024-04-26T03:38:45.781 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-04-26T03:38:45.781 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-04-26T03:38:45.781 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_s3_auth_use_keystone": "false", 2024-04-26T03:38:45.781 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_s3_auth_use_ldap": "false", 2024-04-26T03:38:45.781 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_s3_auth_use_rados": "true", 2024-04-26T03:38:45.782 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_s3_auth_use_sts": "false", 2024-04-26T03:38:45.782 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-04-26T03:38:45.782 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_s3_success_create_obj_status": "0", 2024-04-26T03:38:45.782 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-04-26T03:38:45.782 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_scheduler_type": "throttler", 2024-04-26T03:38:45.782 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_script_uri": "", 2024-04-26T03:38:45.782 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_service_provider_name": "", 2024-04-26T03:38:45.782 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-04-26T03:38:45.782 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_sts_client_id": "", 2024-04-26T03:38:45.782 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_sts_client_secret": "", 2024-04-26T03:38:45.782 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_sts_entry": "sts", 2024-04-26T03:38:45.782 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_sts_key": "sts", 2024-04-26T03:38:45.782 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_sts_max_session_duration": "43200", 2024-04-26T03:38:45.782 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_sts_min_session_duration": "900", 2024-04-26T03:38:45.782 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_sts_token_introspection_url": "", 2024-04-26T03:38:45.783 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_swift_account_in_url": "false", 2024-04-26T03:38:45.783 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_swift_auth_entry": "auth", 2024-04-26T03:38:45.783 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_swift_auth_url": "", 2024-04-26T03:38:45.783 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_swift_custom_header": "", 2024-04-26T03:38:45.783 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_swift_enforce_content_length": "false", 2024-04-26T03:38:45.783 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_swift_need_stats": "true", 2024-04-26T03:38:45.783 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_swift_tenant_name": "", 2024-04-26T03:38:45.783 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_swift_token_expiration": "86400", 2024-04-26T03:38:45.783 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_swift_url": "", 2024-04-26T03:38:45.783 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_swift_url_prefix": "swift", 2024-04-26T03:38:45.783 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_swift_versioning_enabled": "false", 2024-04-26T03:38:45.783 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-04-26T03:38:45.783 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-04-26T03:38:45.783 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_sync_lease_period": "120", 2024-04-26T03:38:45.784 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-04-26T03:38:45.784 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_sync_log_trim_interval": "1200", 2024-04-26T03:38:45.784 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-04-26T03:38:45.784 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-04-26T03:38:45.784 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-04-26T03:38:45.784 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_sync_obj_etag_verify": "false", 2024-04-26T03:38:45.784 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_sync_trace_history_size": "4096", 2024-04-26T03:38:45.784 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-04-26T03:38:45.784 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-04-26T03:38:45.784 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_thread_pool_size": "512", 2024-04-26T03:38:45.784 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_topic_persistency_max_retries": "0", 2024-04-26T03:38:45.784 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-04-26T03:38:45.784 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-04-26T03:38:45.784 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_topic_require_publish_policy": "false", 2024-04-26T03:38:45.784 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_torrent_comment": "", 2024-04-26T03:38:45.785 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_torrent_createby": "", 2024-04-26T03:38:45.785 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_torrent_encoding": "", 2024-04-26T03:38:45.785 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_torrent_flag": "false", 2024-04-26T03:38:45.785 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_torrent_max_size": "5368709120", 2024-04-26T03:38:45.785 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_torrent_origin": "", 2024-04-26T03:38:45.785 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_torrent_sha_unit": "524288", 2024-04-26T03:38:45.785 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_torrent_tracker": "", 2024-04-26T03:38:45.785 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_trust_forwarded_https": "false", 2024-04-26T03:38:45.785 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-04-26T03:38:45.785 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_usage_log_tick_interval": "30", 2024-04-26T03:38:45.785 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_usage_max_shards": "32", 2024-04-26T03:38:45.785 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_usage_max_user_shards": "1", 2024-04-26T03:38:45.785 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_use_opa_authz": "false", 2024-04-26T03:38:45.785 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_user_counters_cache": "false", 2024-04-26T03:38:45.785 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_user_counters_cache_size": "10000", 2024-04-26T03:38:45.786 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-04-26T03:38:45.786 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_user_default_quota_max_size": "-1", 2024-04-26T03:38:45.786 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_user_max_buckets": "1000", 2024-04-26T03:38:45.786 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_user_policies_max_num": "100", 2024-04-26T03:38:45.786 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-04-26T03:38:45.786 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-04-26T03:38:45.786 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_user_quota_sync_interval": "86400", 2024-04-26T03:38:45.786 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-04-26T03:38:45.786 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_user_unique_email": "true", 2024-04-26T03:38:45.786 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_verify_ssl": "true", 2024-04-26T03:38:45.786 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_website_routing_rules_max_num": "50", 2024-04-26T03:38:45.786 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_zone": "", 2024-04-26T03:38:45.786 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_zone_id": "", 2024-04-26T03:38:45.786 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-04-26T03:38:45.787 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_zonegroup": "", 2024-04-26T03:38:45.787 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_zonegroup_id": "", 2024-04-26T03:38:45.787 INFO:teuthology.orchestra.run.smithi028.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-04-26T03:38:45.787 INFO:teuthology.orchestra.run.smithi028.stdout: "rgwlc_auto_session_clear": "true", 2024-04-26T03:38:45.787 INFO:teuthology.orchestra.run.smithi028.stdout: "rgwlc_skip_bucket_step": "false", 2024-04-26T03:38:45.787 INFO:teuthology.orchestra.run.smithi028.stdout: "rocksdb_block_size": "4096", 2024-04-26T03:38:45.787 INFO:teuthology.orchestra.run.smithi028.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-04-26T03:38:45.787 INFO:teuthology.orchestra.run.smithi028.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-04-26T03:38:45.787 INFO:teuthology.orchestra.run.smithi028.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-04-26T03:38:45.787 INFO:teuthology.orchestra.run.smithi028.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-04-26T03:38:45.787 INFO:teuthology.orchestra.run.smithi028.stdout: "rocksdb_cache_shard_bits": "4", 2024-04-26T03:38:45.787 INFO:teuthology.orchestra.run.smithi028.stdout: "rocksdb_cache_size": "536870912", 2024-04-26T03:38:45.787 INFO:teuthology.orchestra.run.smithi028.stdout: "rocksdb_cache_type": "binned_lru", 2024-04-26T03:38:45.787 INFO:teuthology.orchestra.run.smithi028.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-04-26T03:38:45.787 INFO:teuthology.orchestra.run.smithi028.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-04-26T03:38:45.788 INFO:teuthology.orchestra.run.smithi028.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-04-26T03:38:45.788 INFO:teuthology.orchestra.run.smithi028.stdout: "rocksdb_collect_compaction_stats": "false", 2024-04-26T03:38:45.788 INFO:teuthology.orchestra.run.smithi028.stdout: "rocksdb_collect_extended_stats": "false", 2024-04-26T03:38:45.788 INFO:teuthology.orchestra.run.smithi028.stdout: "rocksdb_collect_memory_stats": "false", 2024-04-26T03:38:45.788 INFO:teuthology.orchestra.run.smithi028.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-04-26T03:38:45.788 INFO:teuthology.orchestra.run.smithi028.stdout: "rocksdb_index_type": "binary_search", 2024-04-26T03:38:45.788 INFO:teuthology.orchestra.run.smithi028.stdout: "rocksdb_log_to_ceph_log": "true", 2024-04-26T03:38:45.788 INFO:teuthology.orchestra.run.smithi028.stdout: "rocksdb_metadata_block_size": "4096", 2024-04-26T03:38:45.788 INFO:teuthology.orchestra.run.smithi028.stdout: "rocksdb_partition_filters": "false", 2024-04-26T03:38:45.788 INFO:teuthology.orchestra.run.smithi028.stdout: "rocksdb_perf": "false", 2024-04-26T03:38:45.788 INFO:teuthology.orchestra.run.smithi028.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-04-26T03:38:45.788 INFO:teuthology.orchestra.run.smithi028.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-04-26T03:38:45.788 INFO:teuthology.orchestra.run.smithi028.stdout: "rotating_keys_renewal_timeout": "10", 2024-04-26T03:38:45.788 INFO:teuthology.orchestra.run.smithi028.stdout: "run_dir": "/var/run/ceph", 2024-04-26T03:38:45.789 INFO:teuthology.orchestra.run.smithi028.stdout: "seastore_block_create": "true", 2024-04-26T03:38:45.789 INFO:teuthology.orchestra.run.smithi028.stdout: "seastore_cache_lru_size": "67108864", 2024-04-26T03:38:45.789 INFO:teuthology.orchestra.run.smithi028.stdout: "seastore_cbjournal_size": "5368709120", 2024-04-26T03:38:45.789 INFO:teuthology.orchestra.run.smithi028.stdout: "seastore_data_delta_based_overwrite": "0", 2024-04-26T03:38:45.789 INFO:teuthology.orchestra.run.smithi028.stdout: "seastore_default_max_object_size": "16777216", 2024-04-26T03:38:45.789 INFO:teuthology.orchestra.run.smithi028.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-04-26T03:38:45.789 INFO:teuthology.orchestra.run.smithi028.stdout: "seastore_device_size": "53687091200", 2024-04-26T03:38:45.789 INFO:teuthology.orchestra.run.smithi028.stdout: "seastore_journal_batch_capacity": "16", 2024-04-26T03:38:45.789 INFO:teuthology.orchestra.run.smithi028.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-04-26T03:38:45.789 INFO:teuthology.orchestra.run.smithi028.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-04-26T03:38:45.789 INFO:teuthology.orchestra.run.smithi028.stdout: "seastore_journal_iodepth_limit": "5", 2024-04-26T03:38:45.789 INFO:teuthology.orchestra.run.smithi028.stdout: "seastore_main_device_type": "SSD", 2024-04-26T03:38:45.789 INFO:teuthology.orchestra.run.smithi028.stdout: "seastore_max_concurrent_transactions": "8", 2024-04-26T03:38:45.789 INFO:teuthology.orchestra.run.smithi028.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-04-26T03:38:45.789 INFO:teuthology.orchestra.run.smithi028.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-04-26T03:38:45.790 INFO:teuthology.orchestra.run.smithi028.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-04-26T03:38:45.790 INFO:teuthology.orchestra.run.smithi028.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-04-26T03:38:45.790 INFO:teuthology.orchestra.run.smithi028.stdout: "seastore_segment_size": "67108864", 2024-04-26T03:38:45.790 INFO:teuthology.orchestra.run.smithi028.stdout: "set_keepcaps": "false", 2024-04-26T03:38:45.790 INFO:teuthology.orchestra.run.smithi028.stdout: "setgroup": "", 2024-04-26T03:38:45.790 INFO:teuthology.orchestra.run.smithi028.stdout: "setuser": "", 2024-04-26T03:38:45.790 INFO:teuthology.orchestra.run.smithi028.stdout: "setuser_match_path": "", 2024-04-26T03:38:45.790 INFO:teuthology.orchestra.run.smithi028.stdout: "target_max_misplaced_ratio": "0.050000", 2024-04-26T03:38:45.790 INFO:teuthology.orchestra.run.smithi028.stdout: "thp": "false", 2024-04-26T03:38:45.790 INFO:teuthology.orchestra.run.smithi028.stdout: "threadpool_default_timeout": "60", 2024-04-26T03:38:45.790 INFO:teuthology.orchestra.run.smithi028.stdout: "threadpool_empty_queue_max_wait": "2", 2024-04-26T03:38:45.790 INFO:teuthology.orchestra.run.smithi028.stdout: "throttler_perf_counter": "true" 2024-04-26T03:38:45.790 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:38:45.793 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-04-26T03:38:45.909 INFO:teuthology.orchestra.run.smithi028.stdout:mon.a: {} 2024-04-26T03:38:45.909 INFO:teuthology.orchestra.run.smithi028.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-26T03:38:45.919 INFO:teuthology.orchestra.run.smithi028.stdout:mon.b: {} 2024-04-26T03:38:45.919 INFO:teuthology.orchestra.run.smithi028.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-26T03:38:45.929 INFO:teuthology.orchestra.run.smithi028.stdout:mon.c: {} 2024-04-26T03:38:45.929 INFO:teuthology.orchestra.run.smithi028.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-26T03:38:45.940 DEBUG:teuthology.orchestra.run.smithi028:> type ceph-objectstore-tool 2024-04-26T03:38:45.954 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-04-26T03:38:45.955 DEBUG:teuthology.orchestra.run.smithi042:> type ceph-objectstore-tool 2024-04-26T03:38:45.973 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-04-26T03:38:45.973 DEBUG:teuthology.orchestra.run.smithi092:> type ceph-objectstore-tool 2024-04-26T03:38:45.989 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-04-26T03:38:45.989 DEBUG:teuthology.orchestra.run.smithi066:> type ceph-objectstore-tool 2024-04-26T03:38:46.008 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-04-26T03:38:46.008 INFO:teuthology.run_tasks:Running task rados... 2024-04-26T03:38:46.020 INFO:tasks.rados:Beginning rados... 2024-04-26T03:38:46.021 DEBUG:teuthology.run_tasks:Unwinding manager rados 2024-04-26T03:38:46.033 INFO:tasks.rados:joining rados 2024-04-26T03:38:46.033 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-04-26T03:38:46.033 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-04-26T03:38:46.033 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-26T03:38:46.033 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-04-26T03:38:46.034 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-04-26T03:38:46.034 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-04-26T03:38:46.035 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-04-26T03:38:46.035 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-26T03:38:46.035 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-04-26T03:38:46.036 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-26T03:38:46.036 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-04-26T03:38:46.036 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-26T03:38:46.036 INFO:tasks.rados:clients are ['client.0'] 2024-04-26T03:38:46.037 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set shecprofile c=2 crush-failure-domain=osd k=4 m=3 name=shecprofile plugin=shec 2024-04-26T03:38:46.049 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-04-26T03:38:46.047+0000 7fee562fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31759) UID: 0 2024-04-26T03:38:46.137 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-04-26T03:38:46.135+0000 7fe9223c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31954) UID: 0 2024-04-26T03:38:46.179 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0: {} 2024-04-26T03:38:46.179 INFO:teuthology.orchestra.run.smithi028.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T03:38:46.198 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:46.198 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:46.210 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-26T03:38:46.216 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1: {} 2024-04-26T03:38:46.216 INFO:teuthology.orchestra.run.smithi028.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T03:38:46.238 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:38:46.236+0000 7fb78c544640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31956) UID: 0 2024-04-26T03:38:46.252 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2: {} 2024-04-26T03:38:46.253 INFO:teuthology.orchestra.run.smithi028.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T03:38:46.288 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3: {} 2024-04-26T03:38:46.288 INFO:teuthology.orchestra.run.smithi028.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T03:38:46.322 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:46.322 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:46.325 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4: {} 2024-04-26T03:38:46.325 INFO:teuthology.orchestra.run.smithi028.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T03:38:46.332 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-26T03:38:46.338 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:38:46.337+0000 7fb78c544640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31956) UID: 0 2024-04-26T03:38:46.358 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:38:46.358 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:38:46.358 INFO:teuthology.orchestra.run.smithi028.stdout: "epoch": 29, 2024-04-26T03:38:46.358 INFO:teuthology.orchestra.run.smithi028.stdout: "fsid": "c5a13c99-5c9a-4b8b-a893-4f6f4211677a", 2024-04-26T03:38:46.358 INFO:teuthology.orchestra.run.smithi028.stdout: "created": "2024-04-26T03:38:19.896648+0000", 2024-04-26T03:38:46.359 INFO:teuthology.orchestra.run.smithi028.stdout: "modified": "2024-04-26T03:38:38.431503+0000", 2024-04-26T03:38:46.359 INFO:teuthology.orchestra.run.smithi028.stdout: "last_up_change": "2024-04-26T03:38:33.308561+0000", 2024-04-26T03:38:46.359 INFO:teuthology.orchestra.run.smithi028.stdout: "last_in_change": "2024-04-26T03:38:24.150336+0000", 2024-04-26T03:38:46.359 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-26T03:38:46.359 INFO:teuthology.orchestra.run.smithi028.stdout: "flags_num": 5799936, 2024-04-26T03:38:46.359 INFO:teuthology.orchestra.run.smithi028.stdout: "flags_set": [ 2024-04-26T03:38:46.359 INFO:teuthology.orchestra.run.smithi028.stdout: "pglog_hardlimit", 2024-04-26T03:38:46.359 INFO:teuthology.orchestra.run.smithi028.stdout: "purged_snapdirs", 2024-04-26T03:38:46.359 INFO:teuthology.orchestra.run.smithi028.stdout: "recovery_deletes", 2024-04-26T03:38:46.359 INFO:teuthology.orchestra.run.smithi028.stdout: "sortbitwise" 2024-04-26T03:38:46.359 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-04-26T03:38:46.359 INFO:teuthology.orchestra.run.smithi028.stdout: "crush_version": 5, 2024-04-26T03:38:46.359 INFO:teuthology.orchestra.run.smithi028.stdout: "full_ratio": 0.94999998807907104, 2024-04-26T03:38:46.359 INFO:teuthology.orchestra.run.smithi028.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-04-26T03:38:46.360 INFO:teuthology.orchestra.run.smithi028.stdout: "nearfull_ratio": 0.85000002384185791, 2024-04-26T03:38:46.360 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_snapshot": "", 2024-04-26T03:38:46.360 INFO:teuthology.orchestra.run.smithi028.stdout: "pool_max": 2, 2024-04-26T03:38:46.360 INFO:teuthology.orchestra.run.smithi028.stdout: "max_osd": 16, 2024-04-26T03:38:46.360 INFO:teuthology.orchestra.run.smithi028.stdout: "require_min_compat_client": "luminous", 2024-04-26T03:38:46.360 INFO:teuthology.orchestra.run.smithi028.stdout: "min_compat_client": "jewel", 2024-04-26T03:38:46.360 INFO:teuthology.orchestra.run.smithi028.stdout: "require_osd_release": "squid", 2024-04-26T03:38:46.360 INFO:teuthology.orchestra.run.smithi028.stdout: "allow_crimson": false, 2024-04-26T03:38:46.360 INFO:teuthology.orchestra.run.smithi028.stdout: "pools": [ 2024-04-26T03:38:46.360 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.360 INFO:teuthology.orchestra.run.smithi028.stdout: "pool": 1, 2024-04-26T03:38:46.360 INFO:teuthology.orchestra.run.smithi028.stdout: "pool_name": ".mgr", 2024-04-26T03:38:46.360 INFO:teuthology.orchestra.run.smithi028.stdout: "create_time": "2024-04-26T03:38:32.306696+0000", 2024-04-26T03:38:46.361 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-04-26T03:38:46.361 INFO:teuthology.orchestra.run.smithi028.stdout: "flags_names": "hashpspool", 2024-04-26T03:38:46.361 INFO:teuthology.orchestra.run.smithi028.stdout: "type": 1, 2024-04-26T03:38:46.361 INFO:teuthology.orchestra.run.smithi028.stdout: "size": 2, 2024-04-26T03:38:46.361 INFO:teuthology.orchestra.run.smithi028.stdout: "min_size": 1, 2024-04-26T03:38:46.361 INFO:teuthology.orchestra.run.smithi028.stdout: "crush_rule": 0, 2024-04-26T03:38:46.361 INFO:teuthology.orchestra.run.smithi028.stdout: "peering_crush_bucket_count": 0, 2024-04-26T03:38:46.361 INFO:teuthology.orchestra.run.smithi028.stdout: "peering_crush_bucket_target": 0, 2024-04-26T03:38:46.361 INFO:teuthology.orchestra.run.smithi028.stdout: "peering_crush_bucket_barrier": 0, 2024-04-26T03:38:46.361 INFO:teuthology.orchestra.run.smithi028.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-26T03:38:46.361 INFO:teuthology.orchestra.run.smithi028.stdout: "object_hash": 2, 2024-04-26T03:38:46.361 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_autoscale_mode": "off", 2024-04-26T03:38:46.361 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_num": 1, 2024-04-26T03:38:46.361 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_placement_num": 1, 2024-04-26T03:38:46.362 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_placement_num_target": 1, 2024-04-26T03:38:46.362 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_num_target": 1, 2024-04-26T03:38:46.362 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_num_pending": 1, 2024-04-26T03:38:46.362 INFO:teuthology.orchestra.run.smithi028.stdout: "last_pg_merge_meta": { 2024-04-26T03:38:46.362 INFO:teuthology.orchestra.run.smithi028.stdout: "source_pgid": "0.0", 2024-04-26T03:38:46.362 INFO:teuthology.orchestra.run.smithi028.stdout: "ready_epoch": 0, 2024-04-26T03:38:46.362 INFO:teuthology.orchestra.run.smithi028.stdout: "last_epoch_started": 0, 2024-04-26T03:38:46.362 INFO:teuthology.orchestra.run.smithi028.stdout: "last_epoch_clean": 0, 2024-04-26T03:38:46.362 INFO:teuthology.orchestra.run.smithi028.stdout: "source_version": "0'0", 2024-04-26T03:38:46.362 INFO:teuthology.orchestra.run.smithi028.stdout: "target_version": "0'0" 2024-04-26T03:38:46.362 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.362 INFO:teuthology.orchestra.run.smithi028.stdout: "last_change": "26", 2024-04-26T03:38:46.362 INFO:teuthology.orchestra.run.smithi028.stdout: "last_force_op_resend": "0", 2024-04-26T03:38:46.362 INFO:teuthology.orchestra.run.smithi028.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-26T03:38:46.363 INFO:teuthology.orchestra.run.smithi028.stdout: "last_force_op_resend_preluminous": "0", 2024-04-26T03:38:46.363 INFO:teuthology.orchestra.run.smithi028.stdout: "auid": 0, 2024-04-26T03:38:46.363 INFO:teuthology.orchestra.run.smithi028.stdout: "snap_mode": "selfmanaged", 2024-04-26T03:38:46.363 INFO:teuthology.orchestra.run.smithi028.stdout: "snap_seq": 0, 2024-04-26T03:38:46.363 INFO:teuthology.orchestra.run.smithi028.stdout: "snap_epoch": 0, 2024-04-26T03:38:46.363 INFO:teuthology.orchestra.run.smithi028.stdout: "pool_snaps": [], 2024-04-26T03:38:46.363 INFO:teuthology.orchestra.run.smithi028.stdout: "removed_snaps": "[]", 2024-04-26T03:38:46.363 INFO:teuthology.orchestra.run.smithi028.stdout: "quota_max_bytes": 0, 2024-04-26T03:38:46.363 INFO:teuthology.orchestra.run.smithi028.stdout: "quota_max_objects": 0, 2024-04-26T03:38:46.363 INFO:teuthology.orchestra.run.smithi028.stdout: "tiers": [], 2024-04-26T03:38:46.363 INFO:teuthology.orchestra.run.smithi028.stdout: "tier_of": -1, 2024-04-26T03:38:46.363 INFO:teuthology.orchestra.run.smithi028.stdout: "read_tier": -1, 2024-04-26T03:38:46.363 INFO:teuthology.orchestra.run.smithi028.stdout: "write_tier": -1, 2024-04-26T03:38:46.363 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_mode": "none", 2024-04-26T03:38:46.364 INFO:teuthology.orchestra.run.smithi028.stdout: "target_max_bytes": 0, 2024-04-26T03:38:46.364 INFO:teuthology.orchestra.run.smithi028.stdout: "target_max_objects": 0, 2024-04-26T03:38:46.364 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-26T03:38:46.364 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-26T03:38:46.364 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-26T03:38:46.364 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_min_flush_age": 0, 2024-04-26T03:38:46.364 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_min_evict_age": 0, 2024-04-26T03:38:46.364 INFO:teuthology.orchestra.run.smithi028.stdout: "erasure_code_profile": "", 2024-04-26T03:38:46.364 INFO:teuthology.orchestra.run.smithi028.stdout: "hit_set_params": { 2024-04-26T03:38:46.364 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "none" 2024-04-26T03:38:46.364 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.364 INFO:teuthology.orchestra.run.smithi028.stdout: "hit_set_period": 0, 2024-04-26T03:38:46.364 INFO:teuthology.orchestra.run.smithi028.stdout: "hit_set_count": 0, 2024-04-26T03:38:46.364 INFO:teuthology.orchestra.run.smithi028.stdout: "use_gmt_hitset": true, 2024-04-26T03:38:46.365 INFO:teuthology.orchestra.run.smithi028.stdout: "min_read_recency_for_promote": 0, 2024-04-26T03:38:46.365 INFO:teuthology.orchestra.run.smithi028.stdout: "min_write_recency_for_promote": 0, 2024-04-26T03:38:46.365 INFO:teuthology.orchestra.run.smithi028.stdout: "hit_set_grade_decay_rate": 0, 2024-04-26T03:38:46.365 INFO:teuthology.orchestra.run.smithi028.stdout: "hit_set_search_last_n": 0, 2024-04-26T03:38:46.365 INFO:teuthology.orchestra.run.smithi028.stdout: "grade_table": [], 2024-04-26T03:38:46.365 INFO:teuthology.orchestra.run.smithi028.stdout: "stripe_width": 0, 2024-04-26T03:38:46.365 INFO:teuthology.orchestra.run.smithi028.stdout: "expected_num_objects": 0, 2024-04-26T03:38:46.365 INFO:teuthology.orchestra.run.smithi028.stdout: "fast_read": false, 2024-04-26T03:38:46.365 INFO:teuthology.orchestra.run.smithi028.stdout: "options": { 2024-04-26T03:38:46.365 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_num_max": 32, 2024-04-26T03:38:46.365 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_num_min": 1 2024-04-26T03:38:46.365 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.365 INFO:teuthology.orchestra.run.smithi028.stdout: "application_metadata": { 2024-04-26T03:38:46.366 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": {} 2024-04-26T03:38:46.366 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.366 INFO:teuthology.orchestra.run.smithi028.stdout: "read_balance": { 2024-04-26T03:38:46.366 INFO:teuthology.orchestra.run.smithi028.stdout: "score_type": "Fair distribution", 2024-04-26T03:38:46.366 INFO:teuthology.orchestra.run.smithi028.stdout: "score_acting": 15.380000114440918, 2024-04-26T03:38:46.366 INFO:teuthology.orchestra.run.smithi028.stdout: "score_stable": 15.380000114440918, 2024-04-26T03:38:46.366 INFO:teuthology.orchestra.run.smithi028.stdout: "optimal_score": 0.12999999523162842, 2024-04-26T03:38:46.366 INFO:teuthology.orchestra.run.smithi028.stdout: "raw_score_acting": 2, 2024-04-26T03:38:46.366 INFO:teuthology.orchestra.run.smithi028.stdout: "raw_score_stable": 2, 2024-04-26T03:38:46.366 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity_weighted": 1, 2024-04-26T03:38:46.367 INFO:teuthology.orchestra.run.smithi028.stdout: "average_primary_affinity": 1, 2024-04-26T03:38:46.367 INFO:teuthology.orchestra.run.smithi028.stdout: "average_primary_affinity_weighted": 1 2024-04-26T03:38:46.367 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.367 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.367 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.367 INFO:teuthology.orchestra.run.smithi028.stdout: "pool": 2, 2024-04-26T03:38:46.367 INFO:teuthology.orchestra.run.smithi028.stdout: "pool_name": "rbd", 2024-04-26T03:38:46.367 INFO:teuthology.orchestra.run.smithi028.stdout: "create_time": "2024-04-26T03:38:34.592633+0000", 2024-04-26T03:38:46.367 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-04-26T03:38:46.367 INFO:teuthology.orchestra.run.smithi028.stdout: "flags_names": "hashpspool", 2024-04-26T03:38:46.367 INFO:teuthology.orchestra.run.smithi028.stdout: "type": 1, 2024-04-26T03:38:46.367 INFO:teuthology.orchestra.run.smithi028.stdout: "size": 2, 2024-04-26T03:38:46.367 INFO:teuthology.orchestra.run.smithi028.stdout: "min_size": 1, 2024-04-26T03:38:46.368 INFO:teuthology.orchestra.run.smithi028.stdout: "crush_rule": 0, 2024-04-26T03:38:46.368 INFO:teuthology.orchestra.run.smithi028.stdout: "peering_crush_bucket_count": 0, 2024-04-26T03:38:46.368 INFO:teuthology.orchestra.run.smithi028.stdout: "peering_crush_bucket_target": 0, 2024-04-26T03:38:46.368 INFO:teuthology.orchestra.run.smithi028.stdout: "peering_crush_bucket_barrier": 0, 2024-04-26T03:38:46.368 INFO:teuthology.orchestra.run.smithi028.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-26T03:38:46.368 INFO:teuthology.orchestra.run.smithi028.stdout: "object_hash": 2, 2024-04-26T03:38:46.368 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_autoscale_mode": "off", 2024-04-26T03:38:46.368 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_num": 8, 2024-04-26T03:38:46.368 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_placement_num": 8, 2024-04-26T03:38:46.368 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_placement_num_target": 8, 2024-04-26T03:38:46.369 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_num_target": 8, 2024-04-26T03:38:46.369 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_num_pending": 8, 2024-04-26T03:38:46.369 INFO:teuthology.orchestra.run.smithi028.stdout: "last_pg_merge_meta": { 2024-04-26T03:38:46.369 INFO:teuthology.orchestra.run.smithi028.stdout: "source_pgid": "0.0", 2024-04-26T03:38:46.369 INFO:teuthology.orchestra.run.smithi028.stdout: "ready_epoch": 0, 2024-04-26T03:38:46.369 INFO:teuthology.orchestra.run.smithi028.stdout: "last_epoch_started": 0, 2024-04-26T03:38:46.369 INFO:teuthology.orchestra.run.smithi028.stdout: "last_epoch_clean": 0, 2024-04-26T03:38:46.369 INFO:teuthology.orchestra.run.smithi028.stdout: "source_version": "0'0", 2024-04-26T03:38:46.369 INFO:teuthology.orchestra.run.smithi028.stdout: "target_version": "0'0" 2024-04-26T03:38:46.369 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.369 INFO:teuthology.orchestra.run.smithi028.stdout: "last_change": "29", 2024-04-26T03:38:46.369 INFO:teuthology.orchestra.run.smithi028.stdout: "last_force_op_resend": "0", 2024-04-26T03:38:46.369 INFO:teuthology.orchestra.run.smithi028.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-26T03:38:46.369 INFO:teuthology.orchestra.run.smithi028.stdout: "last_force_op_resend_preluminous": "0", 2024-04-26T03:38:46.369 INFO:teuthology.orchestra.run.smithi028.stdout: "auid": 0, 2024-04-26T03:38:46.370 INFO:teuthology.orchestra.run.smithi028.stdout: "snap_mode": "selfmanaged", 2024-04-26T03:38:46.370 INFO:teuthology.orchestra.run.smithi028.stdout: "snap_seq": 0, 2024-04-26T03:38:46.370 INFO:teuthology.orchestra.run.smithi028.stdout: "snap_epoch": 0, 2024-04-26T03:38:46.370 INFO:teuthology.orchestra.run.smithi028.stdout: "pool_snaps": [], 2024-04-26T03:38:46.370 INFO:teuthology.orchestra.run.smithi028.stdout: "removed_snaps": "[]", 2024-04-26T03:38:46.370 INFO:teuthology.orchestra.run.smithi028.stdout: "quota_max_bytes": 0, 2024-04-26T03:38:46.370 INFO:teuthology.orchestra.run.smithi028.stdout: "quota_max_objects": 0, 2024-04-26T03:38:46.370 INFO:teuthology.orchestra.run.smithi028.stdout: "tiers": [], 2024-04-26T03:38:46.370 INFO:teuthology.orchestra.run.smithi028.stdout: "tier_of": -1, 2024-04-26T03:38:46.370 INFO:teuthology.orchestra.run.smithi028.stdout: "read_tier": -1, 2024-04-26T03:38:46.370 INFO:teuthology.orchestra.run.smithi028.stdout: "write_tier": -1, 2024-04-26T03:38:46.370 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_mode": "none", 2024-04-26T03:38:46.370 INFO:teuthology.orchestra.run.smithi028.stdout: "target_max_bytes": 0, 2024-04-26T03:38:46.371 INFO:teuthology.orchestra.run.smithi028.stdout: "target_max_objects": 0, 2024-04-26T03:38:46.371 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-26T03:38:46.371 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-26T03:38:46.371 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-26T03:38:46.371 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_min_flush_age": 0, 2024-04-26T03:38:46.371 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_min_evict_age": 0, 2024-04-26T03:38:46.371 INFO:teuthology.orchestra.run.smithi028.stdout: "erasure_code_profile": "", 2024-04-26T03:38:46.371 INFO:teuthology.orchestra.run.smithi028.stdout: "hit_set_params": { 2024-04-26T03:38:46.371 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "none" 2024-04-26T03:38:46.371 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.372 INFO:teuthology.orchestra.run.smithi028.stdout: "hit_set_period": 0, 2024-04-26T03:38:46.372 INFO:teuthology.orchestra.run.smithi028.stdout: "hit_set_count": 0, 2024-04-26T03:38:46.372 INFO:teuthology.orchestra.run.smithi028.stdout: "use_gmt_hitset": true, 2024-04-26T03:38:46.372 INFO:teuthology.orchestra.run.smithi028.stdout: "min_read_recency_for_promote": 0, 2024-04-26T03:38:46.372 INFO:teuthology.orchestra.run.smithi028.stdout: "min_write_recency_for_promote": 0, 2024-04-26T03:38:46.372 INFO:teuthology.orchestra.run.smithi028.stdout: "hit_set_grade_decay_rate": 0, 2024-04-26T03:38:46.372 INFO:teuthology.orchestra.run.smithi028.stdout: "hit_set_search_last_n": 0, 2024-04-26T03:38:46.372 INFO:teuthology.orchestra.run.smithi028.stdout: "grade_table": [], 2024-04-26T03:38:46.372 INFO:teuthology.orchestra.run.smithi028.stdout: "stripe_width": 0, 2024-04-26T03:38:46.372 INFO:teuthology.orchestra.run.smithi028.stdout: "expected_num_objects": 0, 2024-04-26T03:38:46.372 INFO:teuthology.orchestra.run.smithi028.stdout: "fast_read": false, 2024-04-26T03:38:46.372 INFO:teuthology.orchestra.run.smithi028.stdout: "options": {}, 2024-04-26T03:38:46.372 INFO:teuthology.orchestra.run.smithi028.stdout: "application_metadata": { 2024-04-26T03:38:46.372 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd": {} 2024-04-26T03:38:46.373 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.373 INFO:teuthology.orchestra.run.smithi028.stdout: "read_balance": { 2024-04-26T03:38:46.373 INFO:teuthology.orchestra.run.smithi028.stdout: "score_type": "Fair distribution", 2024-04-26T03:38:46.373 INFO:teuthology.orchestra.run.smithi028.stdout: "score_acting": 3.9900000095367432, 2024-04-26T03:38:46.373 INFO:teuthology.orchestra.run.smithi028.stdout: "score_stable": 3.9900000095367432, 2024-04-26T03:38:46.373 INFO:teuthology.orchestra.run.smithi028.stdout: "optimal_score": 0.68999999761581421, 2024-04-26T03:38:46.373 INFO:teuthology.orchestra.run.smithi028.stdout: "raw_score_acting": 2.75, 2024-04-26T03:38:46.373 INFO:teuthology.orchestra.run.smithi028.stdout: "raw_score_stable": 2.75, 2024-04-26T03:38:46.373 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity_weighted": 1, 2024-04-26T03:38:46.373 INFO:teuthology.orchestra.run.smithi028.stdout: "average_primary_affinity": 1, 2024-04-26T03:38:46.373 INFO:teuthology.orchestra.run.smithi028.stdout: "average_primary_affinity_weighted": 1 2024-04-26T03:38:46.373 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.373 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.373 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-04-26T03:38:46.374 INFO:teuthology.orchestra.run.smithi028.stdout: "osds": [ 2024-04-26T03:38:46.374 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.374 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 0, 2024-04-26T03:38:46.374 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "31b0e7a9-455d-4874-9c60-26efac8d0b1e", 2024-04-26T03:38:46.374 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:38:46.374 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:38:46.374 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:38:46.374 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:38:46.374 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:38:46.374 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:38:46.374 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 23, 2024-04-26T03:38:46.374 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 0, 2024-04-26T03:38:46.374 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:38:46.375 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:38:46.375 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:38:46.375 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.375 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.375 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.375 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6800", 2024-04-26T03:38:46.375 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377656090 2024-04-26T03:38:46.375 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.375 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.375 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.375 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6801", 2024-04-26T03:38:46.375 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377656090 2024-04-26T03:38:46.375 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.375 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.376 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.376 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:38:46.376 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.376 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.376 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.376 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6802", 2024-04-26T03:38:46.376 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377656090 2024-04-26T03:38:46.376 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.376 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.376 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.376 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6803", 2024-04-26T03:38:46.376 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377656090 2024-04-26T03:38:46.377 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.377 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.377 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.377 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:38:46.377 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.377 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.377 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.377 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6806", 2024-04-26T03:38:46.377 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377656090 2024-04-26T03:38:46.377 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.377 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.378 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.378 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6807", 2024-04-26T03:38:46.378 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377656090 2024-04-26T03:38:46.378 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.378 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.378 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.378 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:38:46.378 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.378 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.378 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.378 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6804", 2024-04-26T03:38:46.378 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377656090 2024-04-26T03:38:46.378 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.378 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.379 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.379 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6805", 2024-04-26T03:38:46.379 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377656090 2024-04-26T03:38:46.379 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.379 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.379 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.379 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.28:6801/2377656090", 2024-04-26T03:38:46.379 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.28:6803/2377656090", 2024-04-26T03:38:46.379 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.28:6807/2377656090", 2024-04-26T03:38:46.379 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.28:6805/2377656090", 2024-04-26T03:38:46.379 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:38:46.379 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:38:46.379 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:38:46.379 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.380 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.380 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.380 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 1, 2024-04-26T03:38:46.380 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "52819ace-a070-4822-b33e-3c54466e61ff", 2024-04-26T03:38:46.380 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:38:46.380 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:38:46.380 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:38:46.380 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:38:46.380 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:38:46.380 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:38:46.380 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 23, 2024-04-26T03:38:46.380 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 0, 2024-04-26T03:38:46.380 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:38:46.380 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:38:46.381 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:38:46.381 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.381 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.381 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.381 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6800", 2024-04-26T03:38:46.381 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1678150206 2024-04-26T03:38:46.381 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.381 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.381 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.381 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6801", 2024-04-26T03:38:46.382 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1678150206 2024-04-26T03:38:46.382 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.382 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.382 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.382 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:38:46.382 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.382 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.382 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.382 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6802", 2024-04-26T03:38:46.382 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1678150206 2024-04-26T03:38:46.382 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.382 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.382 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.382 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6803", 2024-04-26T03:38:46.383 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1678150206 2024-04-26T03:38:46.383 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.383 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.383 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.383 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:38:46.383 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.383 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.383 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.383 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6806", 2024-04-26T03:38:46.383 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1678150206 2024-04-26T03:38:46.383 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.383 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.383 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.383 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6807", 2024-04-26T03:38:46.384 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1678150206 2024-04-26T03:38:46.384 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.384 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.384 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.384 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:38:46.384 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.384 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.384 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.384 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6804", 2024-04-26T03:38:46.384 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1678150206 2024-04-26T03:38:46.384 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.384 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.384 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.384 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6805", 2024-04-26T03:38:46.385 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1678150206 2024-04-26T03:38:46.385 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.385 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.385 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.385 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.42:6801/1678150206", 2024-04-26T03:38:46.385 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.42:6803/1678150206", 2024-04-26T03:38:46.385 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.42:6807/1678150206", 2024-04-26T03:38:46.385 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.42:6805/1678150206", 2024-04-26T03:38:46.385 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:38:46.385 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:38:46.385 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:38:46.385 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.385 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.385 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.386 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 2, 2024-04-26T03:38:46.386 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "858f8ac1-2edc-43d5-914c-c1914bdc0c9a", 2024-04-26T03:38:46.386 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:38:46.386 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:38:46.386 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:38:46.386 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:38:46.386 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:38:46.386 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:38:46.386 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 23, 2024-04-26T03:38:46.386 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 0, 2024-04-26T03:38:46.386 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:38:46.386 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:38:46.386 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:38:46.386 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.386 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.387 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.387 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6800", 2024-04-26T03:38:46.387 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 714535415 2024-04-26T03:38:46.387 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.387 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.387 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.387 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6801", 2024-04-26T03:38:46.387 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 714535415 2024-04-26T03:38:46.387 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.388 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.388 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.388 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:38:46.388 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.388 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.388 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.388 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6802", 2024-04-26T03:38:46.388 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 714535415 2024-04-26T03:38:46.388 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.389 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.389 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.389 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6803", 2024-04-26T03:38:46.389 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 714535415 2024-04-26T03:38:46.389 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.389 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.389 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.389 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:38:46.389 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.389 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.389 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.389 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6806", 2024-04-26T03:38:46.389 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 714535415 2024-04-26T03:38:46.389 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.389 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.390 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.390 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6807", 2024-04-26T03:38:46.390 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 714535415 2024-04-26T03:38:46.390 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.390 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.390 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.390 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:38:46.390 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.390 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.390 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.390 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6804", 2024-04-26T03:38:46.390 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 714535415 2024-04-26T03:38:46.390 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.391 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.391 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.391 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6805", 2024-04-26T03:38:46.391 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 714535415 2024-04-26T03:38:46.391 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.391 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.391 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.391 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.66:6801/714535415", 2024-04-26T03:38:46.391 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.66:6803/714535415", 2024-04-26T03:38:46.391 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.66:6807/714535415", 2024-04-26T03:38:46.391 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.66:6805/714535415", 2024-04-26T03:38:46.391 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:38:46.391 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:38:46.391 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:38:46.391 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.392 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.392 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.392 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 3, 2024-04-26T03:38:46.392 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "531a2e30-3351-4f84-804b-5fb5050b84ed", 2024-04-26T03:38:46.392 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:38:46.392 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:38:46.392 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:38:46.392 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:38:46.392 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:38:46.392 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:38:46.392 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 24, 2024-04-26T03:38:46.392 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 26, 2024-04-26T03:38:46.392 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:38:46.392 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:38:46.393 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:38:46.393 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.393 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.393 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.393 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6810", 2024-04-26T03:38:46.393 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377983625 2024-04-26T03:38:46.393 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.393 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.393 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.393 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6811", 2024-04-26T03:38:46.393 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377983625 2024-04-26T03:38:46.393 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.393 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.393 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.393 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:38:46.394 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.394 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.394 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.394 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6812", 2024-04-26T03:38:46.394 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377983625 2024-04-26T03:38:46.394 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.394 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.394 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.394 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6813", 2024-04-26T03:38:46.394 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377983625 2024-04-26T03:38:46.394 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.394 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.394 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.394 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:38:46.395 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.395 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.395 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.395 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6816", 2024-04-26T03:38:46.395 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377983625 2024-04-26T03:38:46.395 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.395 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.395 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.395 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6817", 2024-04-26T03:38:46.395 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377983625 2024-04-26T03:38:46.395 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.395 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.395 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.395 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:38:46.396 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.396 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.396 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.396 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6814", 2024-04-26T03:38:46.396 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377983625 2024-04-26T03:38:46.396 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.396 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.396 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.396 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6815", 2024-04-26T03:38:46.396 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377983625 2024-04-26T03:38:46.396 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.396 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.396 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.396 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.92:6811/2377983625", 2024-04-26T03:38:46.397 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.92:6813/2377983625", 2024-04-26T03:38:46.397 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.92:6817/2377983625", 2024-04-26T03:38:46.397 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.92:6815/2377983625", 2024-04-26T03:38:46.397 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:38:46.397 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:38:46.397 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:38:46.397 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.397 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.397 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.397 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 4, 2024-04-26T03:38:46.398 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "6d38ffda-40eb-4fd7-9300-2a09f2673e66", 2024-04-26T03:38:46.398 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:38:46.398 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:38:46.398 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:38:46.398 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:38:46.398 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:38:46.398 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:38:46.398 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 23, 2024-04-26T03:38:46.398 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 26, 2024-04-26T03:38:46.398 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:38:46.398 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:38:46.398 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:38:46.398 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.399 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.399 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.399 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6808", 2024-04-26T03:38:46.399 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 183752455 2024-04-26T03:38:46.399 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.399 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.399 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.399 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6809", 2024-04-26T03:38:46.399 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 183752455 2024-04-26T03:38:46.399 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.399 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.399 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.399 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:38:46.399 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.399 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.400 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.400 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6810", 2024-04-26T03:38:46.400 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 183752455 2024-04-26T03:38:46.400 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.400 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.400 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.400 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6811", 2024-04-26T03:38:46.400 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 183752455 2024-04-26T03:38:46.400 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.400 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.400 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.400 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:38:46.400 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.400 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.401 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.401 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6814", 2024-04-26T03:38:46.401 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 183752455 2024-04-26T03:38:46.401 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.401 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.401 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.401 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6815", 2024-04-26T03:38:46.401 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 183752455 2024-04-26T03:38:46.401 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.401 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.401 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.401 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:38:46.401 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.401 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.402 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.402 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6812", 2024-04-26T03:38:46.402 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 183752455 2024-04-26T03:38:46.402 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.402 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.402 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.402 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6813", 2024-04-26T03:38:46.402 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 183752455 2024-04-26T03:38:46.402 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.402 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.402 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.402 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.28:6809/183752455", 2024-04-26T03:38:46.402 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.28:6811/183752455", 2024-04-26T03:38:46.402 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.28:6815/183752455", 2024-04-26T03:38:46.403 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.28:6813/183752455", 2024-04-26T03:38:46.403 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:38:46.403 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:38:46.403 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:38:46.403 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.403 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.403 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.403 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 5, 2024-04-26T03:38:46.403 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "361aa40d-e53b-42af-afd4-81fbcac28725", 2024-04-26T03:38:46.403 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:38:46.403 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:38:46.403 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:38:46.403 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:38:46.403 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:38:46.404 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:38:46.404 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 24, 2024-04-26T03:38:46.404 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 0, 2024-04-26T03:38:46.404 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:38:46.404 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:38:46.404 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:38:46.404 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.404 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.404 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.404 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6808", 2024-04-26T03:38:46.404 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2012261939 2024-04-26T03:38:46.404 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.404 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.404 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.405 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6809", 2024-04-26T03:38:46.405 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2012261939 2024-04-26T03:38:46.405 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.405 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.405 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.405 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:38:46.405 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.405 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.405 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.405 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6810", 2024-04-26T03:38:46.405 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2012261939 2024-04-26T03:38:46.405 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.405 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.405 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.406 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6811", 2024-04-26T03:38:46.406 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2012261939 2024-04-26T03:38:46.406 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.406 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.406 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.406 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:38:46.406 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.406 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.406 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.406 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6814", 2024-04-26T03:38:46.406 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2012261939 2024-04-26T03:38:46.406 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.406 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.407 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.407 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6815", 2024-04-26T03:38:46.407 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2012261939 2024-04-26T03:38:46.407 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.407 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.407 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.407 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:38:46.407 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.408 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.408 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.408 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6812", 2024-04-26T03:38:46.408 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2012261939 2024-04-26T03:38:46.408 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.408 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.408 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.408 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6813", 2024-04-26T03:38:46.408 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2012261939 2024-04-26T03:38:46.408 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.408 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.408 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.409 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.42:6809/2012261939", 2024-04-26T03:38:46.409 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.42:6811/2012261939", 2024-04-26T03:38:46.409 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.42:6815/2012261939", 2024-04-26T03:38:46.409 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.42:6813/2012261939", 2024-04-26T03:38:46.409 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:38:46.409 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:38:46.409 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:38:46.409 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.409 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.409 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.409 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 6, 2024-04-26T03:38:46.409 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "f610ad3e-1fd8-42bd-af4b-0d79c3bb9cf8", 2024-04-26T03:38:46.409 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:38:46.409 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:38:46.410 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:38:46.410 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:38:46.410 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:38:46.410 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:38:46.410 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 22, 2024-04-26T03:38:46.410 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 24, 2024-04-26T03:38:46.410 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:38:46.410 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:38:46.410 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:38:46.410 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.410 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.410 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.410 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6808", 2024-04-26T03:38:46.411 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3533460882 2024-04-26T03:38:46.411 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.411 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.411 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.411 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6809", 2024-04-26T03:38:46.411 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3533460882 2024-04-26T03:38:46.411 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.411 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.411 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.411 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:38:46.411 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.411 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.411 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.411 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6810", 2024-04-26T03:38:46.412 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3533460882 2024-04-26T03:38:46.412 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.412 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.412 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.412 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6811", 2024-04-26T03:38:46.412 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3533460882 2024-04-26T03:38:46.412 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.412 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.412 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.412 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:38:46.412 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.413 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.413 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.413 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6814", 2024-04-26T03:38:46.413 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3533460882 2024-04-26T03:38:46.413 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.413 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.413 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.413 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6815", 2024-04-26T03:38:46.413 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3533460882 2024-04-26T03:38:46.413 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.413 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.413 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.413 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:38:46.413 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.414 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.414 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.414 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6812", 2024-04-26T03:38:46.414 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3533460882 2024-04-26T03:38:46.414 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.414 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.414 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.414 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6813", 2024-04-26T03:38:46.414 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3533460882 2024-04-26T03:38:46.414 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.414 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.414 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.414 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.66:6809/3533460882", 2024-04-26T03:38:46.414 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.66:6811/3533460882", 2024-04-26T03:38:46.415 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.66:6815/3533460882", 2024-04-26T03:38:46.415 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.66:6813/3533460882", 2024-04-26T03:38:46.415 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:38:46.415 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:38:46.415 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:38:46.415 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.415 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.415 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.415 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 7, 2024-04-26T03:38:46.415 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "a4bae6fb-e087-4563-a129-6938e18eed8c", 2024-04-26T03:38:46.415 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:38:46.416 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:38:46.416 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:38:46.416 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:38:46.416 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:38:46.416 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:38:46.416 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 22, 2024-04-26T03:38:46.416 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 0, 2024-04-26T03:38:46.416 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:38:46.416 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:38:46.416 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:38:46.416 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.416 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.416 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.416 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6802", 2024-04-26T03:38:46.417 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1760163933 2024-04-26T03:38:46.417 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.417 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.417 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.417 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6803", 2024-04-26T03:38:46.417 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1760163933 2024-04-26T03:38:46.417 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.417 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.417 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.417 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:38:46.418 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.418 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.418 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.418 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6804", 2024-04-26T03:38:46.418 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1760163933 2024-04-26T03:38:46.418 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.418 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.418 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.418 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6805", 2024-04-26T03:38:46.418 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1760163933 2024-04-26T03:38:46.419 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.419 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.419 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.419 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:38:46.419 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.419 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.419 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.419 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6808", 2024-04-26T03:38:46.419 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1760163933 2024-04-26T03:38:46.419 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.419 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.419 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.419 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6809", 2024-04-26T03:38:46.419 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1760163933 2024-04-26T03:38:46.419 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.420 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.420 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.420 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:38:46.420 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.420 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.420 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.420 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6806", 2024-04-26T03:38:46.420 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1760163933 2024-04-26T03:38:46.420 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.420 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.420 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.420 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6807", 2024-04-26T03:38:46.420 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1760163933 2024-04-26T03:38:46.420 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.421 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.421 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.421 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.92:6803/1760163933", 2024-04-26T03:38:46.421 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.92:6805/1760163933", 2024-04-26T03:38:46.421 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.92:6809/1760163933", 2024-04-26T03:38:46.421 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.92:6807/1760163933", 2024-04-26T03:38:46.421 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:38:46.421 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:38:46.421 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:38:46.421 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.421 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.421 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.421 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 8, 2024-04-26T03:38:46.421 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "479f3885-452e-4231-bdc2-b79668384ba6", 2024-04-26T03:38:46.421 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:38:46.422 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:38:46.422 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:38:46.422 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:38:46.422 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:38:46.422 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:38:46.422 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 22, 2024-04-26T03:38:46.422 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 26, 2024-04-26T03:38:46.422 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:38:46.422 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:38:46.422 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:38:46.422 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.422 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.422 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.422 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6816", 2024-04-26T03:38:46.423 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4127602407 2024-04-26T03:38:46.423 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.423 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.423 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.423 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6817", 2024-04-26T03:38:46.423 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4127602407 2024-04-26T03:38:46.423 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.423 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.423 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.423 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:38:46.423 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.423 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.423 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.424 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6818", 2024-04-26T03:38:46.424 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4127602407 2024-04-26T03:38:46.424 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.424 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.424 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.424 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6819", 2024-04-26T03:38:46.424 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4127602407 2024-04-26T03:38:46.424 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.424 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.424 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.424 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:38:46.424 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.424 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.424 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.425 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6822", 2024-04-26T03:38:46.425 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4127602407 2024-04-26T03:38:46.425 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.425 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.425 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.425 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6823", 2024-04-26T03:38:46.425 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4127602407 2024-04-26T03:38:46.425 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.425 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.425 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.425 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:38:46.425 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.425 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.425 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.425 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6820", 2024-04-26T03:38:46.426 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4127602407 2024-04-26T03:38:46.426 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.426 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.426 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.426 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6821", 2024-04-26T03:38:46.426 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4127602407 2024-04-26T03:38:46.426 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.426 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.426 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.426 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.28:6817/4127602407", 2024-04-26T03:38:46.426 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.28:6819/4127602407", 2024-04-26T03:38:46.426 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.28:6823/4127602407", 2024-04-26T03:38:46.426 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.28:6821/4127602407", 2024-04-26T03:38:46.427 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:38:46.427 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:38:46.427 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:38:46.427 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.427 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.427 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.427 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 9, 2024-04-26T03:38:46.427 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "9a603a47-7c5a-47d4-a298-56b3ac991892", 2024-04-26T03:38:46.427 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:38:46.427 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:38:46.427 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:38:46.427 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:38:46.427 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:38:46.427 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:38:46.428 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 22, 2024-04-26T03:38:46.428 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 26, 2024-04-26T03:38:46.428 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:38:46.428 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:38:46.428 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:38:46.428 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.428 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.428 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.428 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6816", 2024-04-26T03:38:46.428 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4168729845 2024-04-26T03:38:46.428 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.428 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.428 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.428 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6818", 2024-04-26T03:38:46.429 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4168729845 2024-04-26T03:38:46.429 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.429 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.429 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.429 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:38:46.429 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.429 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.429 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.429 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6820", 2024-04-26T03:38:46.429 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4168729845 2024-04-26T03:38:46.429 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.429 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.429 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.429 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6822", 2024-04-26T03:38:46.430 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4168729845 2024-04-26T03:38:46.430 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.430 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.430 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.430 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:38:46.430 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.430 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.430 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.430 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6828", 2024-04-26T03:38:46.430 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4168729845 2024-04-26T03:38:46.430 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.430 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.430 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.430 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6830", 2024-04-26T03:38:46.431 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4168729845 2024-04-26T03:38:46.431 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.431 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.431 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.431 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:38:46.431 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.431 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.431 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.431 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6824", 2024-04-26T03:38:46.431 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4168729845 2024-04-26T03:38:46.431 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.431 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.432 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.432 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6826", 2024-04-26T03:38:46.432 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4168729845 2024-04-26T03:38:46.432 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.432 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.432 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.432 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.42:6818/4168729845", 2024-04-26T03:38:46.432 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.42:6822/4168729845", 2024-04-26T03:38:46.432 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.42:6830/4168729845", 2024-04-26T03:38:46.432 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.42:6826/4168729845", 2024-04-26T03:38:46.432 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:38:46.432 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:38:46.432 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:38:46.432 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.432 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.433 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.433 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 10, 2024-04-26T03:38:46.433 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "1cd42945-6ee1-49f0-b17b-4093dd36b4ec", 2024-04-26T03:38:46.433 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:38:46.433 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:38:46.433 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:38:46.433 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:38:46.433 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:38:46.433 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:38:46.433 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 22, 2024-04-26T03:38:46.433 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 0, 2024-04-26T03:38:46.433 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:38:46.433 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:38:46.433 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:38:46.434 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.434 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.434 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.434 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6816", 2024-04-26T03:38:46.434 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4180572718 2024-04-26T03:38:46.434 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.434 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.434 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.434 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6817", 2024-04-26T03:38:46.434 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4180572718 2024-04-26T03:38:46.434 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.434 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.434 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.434 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:38:46.435 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.435 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.435 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.435 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6818", 2024-04-26T03:38:46.435 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4180572718 2024-04-26T03:38:46.435 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.435 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.435 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.435 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6819", 2024-04-26T03:38:46.435 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4180572718 2024-04-26T03:38:46.435 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.435 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.435 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.435 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:38:46.435 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.436 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.436 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.436 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6822", 2024-04-26T03:38:46.436 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4180572718 2024-04-26T03:38:46.436 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.436 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.436 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.436 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6823", 2024-04-26T03:38:46.436 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4180572718 2024-04-26T03:38:46.436 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.436 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.436 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.436 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:38:46.436 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.437 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.437 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.437 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6820", 2024-04-26T03:38:46.437 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4180572718 2024-04-26T03:38:46.437 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.437 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.437 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.437 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6821", 2024-04-26T03:38:46.437 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4180572718 2024-04-26T03:38:46.437 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.437 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.437 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.437 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.66:6817/4180572718", 2024-04-26T03:38:46.437 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.66:6819/4180572718", 2024-04-26T03:38:46.437 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.66:6823/4180572718", 2024-04-26T03:38:46.438 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.66:6821/4180572718", 2024-04-26T03:38:46.438 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:38:46.438 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:38:46.438 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:38:46.438 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.438 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.438 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.438 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 11, 2024-04-26T03:38:46.438 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "b41e8100-1d4f-4b7a-991a-7a27bc29feec", 2024-04-26T03:38:46.438 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:38:46.438 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:38:46.438 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:38:46.438 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:38:46.439 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:38:46.439 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:38:46.439 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 21, 2024-04-26T03:38:46.439 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 0, 2024-04-26T03:38:46.439 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:38:46.439 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:38:46.439 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:38:46.439 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.439 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.439 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.439 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6818", 2024-04-26T03:38:46.439 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 295713815 2024-04-26T03:38:46.439 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.439 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.439 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.440 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6819", 2024-04-26T03:38:46.440 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 295713815 2024-04-26T03:38:46.440 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.440 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.440 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.440 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:38:46.440 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.440 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.440 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.440 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6820", 2024-04-26T03:38:46.440 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 295713815 2024-04-26T03:38:46.440 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.440 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.440 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.440 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6821", 2024-04-26T03:38:46.441 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 295713815 2024-04-26T03:38:46.441 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.441 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.441 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.441 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:38:46.441 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.441 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.441 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.441 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6824", 2024-04-26T03:38:46.441 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 295713815 2024-04-26T03:38:46.441 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.441 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.441 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.441 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6825", 2024-04-26T03:38:46.442 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 295713815 2024-04-26T03:38:46.442 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.442 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.442 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.442 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:38:46.442 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.442 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.442 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.442 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6822", 2024-04-26T03:38:46.442 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 295713815 2024-04-26T03:38:46.442 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.442 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.442 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.442 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6823", 2024-04-26T03:38:46.443 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 295713815 2024-04-26T03:38:46.443 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.443 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.443 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.443 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.92:6819/295713815", 2024-04-26T03:38:46.443 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.92:6821/295713815", 2024-04-26T03:38:46.443 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.92:6825/295713815", 2024-04-26T03:38:46.443 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.92:6823/295713815", 2024-04-26T03:38:46.443 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:38:46.443 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:38:46.443 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:38:46.443 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.443 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.443 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.443 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 12, 2024-04-26T03:38:46.444 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "458cf150-32e1-449e-b533-ac6ab5226f85", 2024-04-26T03:38:46.444 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:38:46.444 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:38:46.444 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:38:46.444 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:38:46.444 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:38:46.444 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:38:46.444 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 22, 2024-04-26T03:38:46.444 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 0, 2024-04-26T03:38:46.444 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:38:46.444 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:38:46.444 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:38:46.444 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.444 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.445 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.445 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6824", 2024-04-26T03:38:46.445 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2081141953 2024-04-26T03:38:46.445 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.445 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.445 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.445 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6825", 2024-04-26T03:38:46.445 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2081141953 2024-04-26T03:38:46.445 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.445 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.445 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.445 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:38:46.445 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.445 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.446 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.446 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6826", 2024-04-26T03:38:46.446 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2081141953 2024-04-26T03:38:46.446 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.446 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.446 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.446 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6827", 2024-04-26T03:38:46.446 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2081141953 2024-04-26T03:38:46.446 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.446 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.446 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.446 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:38:46.446 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.446 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.447 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.447 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6830", 2024-04-26T03:38:46.447 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2081141953 2024-04-26T03:38:46.447 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.447 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.447 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.447 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6831", 2024-04-26T03:38:46.447 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2081141953 2024-04-26T03:38:46.447 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.447 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.447 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.447 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:38:46.447 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.447 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.447 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.448 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6828", 2024-04-26T03:38:46.448 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2081141953 2024-04-26T03:38:46.448 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.448 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.448 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.448 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6829", 2024-04-26T03:38:46.448 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2081141953 2024-04-26T03:38:46.448 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.448 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.448 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.448 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.28:6825/2081141953", 2024-04-26T03:38:46.448 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.28:6827/2081141953", 2024-04-26T03:38:46.448 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.28:6831/2081141953", 2024-04-26T03:38:46.448 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.28:6829/2081141953", 2024-04-26T03:38:46.448 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:38:46.449 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:38:46.449 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:38:46.449 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.449 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.449 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.449 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 13, 2024-04-26T03:38:46.449 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "f008462e-a735-4845-9ecf-dd7d4a6941af", 2024-04-26T03:38:46.449 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:38:46.449 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:38:46.449 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:38:46.449 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:38:46.449 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:38:46.449 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:38:46.449 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 22, 2024-04-26T03:38:46.450 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 0, 2024-04-26T03:38:46.450 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:38:46.450 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:38:46.450 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:38:46.450 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.450 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.450 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.450 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6817", 2024-04-26T03:38:46.450 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 301360893 2024-04-26T03:38:46.450 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.450 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.450 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.450 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6819", 2024-04-26T03:38:46.450 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 301360893 2024-04-26T03:38:46.450 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.451 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.451 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.451 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:38:46.451 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.451 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.451 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.451 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6821", 2024-04-26T03:38:46.451 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 301360893 2024-04-26T03:38:46.451 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.451 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.451 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.451 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6823", 2024-04-26T03:38:46.451 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 301360893 2024-04-26T03:38:46.451 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.452 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.452 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.452 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:38:46.452 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.452 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.452 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.452 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6829", 2024-04-26T03:38:46.452 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 301360893 2024-04-26T03:38:46.452 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.452 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.452 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.452 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6831", 2024-04-26T03:38:46.452 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 301360893 2024-04-26T03:38:46.452 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.453 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.453 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.453 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:38:46.453 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.453 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.453 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.453 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6825", 2024-04-26T03:38:46.453 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 301360893 2024-04-26T03:38:46.453 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.453 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.453 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.453 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6827", 2024-04-26T03:38:46.453 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 301360893 2024-04-26T03:38:46.453 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.453 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.454 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.454 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.42:6819/301360893", 2024-04-26T03:38:46.454 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.42:6823/301360893", 2024-04-26T03:38:46.454 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.42:6831/301360893", 2024-04-26T03:38:46.454 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.42:6827/301360893", 2024-04-26T03:38:46.454 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:38:46.454 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:38:46.454 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:38:46.454 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.454 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.454 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.454 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 14, 2024-04-26T03:38:46.454 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "6f66ae97-eb8a-48eb-9bcc-20f65fbf690b", 2024-04-26T03:38:46.454 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:38:46.455 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:38:46.455 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:38:46.455 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:38:46.455 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:38:46.455 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:38:46.455 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 22, 2024-04-26T03:38:46.455 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 26, 2024-04-26T03:38:46.455 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:38:46.455 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:38:46.455 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:38:46.455 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.455 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.455 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.455 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6824", 2024-04-26T03:38:46.455 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1534905602 2024-04-26T03:38:46.456 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.456 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.456 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.456 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6825", 2024-04-26T03:38:46.456 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1534905602 2024-04-26T03:38:46.456 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.456 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.456 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.456 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:38:46.456 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.456 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.456 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.456 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6826", 2024-04-26T03:38:46.456 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1534905602 2024-04-26T03:38:46.456 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.457 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.457 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.457 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6827", 2024-04-26T03:38:46.457 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1534905602 2024-04-26T03:38:46.457 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.457 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.457 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.457 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:38:46.457 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.457 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.457 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.457 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6830", 2024-04-26T03:38:46.457 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1534905602 2024-04-26T03:38:46.457 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.458 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.458 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.458 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6831", 2024-04-26T03:38:46.458 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1534905602 2024-04-26T03:38:46.458 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.458 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.458 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.458 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:38:46.458 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.458 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.458 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.458 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6828", 2024-04-26T03:38:46.458 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1534905602 2024-04-26T03:38:46.459 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.459 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.459 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.459 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6829", 2024-04-26T03:38:46.459 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1534905602 2024-04-26T03:38:46.459 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.459 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.459 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.459 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.66:6825/1534905602", 2024-04-26T03:38:46.459 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.66:6827/1534905602", 2024-04-26T03:38:46.459 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.66:6831/1534905602", 2024-04-26T03:38:46.459 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.66:6829/1534905602", 2024-04-26T03:38:46.459 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:38:46.459 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:38:46.460 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:38:46.460 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.460 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.460 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.460 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 15, 2024-04-26T03:38:46.460 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "b856bfc8-253b-4112-be5c-f67d61d01768", 2024-04-26T03:38:46.460 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:38:46.460 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:38:46.460 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:38:46.460 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:38:46.460 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:38:46.460 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:38:46.460 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 22, 2024-04-26T03:38:46.460 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 26, 2024-04-26T03:38:46.460 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:38:46.461 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:38:46.461 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:38:46.461 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.461 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.461 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.461 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6826", 2024-04-26T03:38:46.461 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3982634323 2024-04-26T03:38:46.461 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.461 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.461 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.461 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6827", 2024-04-26T03:38:46.461 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3982634323 2024-04-26T03:38:46.461 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.461 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.461 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.462 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:38:46.462 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.462 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.462 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.462 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6828", 2024-04-26T03:38:46.462 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3982634323 2024-04-26T03:38:46.462 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.462 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.462 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.462 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6829", 2024-04-26T03:38:46.462 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3982634323 2024-04-26T03:38:46.462 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.462 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.462 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.463 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:38:46.463 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.463 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.463 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.463 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6832", 2024-04-26T03:38:46.463 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3982634323 2024-04-26T03:38:46.463 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.463 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.463 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.463 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6833", 2024-04-26T03:38:46.463 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3982634323 2024-04-26T03:38:46.463 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.463 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.463 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.463 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:38:46.464 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:38:46.464 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.464 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:38:46.464 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6830", 2024-04-26T03:38:46.464 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3982634323 2024-04-26T03:38:46.464 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.464 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.464 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:38:46.464 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6831", 2024-04-26T03:38:46.464 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3982634323 2024-04-26T03:38:46.464 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.464 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.464 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.464 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.92:6827/3982634323", 2024-04-26T03:38:46.465 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.92:6829/3982634323", 2024-04-26T03:38:46.465 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.92:6833/3982634323", 2024-04-26T03:38:46.465 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.92:6831/3982634323", 2024-04-26T03:38:46.465 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:38:46.465 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:38:46.465 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:38:46.465 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:46.465 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.465 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-04-26T03:38:46.465 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_xinfo": [ 2024-04-26T03:38:46.465 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.465 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 0, 2024-04-26T03:38:46.465 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:38:46.465 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:38:46.466 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:38:46.466 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:38:46.466 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:38:46.466 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.265053+0000", 2024-04-26T03:38:46.466 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:38:46.466 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.466 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.466 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 1, 2024-04-26T03:38:46.466 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:38:46.466 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:38:46.466 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:38:46.466 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:38:46.466 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:38:46.466 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.243502+0000", 2024-04-26T03:38:46.466 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:38:46.467 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.467 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.467 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 2, 2024-04-26T03:38:46.467 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:38:46.467 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:38:46.467 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:38:46.467 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:38:46.467 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:38:46.467 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.252440+0000", 2024-04-26T03:38:46.467 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:38:46.467 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.467 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.467 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 3, 2024-04-26T03:38:46.467 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:38:46.467 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:38:46.468 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:38:46.468 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:38:46.468 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:38:46.468 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.295331+0000", 2024-04-26T03:38:46.468 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:38:46.468 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.468 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.468 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 4, 2024-04-26T03:38:46.468 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:38:46.468 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:38:46.468 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:38:46.468 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:38:46.468 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:38:46.468 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.213076+0000", 2024-04-26T03:38:46.469 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:38:46.469 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.469 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.469 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 5, 2024-04-26T03:38:46.469 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:38:46.469 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:38:46.469 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:38:46.469 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:38:46.469 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:38:46.469 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.281193+0000", 2024-04-26T03:38:46.469 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:38:46.469 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.469 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.469 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 6, 2024-04-26T03:38:46.470 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:38:46.470 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:38:46.470 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:38:46.470 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:38:46.470 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:38:46.470 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.315801+0000", 2024-04-26T03:38:46.470 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:38:46.470 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.470 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.470 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 7, 2024-04-26T03:38:46.470 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:38:46.470 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:38:46.470 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:38:46.470 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:38:46.471 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:38:46.471 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.329538+0000", 2024-04-26T03:38:46.471 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:38:46.471 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.471 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.471 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 8, 2024-04-26T03:38:46.471 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:38:46.471 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:38:46.471 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:38:46.471 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:38:46.471 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:38:46.471 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.293303+0000", 2024-04-26T03:38:46.472 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:38:46.472 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.472 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.472 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 9, 2024-04-26T03:38:46.472 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:38:46.472 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:38:46.472 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:38:46.472 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:38:46.472 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:38:46.472 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.334962+0000", 2024-04-26T03:38:46.472 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:38:46.472 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.472 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.472 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 10, 2024-04-26T03:38:46.473 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:38:46.473 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:38:46.473 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:38:46.473 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:38:46.473 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:38:46.473 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.233032+0000", 2024-04-26T03:38:46.473 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:38:46.473 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.473 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.473 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 11, 2024-04-26T03:38:46.473 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:38:46.473 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:38:46.473 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:38:46.473 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:38:46.473 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:38:46.474 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.305138+0000", 2024-04-26T03:38:46.474 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:38:46.474 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.474 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.474 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 12, 2024-04-26T03:38:46.474 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:38:46.474 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:38:46.474 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:38:46.474 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:38:46.474 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:38:46.474 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.351964+0000", 2024-04-26T03:38:46.474 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:38:46.474 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.474 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.475 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 13, 2024-04-26T03:38:46.475 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:38:46.475 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:38:46.475 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:38:46.475 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:38:46.475 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:38:46.475 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.241086+0000", 2024-04-26T03:38:46.475 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:38:46.475 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.475 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.475 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 14, 2024-04-26T03:38:46.475 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:38:46.475 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:38:46.475 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:38:46.476 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:38:46.476 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:38:46.476 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.352515+0000", 2024-04-26T03:38:46.476 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:38:46.476 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.476 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:46.476 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 15, 2024-04-26T03:38:46.476 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:38:46.476 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:38:46.476 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:38:46.476 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:38:46.476 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:38:46.476 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.357462+0000", 2024-04-26T03:38:46.476 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:38:46.476 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.477 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-04-26T03:38:46.477 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_upmap": [], 2024-04-26T03:38:46.477 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_upmap_items": [], 2024-04-26T03:38:46.477 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_upmap_primaries": [], 2024-04-26T03:38:46.477 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_temp": [], 2024-04-26T03:38:46.477 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_temp": [], 2024-04-26T03:38:46.477 INFO:teuthology.orchestra.run.smithi028.stdout: "blocklist": {}, 2024-04-26T03:38:46.477 INFO:teuthology.orchestra.run.smithi028.stdout: "range_blocklist": {}, 2024-04-26T03:38:46.477 INFO:teuthology.orchestra.run.smithi028.stdout: "erasure_code_profiles": { 2024-04-26T03:38:46.477 INFO:teuthology.orchestra.run.smithi028.stdout: "default": { 2024-04-26T03:38:46.477 INFO:teuthology.orchestra.run.smithi028.stdout: "crush-failure-domain": "osd", 2024-04-26T03:38:46.477 INFO:teuthology.orchestra.run.smithi028.stdout: "k": "2", 2024-04-26T03:38:46.477 INFO:teuthology.orchestra.run.smithi028.stdout: "m": "1", 2024-04-26T03:38:46.477 INFO:teuthology.orchestra.run.smithi028.stdout: "plugin": "jerasure", 2024-04-26T03:38:46.478 INFO:teuthology.orchestra.run.smithi028.stdout: "technique": "reed_sol_van" 2024-04-26T03:38:46.478 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.478 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:46.478 INFO:teuthology.orchestra.run.smithi028.stdout: "removed_snaps_queue": [], 2024-04-26T03:38:46.478 INFO:teuthology.orchestra.run.smithi028.stdout: "new_removed_snaps": [], 2024-04-26T03:38:46.478 INFO:teuthology.orchestra.run.smithi028.stdout: "new_purged_snaps": [], 2024-04-26T03:38:46.478 INFO:teuthology.orchestra.run.smithi028.stdout: "crush_node_flags": {}, 2024-04-26T03:38:46.478 INFO:teuthology.orchestra.run.smithi028.stdout: "device_class_flags": {}, 2024-04-26T03:38:46.478 INFO:teuthology.orchestra.run.smithi028.stdout: "stretch_mode": { 2024-04-26T03:38:46.478 INFO:teuthology.orchestra.run.smithi028.stdout: "stretch_mode_enabled": false, 2024-04-26T03:38:46.478 INFO:teuthology.orchestra.run.smithi028.stdout: "stretch_bucket_count": 0, 2024-04-26T03:38:46.478 INFO:teuthology.orchestra.run.smithi028.stdout: "degraded_stretch_mode": 0, 2024-04-26T03:38:46.478 INFO:teuthology.orchestra.run.smithi028.stdout: "recovering_stretch_mode": 0, 2024-04-26T03:38:46.478 INFO:teuthology.orchestra.run.smithi028.stdout: "stretch_mode_bucket": 0 2024-04-26T03:38:46.479 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:46.479 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:38:46.480 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5: {} 2024-04-26T03:38:46.480 INFO:teuthology.orchestra.run.smithi028.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T03:38:46.483 INFO:tasks.thrashosds.thrasher:j is {'epoch': 29, 'fsid': 'c5a13c99-5c9a-4b8b-a893-4f6f4211677a', 'created': '2024-04-26T03:38:19.896648+0000', 'modified': '2024-04-26T03:38:38.431503+0000', 'last_up_change': '2024-04-26T03:38:33.308561+0000', 'last_in_change': '2024-04-26T03:38:24.150336+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 2, 'max_osd': 16, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T03:38:32.306696+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 15.380000114440918, 'score_stable': 15.380000114440918, 'optimal_score': 0.12999999523162842, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-04-26T03:38:34.592633+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '29', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3.990000009536743, 'score_stable': 3.990000009536743, 'optimal_score': 0.6899999976158142, 'raw_score_acting': 2.75, 'raw_score_stable': 2.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '31b0e7a9-455d-4874-9c60-26efac8d0b1e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6800', 'nonce': 2377656090}, {'type': 'v1', 'addr': '172.21.15.28:6801', 'nonce': 2377656090}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6802', 'nonce': 2377656090}, {'type': 'v1', 'addr': '172.21.15.28:6803', 'nonce': 2377656090}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6806', 'nonce': 2377656090}, {'type': 'v1', 'addr': '172.21.15.28:6807', 'nonce': 2377656090}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6804', 'nonce': 2377656090}, {'type': 'v1', 'addr': '172.21.15.28:6805', 'nonce': 2377656090}]}, 'public_addr': '172.21.15.28:6801/2377656090', 'cluster_addr': '172.21.15.28:6803/2377656090', 'heartbeat_back_addr': '172.21.15.28:6807/2377656090', 'heartbeat_front_addr': '172.21.15.28:6805/2377656090', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '52819ace-a070-4822-b33e-3c54466e61ff', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6800', 'nonce': 1678150206}, {'type': 'v1', 'addr': '172.21.15.42:6801', 'nonce': 1678150206}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6802', 'nonce': 1678150206}, {'type': 'v1', 'addr': '172.21.15.42:6803', 'nonce': 1678150206}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6806', 'nonce': 1678150206}, {'type': 'v1', 'addr': '172.21.15.42:6807', 'nonce': 1678150206}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6804', 'nonce': 1678150206}, {'type': 'v1', 'addr': '172.21.15.42:6805', 'nonce': 1678150206}]}, 'public_addr': '172.21.15.42:6801/1678150206', 'cluster_addr': '172.21.15.42:6803/1678150206', 'heartbeat_back_addr': '172.21.15.42:6807/1678150206', 'heartbeat_front_addr': '172.21.15.42:6805/1678150206', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '858f8ac1-2edc-43d5-914c-c1914bdc0c9a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6800', 'nonce': 714535415}, {'type': 'v1', 'addr': '172.21.15.66:6801', 'nonce': 714535415}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6802', 'nonce': 714535415}, {'type': 'v1', 'addr': '172.21.15.66:6803', 'nonce': 714535415}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6806', 'nonce': 714535415}, {'type': 'v1', 'addr': '172.21.15.66:6807', 'nonce': 714535415}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6804', 'nonce': 714535415}, {'type': 'v1', 'addr': '172.21.15.66:6805', 'nonce': 714535415}]}, 'public_addr': '172.21.15.66:6801/714535415', 'cluster_addr': '172.21.15.66:6803/714535415', 'heartbeat_back_addr': '172.21.15.66:6807/714535415', 'heartbeat_front_addr': '172.21.15.66:6805/714535415', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '531a2e30-3351-4f84-804b-5fb5050b84ed', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 24, 'up_thru': 26, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6810', 'nonce': 2377983625}, {'type': 'v1', 'addr': '172.21.15.92:6811', 'nonce': 2377983625}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6812', 'nonce': 2377983625}, {'type': 'v1', 'addr': '172.21.15.92:6813', 'nonce': 2377983625}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6816', 'nonce': 2377983625}, {'type': 'v1', 'addr': '172.21.15.92:6817', 'nonce': 2377983625}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6814', 'nonce': 2377983625}, {'type': 'v1', 'addr': '172.21.15.92:6815', 'nonce': 2377983625}]}, 'public_addr': '172.21.15.92:6811/2377983625', 'cluster_addr': '172.21.15.92:6813/2377983625', 'heartbeat_back_addr': '172.21.15.92:6817/2377983625', 'heartbeat_front_addr': '172.21.15.92:6815/2377983625', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '6d38ffda-40eb-4fd7-9300-2a09f2673e66', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 26, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6808', 'nonce': 183752455}, {'type': 'v1', 'addr': '172.21.15.28:6809', 'nonce': 183752455}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6810', 'nonce': 183752455}, {'type': 'v1', 'addr': '172.21.15.28:6811', 'nonce': 183752455}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6814', 'nonce': 183752455}, {'type': 'v1', 'addr': '172.21.15.28:6815', 'nonce': 183752455}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6812', 'nonce': 183752455}, {'type': 'v1', 'addr': '172.21.15.28:6813', 'nonce': 183752455}]}, 'public_addr': '172.21.15.28:6809/183752455', 'cluster_addr': '172.21.15.28:6811/183752455', 'heartbeat_back_addr': '172.21.15.28:6815/183752455', 'heartbeat_front_addr': '172.21.15.28:6813/183752455', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '361aa40d-e53b-42af-afd4-81fbcac28725', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 24, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6808', 'nonce': 2012261939}, {'type': 'v1', 'addr': '172.21.15.42:6809', 'nonce': 2012261939}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6810', 'nonce': 2012261939}, {'type': 'v1', 'addr': '172.21.15.42:6811', 'nonce': 2012261939}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6814', 'nonce': 2012261939}, {'type': 'v1', 'addr': '172.21.15.42:6815', 'nonce': 2012261939}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6812', 'nonce': 2012261939}, {'type': 'v1', 'addr': '172.21.15.42:6813', 'nonce': 2012261939}]}, 'public_addr': '172.21.15.42:6809/2012261939', 'cluster_addr': '172.21.15.42:6811/2012261939', 'heartbeat_back_addr': '172.21.15.42:6815/2012261939', 'heartbeat_front_addr': '172.21.15.42:6813/2012261939', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'f610ad3e-1fd8-42bd-af4b-0d79c3bb9cf8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6808', 'nonce': 3533460882}, {'type': 'v1', 'addr': '172.21.15.66:6809', 'nonce': 3533460882}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6810', 'nonce': 3533460882}, {'type': 'v1', 'addr': '172.21.15.66:6811', 'nonce': 3533460882}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6814', 'nonce': 3533460882}, {'type': 'v1', 'addr': '172.21.15.66:6815', 'nonce': 3533460882}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6812', 'nonce': 3533460882}, {'type': 'v1', 'addr': '172.21.15.66:6813', 'nonce': 3533460882}]}, 'public_addr': '172.21.15.66:6809/3533460882', 'cluster_addr': '172.21.15.66:6811/3533460882', 'heartbeat_back_addr': '172.21.15.66:6815/3533460882', 'heartbeat_front_addr': '172.21.15.66:6813/3533460882', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'a4bae6fb-e087-4563-a129-6938e18eed8c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6802', 'nonce': 1760163933}, {'type': 'v1', 'addr': '172.21.15.92:6803', 'nonce': 1760163933}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6804', 'nonce': 1760163933}, {'type': 'v1', 'addr': '172.21.15.92:6805', 'nonce': 1760163933}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6808', 'nonce': 1760163933}, {'type': 'v1', 'addr': '172.21.15.92:6809', 'nonce': 1760163933}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6806', 'nonce': 1760163933}, {'type': 'v1', 'addr': '172.21.15.92:6807', 'nonce': 1760163933}]}, 'public_addr': '172.21.15.92:6803/1760163933', 'cluster_addr': '172.21.15.92:6805/1760163933', 'heartbeat_back_addr': '172.21.15.92:6809/1760163933', 'heartbeat_front_addr': '172.21.15.92:6807/1760163933', 'state': ['exists', 'up']}, {'osd': 8, 'uuid': '479f3885-452e-4231-bdc2-b79668384ba6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 26, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6816', 'nonce': 4127602407}, {'type': 'v1', 'addr': '172.21.15.28:6817', 'nonce': 4127602407}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6818', 'nonce': 4127602407}, {'type': 'v1', 'addr': '172.21.15.28:6819', 'nonce': 4127602407}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6822', 'nonce': 4127602407}, {'type': 'v1', 'addr': '172.21.15.28:6823', 'nonce': 4127602407}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6820', 'nonce': 4127602407}, {'type': 'v1', 'addr': '172.21.15.28:6821', 'nonce': 4127602407}]}, 'public_addr': '172.21.15.28:6817/4127602407', 'cluster_addr': '172.21.15.28:6819/4127602407', 'heartbeat_back_addr': '172.21.15.28:6823/4127602407', 'heartbeat_front_addr': '172.21.15.28:6821/4127602407', 'state': ['exists', 'up']}, {'osd': 9, 'uuid': '9a603a47-7c5a-47d4-a298-56b3ac991892', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 26, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6816', 'nonce': 4168729845}, {'type': 'v1', 'addr': '172.21.15.42:6818', 'nonce': 4168729845}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6820', 'nonce': 4168729845}, {'type': 'v1', 'addr': '172.21.15.42:6822', 'nonce': 4168729845}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6828', 'nonce': 4168729845}, {'type': 'v1', 'addr': '172.21.15.42:6830', 'nonce': 4168729845}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6824', 'nonce': 4168729845}, {'type': 'v1', 'addr': '172.21.15.42:6826', 'nonce': 4168729845}]}, 'public_addr': '172.21.15.42:6818/4168729845', 'cluster_addr': '172.21.15.42:6822/4168729845', 'heartbeat_back_addr': '172.21.15.42:6830/4168729845', 'heartbeat_front_addr': '172.21.15.42:6826/4168729845', 'state': ['exists', 'up']}, {'osd': 10, 'uuid': '1cd42945-6ee1-49f0-b17b-4093dd36b4ec', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6816', 'nonce': 4180572718}, {'type': 'v1', 'addr': '172.21.15.66:6817', 'nonce': 4180572718}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6818', 'nonce': 4180572718}, {'type': 'v1', 'addr': '172.21.15.66:6819', 'nonce': 4180572718}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6822', 'nonce': 4180572718}, {'type': 'v1', 'addr': '172.21.15.66:6823', 'nonce': 4180572718}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6820', 'nonce': 4180572718}, {'type': 'v1', 'addr': '172.21.15.66:6821', 'nonce': 4180572718}]}, 'public_addr': '172.21.15.66:6817/4180572718', 'cluster_addr': '172.21.15.66:6819/4180572718', 'heartbeat_back_addr': '172.21.15.66:6823/4180572718', 'heartbeat_front_addr': '172.21.15.66:6821/4180572718', 'state': ['exists', 'up']}, {'osd': 11, 'uuid': 'b41e8100-1d4f-4b7a-991a-7a27bc29feec', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6818', 'nonce': 295713815}, {'type': 'v1', 'addr': '172.21.15.92:6819', 'nonce': 295713815}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6820', 'nonce': 295713815}, {'type': 'v1', 'addr': '172.21.15.92:6821', 'nonce': 295713815}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6824', 'nonce': 295713815}, {'type': 'v1', 'addr': '172.21.15.92:6825', 'nonce': 295713815}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6822', 'nonce': 295713815}, {'type': 'v1', 'addr': '172.21.15.92:6823', 'nonce': 295713815}]}, 'public_addr': '172.21.15.92:6819/295713815', 'cluster_addr': '172.21.15.92:6821/295713815', 'heartbeat_back_addr': '172.21.15.92:6825/295713815', 'heartbeat_front_addr': '172.21.15.92:6823/295713815', 'state': ['exists', 'up']}, {'osd': 12, 'uuid': '458cf150-32e1-449e-b533-ac6ab5226f85', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6824', 'nonce': 2081141953}, {'type': 'v1', 'addr': '172.21.15.28:6825', 'nonce': 2081141953}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6826', 'nonce': 2081141953}, {'type': 'v1', 'addr': '172.21.15.28:6827', 'nonce': 2081141953}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6830', 'nonce': 2081141953}, {'type': 'v1', 'addr': '172.21.15.28:6831', 'nonce': 2081141953}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6828', 'nonce': 2081141953}, {'type': 'v1', 'addr': '172.21.15.28:6829', 'nonce': 2081141953}]}, 'public_addr': '172.21.15.28:6825/2081141953', 'cluster_addr': '172.21.15.28:6827/2081141953', 'heartbeat_back_addr': '172.21.15.28:6831/2081141953', 'heartbeat_front_addr': '172.21.15.28:6829/2081141953', 'state': ['exists', 'up']}, {'osd': 13, 'uuid': 'f008462e-a735-4845-9ecf-dd7d4a6941af', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6817', 'nonce': 301360893}, {'type': 'v1', 'addr': '172.21.15.42:6819', 'nonce': 301360893}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6821', 'nonce': 301360893}, {'type': 'v1', 'addr': '172.21.15.42:6823', 'nonce': 301360893}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6829', 'nonce': 301360893}, {'type': 'v1', 'addr': '172.21.15.42:6831', 'nonce': 301360893}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6825', 'nonce': 301360893}, {'type': 'v1', 'addr': '172.21.15.42:6827', 'nonce': 301360893}]}, 'public_addr': '172.21.15.42:6819/301360893', 'cluster_addr': '172.21.15.42:6823/301360893', 'heartbeat_back_addr': '172.21.15.42:6831/301360893', 'heartbeat_front_addr': '172.21.15.42:6827/301360893', 'state': ['exists', 'up']}, {'osd': 14, 'uuid': '6f66ae97-eb8a-48eb-9bcc-20f65fbf690b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 26, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6824', 'nonce': 1534905602}, {'type': 'v1', 'addr': '172.21.15.66:6825', 'nonce': 1534905602}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6826', 'nonce': 1534905602}, {'type': 'v1', 'addr': '172.21.15.66:6827', 'nonce': 1534905602}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6830', 'nonce': 1534905602}, {'type': 'v1', 'addr': '172.21.15.66:6831', 'nonce': 1534905602}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6828', 'nonce': 1534905602}, {'type': 'v1', 'addr': '172.21.15.66:6829', 'nonce': 1534905602}]}, 'public_addr': '172.21.15.66:6825/1534905602', 'cluster_addr': '172.21.15.66:6827/1534905602', 'heartbeat_back_addr': '172.21.15.66:6831/1534905602', 'heartbeat_front_addr': '172.21.15.66:6829/1534905602', 'state': ['exists', 'up']}, {'osd': 15, 'uuid': 'b856bfc8-253b-4112-be5c-f67d61d01768', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 26, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6826', 'nonce': 3982634323}, {'type': 'v1', 'addr': '172.21.15.92:6827', 'nonce': 3982634323}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6828', 'nonce': 3982634323}, {'type': 'v1', 'addr': '172.21.15.92:6829', 'nonce': 3982634323}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6832', 'nonce': 3982634323}, {'type': 'v1', 'addr': '172.21.15.92:6833', 'nonce': 3982634323}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6830', 'nonce': 3982634323}, {'type': 'v1', 'addr': '172.21.15.92:6831', 'nonce': 3982634323}]}, 'public_addr': '172.21.15.92:6827/3982634323', 'cluster_addr': '172.21.15.92:6829/3982634323', 'heartbeat_back_addr': '172.21.15.92:6833/3982634323', 'heartbeat_front_addr': '172.21.15.92:6831/3982634323', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.265053+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.243502+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.252440+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.295331+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.213076+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.281193+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.315801+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.329538+0000', 'dead_epoch': 0}, {'osd': 8, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.293303+0000', 'dead_epoch': 0}, {'osd': 9, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.334962+0000', 'dead_epoch': 0}, {'osd': 10, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.233032+0000', 'dead_epoch': 0}, {'osd': 11, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.305138+0000', 'dead_epoch': 0}, {'osd': 12, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.351964+0000', 'dead_epoch': 0}, {'osd': 13, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.241086+0000', 'dead_epoch': 0}, {'osd': 14, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.352515+0000', 'dead_epoch': 0}, {'osd': 15, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.357462+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-04-26T03:38:46.483 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T03:38:46.483 INFO:teuthology.orchestra.run.smithi028.stdout:osd.6: {} 2024-04-26T03:38:46.483 INFO:teuthology.orchestra.run.smithi028.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T03:38:46.484 INFO:teuthology.orchestra.run.smithi028.stdout:osd.7: {} 2024-04-26T03:38:46.484 INFO:teuthology.orchestra.run.smithi028.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T03:38:46.484 INFO:teuthology.orchestra.run.smithi028.stdout:osd.8: {} 2024-04-26T03:38:46.485 INFO:teuthology.orchestra.run.smithi028.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T03:38:46.485 INFO:teuthology.orchestra.run.smithi028.stderr:load: shec 2024-04-26T03:38:46.485 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:46.485 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:46.486 INFO:tasks.rados:starting run 0 out of 1 2024-04-26T03:38:46.486 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_0 2024-04-26T03:38:46.486 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_0 16 16 erasure shecprofile 2024-04-26T03:38:46.487 INFO:teuthology.orchestra.run.smithi028.stdout:osd.9: {} 2024-04-26T03:38:46.487 INFO:teuthology.orchestra.run.smithi028.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T03:38:46.487 INFO:tasks.ceph.osd.12.smithi028.stderr:2024-04-26T03:38:46.478+0000 7f114fe14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 31967) UID: 0 2024-04-26T03:38:46.488 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-26T03:38:46.509 INFO:teuthology.orchestra.run.smithi028.stdout:osd.10: {} 2024-04-26T03:38:46.509 INFO:teuthology.orchestra.run.smithi028.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T03:38:46.528 INFO:teuthology.orchestra.run.smithi028.stdout:osd.11: {} 2024-04-26T03:38:46.528 INFO:teuthology.orchestra.run.smithi028.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T03:38:46.548 INFO:teuthology.orchestra.run.smithi028.stdout:osd.12: {} 2024-04-26T03:38:46.548 INFO:teuthology.orchestra.run.smithi028.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T03:38:46.566 INFO:teuthology.orchestra.run.smithi028.stdout:osd.13: {} 2024-04-26T03:38:46.567 INFO:teuthology.orchestra.run.smithi028.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T03:38:46.581 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-26T03:38:46.580+0000 7f610b3a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31816) UID: 0 2024-04-26T03:38:46.585 INFO:teuthology.orchestra.run.smithi028.stdout:osd.14: {} 2024-04-26T03:38:46.585 INFO:teuthology.orchestra.run.smithi028.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T03:38:46.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:38:46.584+0000 7f605ec11640 0 -- 172.21.15.28:0/1249727547 >> [v2:172.21.15.92:6826/3982634323,v1:172.21.15.92:6827/3982634323] conn(0x7f6058222e40 msgr2=0x7f6058225230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-26T03:38:46.627 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:46.627 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:46.637 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-26T03:38:46.682 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:38:46.680+0000 7f60d3c4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31757) UID: 0 2024-04-26T03:38:46.782 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:38:46.781+0000 7fb78c544640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31956) UID: 0 2024-04-26T03:38:46.785 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:46.785 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:46.794 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-26T03:38:46.805 INFO:teuthology.orchestra.run.smithi028.stdout:osd.15: {} 2024-04-26T03:38:46.806 INFO:teuthology.orchestra.run.smithi028.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T03:38:46.818 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:38:46.822 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-04-26T03:38:46.830 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-04-26T03:38:46.458297+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":12,"kb":1509949440,"kb_used":453492,"kb_used_data":4456,"kb_used_omap":24,"kb_used_meta":448743,"kb_avail":1509495948,"statfs":{"total":1546188226560,"available":1545723850752,"internally_reserved":0,"allocated":4562944,"data_stored":3085040,"data_compressed":193724,"data_compressed_allocated":458752,"data_compressed_original":1810432,"omap_allocated":25522,"internal_metadata":459512910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.409166"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T03:38:38.835823+0000","last_change":"2024-04-26T03:38:37.359252+0000","last_active":"2024-04-26T03:38:38.835823+0000","last_peered":"2024-04-26T03:38:38.835823+0000","last_clean":"2024-04-26T03:38:38.835823+0000","last_became_active":"2024-04-26T03:38:37.358910+0000","last_became_peered":"2024-04-26T03:38:37.358910+0000","last_unstale":"2024-04-26T03:38:38.835823+0000","last_undegraded":"2024-04-26T03:38:38.835823+0000","last_fullsized":"2024-04-26T03:38:38.835823+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:04.186001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T03:38:38.836268+0000","last_change":"2024-04-26T03:38:36.484782+0000","last_active":"2024-04-26T03:38:38.836268+0000","last_peered":"2024-04-26T03:38:38.836268+0000","last_clean":"2024-04-26T03:38:38.836268+0000","last_became_active":"2024-04-26T03:38:36.484388+0000","last_became_peered":"2024-04-26T03:38:36.484388+0000","last_unstale":"2024-04-26T03:38:38.836268+0000","last_undegraded":"2024-04-26T03:38:38.836268+0000","last_fullsized":"2024-04-26T03:38:38.836268+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:36.318287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,13],"acting":[8,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T03:38:38.835800+0000","last_change":"2024-04-26T03:38:38.187714+0000","last_active":"2024-04-26T03:38:38.835800+0000","last_peered":"2024-04-26T03:38:38.835800+0000","last_clean":"2024-04-26T03:38:38.835800+0000","last_became_active":"2024-04-26T03:38:38.187383+0000","last_became_peered":"2024-04-26T03:38:38.187383+0000","last_unstale":"2024-04-26T03:38:38.835800+0000","last_undegraded":"2024-04-26T03:38:38.835800+0000","last_fullsized":"2024-04-26T03:38:38.835800+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:59.062623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4],"acting":[9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T03:38:38.443172+0000","last_change":"2024-04-26T03:38:37.255389+0000","last_active":"2024-04-26T03:38:38.443172+0000","last_peered":"2024-04-26T03:38:38.443172+0000","last_clean":"2024-04-26T03:38:38.443172+0000","last_became_active":"2024-04-26T03:38:37.255066+0000","last_became_peered":"2024-04-26T03:38:37.255066+0000","last_unstale":"2024-04-26T03:38:38.443172+0000","last_undegraded":"2024-04-26T03:38:38.443172+0000","last_fullsized":"2024-04-26T03:38:38.443172+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:43.860480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T03:38:38.834966+0000","last_change":"2024-04-26T03:38:36.447388+0000","last_active":"2024-04-26T03:38:38.834966+0000","last_peered":"2024-04-26T03:38:38.834966+0000","last_clean":"2024-04-26T03:38:38.834966+0000","last_became_active":"2024-04-26T03:38:36.447194+0000","last_became_peered":"2024-04-26T03:38:36.447194+0000","last_unstale":"2024-04-26T03:38:38.834966+0000","last_undegraded":"2024-04-26T03:38:38.834966+0000","last_fullsized":"2024-04-26T03:38:38.834966+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:42.465065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T03:38:39.263216+0000","last_change":"2024-04-26T03:38:37.255450+0000","last_active":"2024-04-26T03:38:39.263216+0000","last_peered":"2024-04-26T03:38:39.263216+0000","last_clean":"2024-04-26T03:38:39.263216+0000","last_became_active":"2024-04-26T03:38:37.255181+0000","last_became_peered":"2024-04-26T03:38:37.255181+0000","last_unstale":"2024-04-26T03:38:39.263216+0000","last_undegraded":"2024-04-26T03:38:39.263216+0000","last_fullsized":"2024-04-26T03:38:39.263216+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:00.449388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0],"acting":[14,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T03:38:38.834924+0000","last_change":"2024-04-26T03:38:36.431325+0000","last_active":"2024-04-26T03:38:38.834924+0000","last_peered":"2024-04-26T03:38:38.834924+0000","last_clean":"2024-04-26T03:38:38.834924+0000","last_became_active":"2024-04-26T03:38:36.431121+0000","last_became_peered":"2024-04-26T03:38:36.431121+0000","last_unstale":"2024-04-26T03:38:38.834924+0000","last_undegraded":"2024-04-26T03:38:38.834924+0000","last_fullsized":"2024-04-26T03:38:38.834924+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:53.294619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T03:38:39.263219+0000","last_change":"2024-04-26T03:38:36.431667+0000","last_active":"2024-04-26T03:38:39.263219+0000","last_peered":"2024-04-26T03:38:39.263219+0000","last_clean":"2024-04-26T03:38:39.263219+0000","last_became_active":"2024-04-26T03:38:36.431449+0000","last_became_peered":"2024-04-26T03:38:36.431449+0000","last_unstale":"2024-04-26T03:38:39.263219+0000","last_undegraded":"2024-04-26T03:38:39.263219+0000","last_fullsized":"2024-04-26T03:38:39.263219+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:47.562648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"27'157","reported_seq":215,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T03:38:39.498056+0000","last_change":"2024-04-26T03:38:34.390206+0000","last_active":"2024-04-26T03:38:39.498056+0000","last_peered":"2024-04-26T03:38:39.498056+0000","last_clean":"2024-04-26T03:38:39.498056+0000","last_became_active":"2024-04-26T03:38:34.389099+0000","last_became_peered":"2024-04-26T03:38:34.389099+0000","last_unstale":"2024-04-26T03:38:39.498056+0000","last_undegraded":"2024-04-26T03:38:39.498056+0000","last_fullsized":"2024-04-26T03:38:39.498056+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:32.306810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:32.306810+0000","last_clean_scrub_stamp":"2024-04-26T03:38:32.306810+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:40.216975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":11},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":13852,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":24,"seq":103079215111,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31504,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340336,"statfs":{"total":96636764160,"available":96604504064,"internally_reserved":0,"allocated":331776,"data_stored":655627,"data_compressed":18168,"data_compressed_allocated":57344,"data_compressed_original":561152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[4,6,7,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215111,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,4,5,7,8,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":23,"seq":98784247816,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,7,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280517,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":331776,"data_stored":655627,"data_compressed":18168,"data_compressed_allocated":57344,"data_compressed_original":561152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,4,5,7,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[4,6,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":22,"seq":94489280518,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[2,5,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,2,4,7,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280517,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[3,6,7,8,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313223,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,8,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":12,"up_from":22,"seq":94489280516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,7,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[4,6,7,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247814,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,4,6,7,8,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280516,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,3,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":23,"seq":98784247816,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31460,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,4,6,7,8,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280516,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,4,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247814,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T03:38:46.831 INFO:tasks.thrashosds.thrasher:Setting 2.2 to [3, 11, 8, 10] 2024-04-26T03:38:46.831 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '2.2', '3', '11', '8', '10'] 2024-04-26T03:38:46.831 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 2.2 3 11 8 10 2024-04-26T03:38:46.883 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-04-26T03:38:46.882+0000 7f745118d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 31821) UID: 0 2024-04-26T03:38:46.904 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:46.904 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:46.913 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-04-26T03:38:46.997 INFO:tasks.ceph.osd.12.smithi028.stderr:2024-04-26T03:38:46.995+0000 7f114fe14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 31967) UID: 0 2024-04-26T03:38:47.078 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:47.078 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:47.085 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-04-26T03:38:47.084+0000 7f745118d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 31821) UID: 0 2024-04-26T03:38:47.087 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-04-26T03:38:47.186 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-04-26T03:38:47.184+0000 7fe9223c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31954) UID: 0 2024-04-26T03:38:47.235 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:47.235 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:47.244 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-04-26T03:38:47.287 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:38:47.285+0000 7f60d3c4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31757) UID: 0 2024-04-26T03:38:47.351 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:47.351 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:47.360 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-04-26T03:38:47.388 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-04-26T03:38:47.387+0000 7fee562fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31759) UID: 0 2024-04-26T03:38:47.463 INFO:teuthology.orchestra.run.smithi028.stderr:noscrub is set 2024-04-26T03:38:47.464 INFO:teuthology.orchestra.run.smithi028.stderr:set 2.2 pg_upmap_items mapping to [3->11,8->10] 2024-04-26T03:38:47.470 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:47.470 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:47.480 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-04-26T03:38:47.489 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:38:47.487+0000 7fb78c544640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31956) UID: 0 2024-04-26T03:38:47.590 INFO:tasks.ceph.osd.8.smithi028.stderr:2024-04-26T03:38:47.588+0000 7f5bb75d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 31968) UID: 0 2024-04-26T03:38:47.628 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:47.628 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:47.639 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-04-26T03:38:47.692 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-04-26T03:38:47.690+0000 7fee562fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31759) UID: 0 2024-04-26T03:38:47.785 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:47.786 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:47.793 INFO:tasks.ceph.osd.15.smithi092.stderr:2024-04-26T03:38:47.792+0000 7f6d12dec640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 31824) UID: 0 2024-04-26T03:38:47.795 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-26T03:38:47.894 INFO:tasks.ceph.osd.12.smithi028.stderr:2024-04-26T03:38:47.892+0000 7f114fe14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 31967) UID: 0 2024-04-26T03:38:47.916 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:47.916 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:47.924 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-26T03:38:47.995 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-26T03:38:47.994+0000 7f610b3a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31816) UID: 0 2024-04-26T03:38:48.031 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:48.032 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:48.040 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-26T03:38:48.096 INFO:tasks.ceph.osd.15.smithi092.stderr:2024-04-26T03:38:48.095+0000 7f6d12dec640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 31824) UID: 0 2024-04-26T03:38:48.147 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:48.147 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:48.156 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-26T03:38:48.197 INFO:tasks.ceph.osd.8.smithi028.stderr:2024-04-26T03:38:48.195+0000 7f5bb75d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 31968) UID: 0 2024-04-26T03:38:48.263 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:48.263 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:48.272 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-26T03:38:48.298 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-04-26T03:38:48.297+0000 7fdc2d21c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31819) UID: 0 2024-04-26T03:38:48.378 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:48.379 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:48.387 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-26T03:38:48.432 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:38:48.430+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:38:48.467 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'unique_pool_0' created 2024-04-26T03:38:48.489 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_0 rados --yes-i-really-mean-it || true 2024-04-26T03:38:48.501 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:48.501 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:48.510 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-04-26T03:38:48.530 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:38:48.528+0000 7fba0d2ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31961) UID: 0 2024-04-26T03:38:48.601 INFO:tasks.ceph.osd.12.smithi028.stderr:2024-04-26T03:38:48.599+0000 7f114fe14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 31967) UID: 0 2024-04-26T03:38:48.657 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:48.657 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:48.667 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-04-26T03:38:48.711 INFO:tasks.ceph.osd.9.smithi042.stderr:2024-04-26T03:38:48.709+0000 7fdac63b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 31960) UID: 0 2024-04-26T03:38:48.773 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:48.773 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:48.782 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-04-26T03:38:48.824 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:38:48.822+0000 7fb78c544640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31956) UID: 0 2024-04-26T03:38:48.824 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-26T03:38:48.904 INFO:tasks.ceph.osd.12.smithi028.stderr:2024-04-26T03:38:48.902+0000 7f114fe14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 31967) UID: 0 2024-04-26T03:38:48.929 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:48.930 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:48.939 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-04-26T03:38:48.950 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0: {} 2024-04-26T03:38:48.950 INFO:teuthology.orchestra.run.smithi028.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T03:38:48.968 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1: {} 2024-04-26T03:38:48.968 INFO:teuthology.orchestra.run.smithi028.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T03:38:48.985 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2: {} 2024-04-26T03:38:48.986 INFO:teuthology.orchestra.run.smithi028.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T03:38:49.005 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3: {} 2024-04-26T03:38:49.005 INFO:teuthology.orchestra.run.smithi028.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T03:38:49.005 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-26T03:38:49.002+0000 7f610b3a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31816) UID: 0 2024-04-26T03:38:49.046 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:49.046 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:49.055 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-04-26T03:38:49.106 INFO:tasks.ceph.osd.12.smithi028.stderr:2024-04-26T03:38:49.104+0000 7f114fe14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 31967) UID: 0 2024-04-26T03:38:49.161 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:49.162 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:49.170 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-04-26T03:38:49.207 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-04-26T03:38:49.205+0000 7f745118d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 31821) UID: 0 2024-04-26T03:38:49.222 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4: {} 2024-04-26T03:38:49.223 INFO:teuthology.orchestra.run.smithi028.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T03:38:49.240 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5: {} 2024-04-26T03:38:49.240 INFO:teuthology.orchestra.run.smithi028.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T03:38:49.260 INFO:teuthology.orchestra.run.smithi028.stdout:osd.6: {} 2024-04-26T03:38:49.260 INFO:teuthology.orchestra.run.smithi028.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T03:38:49.278 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:49.278 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:49.279 INFO:teuthology.orchestra.run.smithi028.stdout:osd.7: {} 2024-04-26T03:38:49.279 INFO:teuthology.orchestra.run.smithi028.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T03:38:49.286 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-26T03:38:49.295 INFO:teuthology.orchestra.run.smithi028.stdout:osd.8: {} 2024-04-26T03:38:49.295 INFO:teuthology.orchestra.run.smithi028.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T03:38:49.308 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:38:49.306+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:38:49.313 INFO:teuthology.orchestra.run.smithi028.stdout:osd.9: {} 2024-04-26T03:38:49.313 INFO:teuthology.orchestra.run.smithi028.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T03:38:49.331 INFO:teuthology.orchestra.run.smithi028.stdout:osd.10: {} 2024-04-26T03:38:49.331 INFO:teuthology.orchestra.run.smithi028.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T03:38:49.349 INFO:teuthology.orchestra.run.smithi028.stdout:osd.11: {} 2024-04-26T03:38:49.349 INFO:teuthology.orchestra.run.smithi028.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T03:38:49.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:38:49.348+0000 7f2f837fe640 0 -- 172.21.15.28:0/3752926227 >> [v2:172.21.15.28:6824/2081141953,v1:172.21.15.28:6825/2081141953] conn(0x7f2f842156e0 msgr2=0x7f2f84217ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-26T03:38:49.366 INFO:teuthology.orchestra.run.smithi028.stdout:osd.12: {} 2024-04-26T03:38:49.366 INFO:teuthology.orchestra.run.smithi028.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T03:38:49.384 INFO:teuthology.orchestra.run.smithi028.stdout:osd.13: {} 2024-04-26T03:38:49.384 INFO:teuthology.orchestra.run.smithi028.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T03:38:49.401 INFO:teuthology.orchestra.run.smithi028.stdout:osd.14: {} 2024-04-26T03:38:49.401 INFO:teuthology.orchestra.run.smithi028.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T03:38:49.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:38:49.400+0000 7f2f837fe640 0 -- 172.21.15.28:0/3752926227 >> [v2:172.21.15.92:6826/3982634323,v1:172.21.15.92:6827/3982634323] conn(0x7f2f8421c7a0 msgr2=0x7f2f8421eb90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-26T03:38:49.406 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:38:49.406 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-04-26T03:38:49.406 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 0 2024-04-26T03:38:49.406 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:38:49.409 INFO:tasks.ceph.osd.15.smithi092.stderr:2024-04-26T03:38:49.406+0000 7f6d12dec640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 31824) UID: 0 2024-04-26T03:38:49.415 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-26T03:38:49.419 INFO:teuthology.orchestra.run.smithi028.stdout:osd.15: {} 2024-04-26T03:38:49.420 INFO:teuthology.orchestra.run.smithi028.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T03:38:49.471 INFO:teuthology.orchestra.run.smithi028.stderr:enabled application 'rados' on pool 'unique_pool_0' 2024-04-26T03:38:49.483 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-26T03:38:49.533 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:38:49.533 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-04-26T03:38:49.533 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-04-26T03:38:49.533 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-04-26T03:38:49.533 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:38:49.534 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:38:49.532+0000 7fba0d2ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31961) UID: 0 2024-04-26T03:38:49.542 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-26T03:38:49.610 INFO:tasks.ceph.osd.8.smithi028.stderr:2024-04-26T03:38:49.608+0000 7f5bb75d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 31968) UID: 0 2024-04-26T03:38:49.649 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:38:49.649 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-04-26T03:38:49.649 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-04-26T03:38:49.649 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [] 2024-04-26T03:38:49.649 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:38:49.658 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-26T03:38:49.711 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-04-26T03:38:49.709+0000 7f745118d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 31821) UID: 0 2024-04-26T03:38:49.764 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:38:49.764 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-04-26T03:38:49.764 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 0 2024-04-26T03:38:49.764 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:38:49.773 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-26T03:38:49.813 INFO:tasks.ceph.osd.8.smithi028.stderr:2024-04-26T03:38:49.812+0000 7f5bb75d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 31968) UID: 0 2024-04-26T03:38:49.879 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:38:49.879 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-04-26T03:38:49.879 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-04-26T03:38:49.879 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-04-26T03:38:49.879 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:38:49.892 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-26T03:38:49.914 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:38:49.913+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:38:50.002 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:38:50.002 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-04-26T03:38:50.002 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-04-26T03:38:50.002 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-04-26T03:38:50.002 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.002 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4213.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-04-26T03:38:50.002 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:33.315386+0000", 2024-04-26T03:38:50.003 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 16.683914864999998, 2024-04-26T03:38:50.003 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.075394833, 2024-04-26T03:38:50.003 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:38:50.003 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:38:50.003 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:38:50.003 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:38:50.003 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4213", 2024-04-26T03:38:50.003 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/331201604", 2024-04-26T03:38:50.003 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 1 2024-04-26T03:38:50.003 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.003 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:38:50.003 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.003 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:38:50.003 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:33.315386+0000", 2024-04-26T03:38:50.004 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:50.004 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.004 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.004 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:38:50.004 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:33.315386+0000", 2024-04-26T03:38:50.004 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:50.004 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.004 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.004 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:38:50.004 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:33.315388+0000", 2024-04-26T03:38:50.004 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.897e-06 2024-04-26T03:38:50.004 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.004 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.004 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:38:50.005 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:33.315417+0000", 2024-04-26T03:38:50.005 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8983e-05 2024-04-26T03:38:50.005 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.005 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.005 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:38:50.005 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:33.315447+0000", 2024-04-26T03:38:50.005 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.9920000000000002e-05 2024-04-26T03:38:50.005 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.005 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.005 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:38:50.005 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:33.315460+0000", 2024-04-26T03:38:50.005 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3064999999999999e-05 2024-04-26T03:38:50.005 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.005 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.006 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:38:50.006 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:33.315770+0000", 2024-04-26T03:38:50.006 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000309687 2024-04-26T03:38:50.006 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.006 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.006 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for peered", 2024-04-26T03:38:50.006 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:33.315784+0000", 2024-04-26T03:38:50.006 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3767e-05 2024-04-26T03:38:50.006 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.006 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.006 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:38:50.006 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.389505+0000", 2024-04-26T03:38:50.006 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0737211259999999 2024-04-26T03:38:50.007 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.007 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.007 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:38:50.007 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.389667+0000", 2024-04-26T03:38:50.007 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000162435 2024-04-26T03:38:50.007 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.007 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.007 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 5", 2024-04-26T03:38:50.007 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.389837+0000", 2024-04-26T03:38:50.007 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00016988999999999999 2024-04-26T03:38:50.007 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.007 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.008 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-04-26T03:38:50.008 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.390620+0000", 2024-04-26T03:38:50.008 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000782941 2024-04-26T03:38:50.008 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.008 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.008 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-04-26T03:38:50.008 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.390712+0000", 2024-04-26T03:38:50.008 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.2119000000000004e-05 2024-04-26T03:38:50.008 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.008 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.008 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:38:50.008 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.390751+0000", 2024-04-26T03:38:50.008 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9168999999999997e-05 2024-04-26T03:38:50.008 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.008 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.009 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:38:50.009 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.390781+0000", 2024-04-26T03:38:50.009 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.9833999999999999e-05 2024-04-26T03:38:50.009 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:50.009 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:38:50.009 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:50.009 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.009 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.009 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4213.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-04-26T03:38:50.009 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:34.628711+0000", 2024-04-26T03:38:50.009 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 15.370589800999999, 2024-04-26T03:38:50.009 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0077224709999999998, 2024-04-26T03:38:50.009 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:38:50.009 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:38:50.010 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:38:50.010 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:38:50.010 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4213", 2024-04-26T03:38:50.010 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/331201604", 2024-04-26T03:38:50.010 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 116 2024-04-26T03:38:50.010 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.010 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:38:50.010 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.010 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:38:50.010 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.628711+0000", 2024-04-26T03:38:50.010 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:50.010 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.010 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.010 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:38:50.011 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.628711+0000", 2024-04-26T03:38:50.011 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:50.011 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.011 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.011 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:38:50.011 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.628712+0000", 2024-04-26T03:38:50.011 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.5600000000000001e-07 2024-04-26T03:38:50.011 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.011 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.011 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:38:50.011 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.628714+0000", 2024-04-26T03:38:50.011 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2010000000000002e-06 2024-04-26T03:38:50.011 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.011 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.012 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:38:50.012 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.628717+0000", 2024-04-26T03:38:50.012 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.4570000000000001e-06 2024-04-26T03:38:50.012 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.012 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.012 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:38:50.012 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.628726+0000", 2024-04-26T03:38:50.012 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.9730000000000003e-06 2024-04-26T03:38:50.012 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.012 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.012 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:38:50.012 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.635362+0000", 2024-04-26T03:38:50.012 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0066352520000000003 2024-04-26T03:38:50.012 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.013 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.013 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:38:50.013 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.635415+0000", 2024-04-26T03:38:50.013 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.3186000000000002e-05 2024-04-26T03:38:50.013 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.013 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.013 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 5", 2024-04-26T03:38:50.013 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.635492+0000", 2024-04-26T03:38:50.013 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.6916999999999996e-05 2024-04-26T03:38:50.013 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.013 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.013 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-04-26T03:38:50.013 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.636171+0000", 2024-04-26T03:38:50.013 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00067913999999999995 2024-04-26T03:38:50.013 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.014 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.014 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-04-26T03:38:50.014 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.636370+0000", 2024-04-26T03:38:50.014 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000199178 2024-04-26T03:38:50.014 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.014 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.014 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:38:50.014 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.636400+0000", 2024-04-26T03:38:50.014 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.0209000000000002e-05 2024-04-26T03:38:50.014 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.014 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.014 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:38:50.014 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.636434+0000", 2024-04-26T03:38:50.015 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.3602000000000001e-05 2024-04-26T03:38:50.015 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:50.015 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:38:50.015 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:50.015 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.015 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.015 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4213.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-04-26T03:38:50.015 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:34.628731+0000", 2024-04-26T03:38:50.015 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 15.370569765000001, 2024-04-26T03:38:50.015 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0078760110000000005, 2024-04-26T03:38:50.015 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:38:50.015 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:38:50.015 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:38:50.016 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:38:50.016 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4213", 2024-04-26T03:38:50.016 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/331201604", 2024-04-26T03:38:50.016 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 117 2024-04-26T03:38:50.016 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.016 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:38:50.016 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.016 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:38:50.016 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.628731+0000", 2024-04-26T03:38:50.016 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:50.016 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.016 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.016 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:38:50.016 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.628731+0000", 2024-04-26T03:38:50.017 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:50.017 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.017 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.017 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:38:50.017 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.628732+0000", 2024-04-26T03:38:50.017 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7000000000000001e-07 2024-04-26T03:38:50.017 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.017 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.017 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:38:50.017 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.628734+0000", 2024-04-26T03:38:50.017 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.351e-06 2024-04-26T03:38:50.017 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.018 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.018 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:38:50.018 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.628739+0000", 2024-04-26T03:38:50.018 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6079999999999998e-06 2024-04-26T03:38:50.018 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.018 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.018 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:38:50.018 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.628741+0000", 2024-04-26T03:38:50.018 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8080000000000001e-06 2024-04-26T03:38:50.018 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.018 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.018 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:38:50.018 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.635731+0000", 2024-04-26T03:38:50.019 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0069894329999999998 2024-04-26T03:38:50.019 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.019 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.019 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:38:50.019 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.635814+0000", 2024-04-26T03:38:50.019 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.3084999999999995e-05 2024-04-26T03:38:50.019 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.019 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.019 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 5", 2024-04-26T03:38:50.019 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.635942+0000", 2024-04-26T03:38:50.019 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012807399999999999 2024-04-26T03:38:50.019 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.019 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.019 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-04-26T03:38:50.020 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.636458+0000", 2024-04-26T03:38:50.020 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00051629099999999999 2024-04-26T03:38:50.020 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.020 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.020 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-04-26T03:38:50.020 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.636560+0000", 2024-04-26T03:38:50.020 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000102061 2024-04-26T03:38:50.020 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.020 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.020 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:38:50.020 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.636587+0000", 2024-04-26T03:38:50.020 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6869999999999999e-05 2024-04-26T03:38:50.020 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.020 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.021 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:38:50.021 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.636607+0000", 2024-04-26T03:38:50.021 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.016e-05 2024-04-26T03:38:50.021 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:50.021 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:38:50.021 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:50.021 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.021 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.021 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4213.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-04-26T03:38:50.021 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:34.771545+0000", 2024-04-26T03:38:50.021 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 15.227755664, 2024-04-26T03:38:50.021 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0070811010000000002, 2024-04-26T03:38:50.021 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:38:50.021 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:38:50.021 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:38:50.022 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:38:50.022 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4213", 2024-04-26T03:38:50.022 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/331201604", 2024-04-26T03:38:50.022 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 140 2024-04-26T03:38:50.022 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.022 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:38:50.022 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.022 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:38:50.022 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.771545+0000", 2024-04-26T03:38:50.022 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:50.022 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.022 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.022 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:38:50.023 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.771545+0000", 2024-04-26T03:38:50.023 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:50.023 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.023 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.023 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:38:50.023 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.771546+0000", 2024-04-26T03:38:50.023 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.2599999999999998e-07 2024-04-26T03:38:50.023 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.023 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.023 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:38:50.023 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.771636+0000", 2024-04-26T03:38:50.023 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.0109000000000001e-05 2024-04-26T03:38:50.023 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.023 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.024 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:38:50.024 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.771645+0000", 2024-04-26T03:38:50.024 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.8959999999999992e-06 2024-04-26T03:38:50.024 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.024 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.024 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:38:50.024 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.771653+0000", 2024-04-26T03:38:50.024 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8900000000000007e-06 2024-04-26T03:38:50.024 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.024 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.025 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:38:50.025 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.776736+0000", 2024-04-26T03:38:50.025 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0050836270000000003 2024-04-26T03:38:50.025 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.025 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.025 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:38:50.025 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.776781+0000", 2024-04-26T03:38:50.025 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.4851000000000001e-05 2024-04-26T03:38:50.025 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.025 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.025 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 5", 2024-04-26T03:38:50.026 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.776850+0000", 2024-04-26T03:38:50.026 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.8385000000000003e-05 2024-04-26T03:38:50.026 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.026 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.026 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-04-26T03:38:50.026 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.778264+0000", 2024-04-26T03:38:50.026 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0014146429999999999 2024-04-26T03:38:50.026 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.026 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.026 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-04-26T03:38:50.027 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.778577+0000", 2024-04-26T03:38:50.027 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00031283599999999998 2024-04-26T03:38:50.027 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.027 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.027 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:38:50.027 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.778605+0000", 2024-04-26T03:38:50.027 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7614e-05 2024-04-26T03:38:50.027 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.027 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.027 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:38:50.027 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.778627+0000", 2024-04-26T03:38:50.027 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1824e-05 2024-04-26T03:38:50.027 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:50.027 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:38:50.027 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:50.028 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.028 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.028 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4213.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-04-26T03:38:50.028 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:34.771665+0000", 2024-04-26T03:38:50.028 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 15.227636308999999, 2024-04-26T03:38:50.028 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0072578510000000001, 2024-04-26T03:38:50.028 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:38:50.028 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:38:50.028 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:38:50.028 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:38:50.028 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4213", 2024-04-26T03:38:50.028 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/331201604", 2024-04-26T03:38:50.028 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 141 2024-04-26T03:38:50.028 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.028 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:38:50.029 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.029 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:38:50.029 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.771665+0000", 2024-04-26T03:38:50.029 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:50.029 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.029 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.029 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:38:50.029 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.771665+0000", 2024-04-26T03:38:50.029 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:50.029 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.029 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.029 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:38:50.029 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.771665+0000", 2024-04-26T03:38:50.029 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.06e-07 2024-04-26T03:38:50.030 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.030 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.030 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:38:50.030 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.771669+0000", 2024-04-26T03:38:50.030 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8850000000000001e-06 2024-04-26T03:38:50.030 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.030 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.030 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:38:50.030 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.771676+0000", 2024-04-26T03:38:50.030 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7379999999999997e-06 2024-04-26T03:38:50.030 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.030 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.030 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:38:50.030 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.771681+0000", 2024-04-26T03:38:50.031 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.366e-06 2024-04-26T03:38:50.031 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.031 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.031 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:38:50.031 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.777610+0000", 2024-04-26T03:38:50.031 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0059283000000000001 2024-04-26T03:38:50.031 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.031 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.031 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:38:50.031 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.777656+0000", 2024-04-26T03:38:50.031 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.5918999999999999e-05 2024-04-26T03:38:50.031 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.031 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.031 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 5", 2024-04-26T03:38:50.032 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.777733+0000", 2024-04-26T03:38:50.032 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.7438e-05 2024-04-26T03:38:50.032 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.032 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.032 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-04-26T03:38:50.032 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.778508+0000", 2024-04-26T03:38:50.032 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00077498400000000005 2024-04-26T03:38:50.032 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.032 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.032 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-04-26T03:38:50.032 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.778882+0000", 2024-04-26T03:38:50.032 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00037384199999999999 2024-04-26T03:38:50.032 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.032 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.033 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:38:50.033 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.778906+0000", 2024-04-26T03:38:50.033 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4511e-05 2024-04-26T03:38:50.033 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.033 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.033 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:38:50.033 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.778923+0000", 2024-04-26T03:38:50.033 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6262000000000001e-05 2024-04-26T03:38:50.033 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:50.033 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:38:50.033 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:50.033 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.033 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.033 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4213.0:161 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-04-26T03:38:50.033 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:34.911666+0000", 2024-04-26T03:38:50.034 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 15.087635069999999, 2024-04-26T03:38:50.034 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.006911726, 2024-04-26T03:38:50.034 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:38:50.034 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:38:50.034 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:38:50.034 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:38:50.034 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4213", 2024-04-26T03:38:50.034 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/331201604", 2024-04-26T03:38:50.034 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 161 2024-04-26T03:38:50.034 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.034 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:38:50.034 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.034 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:38:50.035 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.911666+0000", 2024-04-26T03:38:50.035 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:50.035 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.035 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.035 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:38:50.035 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.911666+0000", 2024-04-26T03:38:50.035 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:50.035 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.035 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.035 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:38:50.035 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.911666+0000", 2024-04-26T03:38:50.035 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7800000000000002e-07 2024-04-26T03:38:50.035 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.035 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.036 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:38:50.036 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.911714+0000", 2024-04-26T03:38:50.036 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.7568000000000003e-05 2024-04-26T03:38:50.036 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.036 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.036 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:38:50.036 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.911723+0000", 2024-04-26T03:38:50.036 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.6410000000000008e-06 2024-04-26T03:38:50.036 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.036 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.036 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:38:50.036 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.911727+0000", 2024-04-26T03:38:50.036 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.0110000000000002e-06 2024-04-26T03:38:50.036 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.037 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.037 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:38:50.037 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.915719+0000", 2024-04-26T03:38:50.037 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0039928669999999998 2024-04-26T03:38:50.037 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.037 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.037 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:38:50.037 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.915791+0000", 2024-04-26T03:38:50.037 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.1668000000000006e-05 2024-04-26T03:38:50.037 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.037 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.037 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 5", 2024-04-26T03:38:50.037 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.915915+0000", 2024-04-26T03:38:50.037 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000124263 2024-04-26T03:38:50.037 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.038 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.038 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-04-26T03:38:50.038 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.917068+0000", 2024-04-26T03:38:50.038 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0011523010000000001 2024-04-26T03:38:50.038 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.038 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.038 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-04-26T03:38:50.038 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.918487+0000", 2024-04-26T03:38:50.038 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001419153 2024-04-26T03:38:50.038 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.038 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.038 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:38:50.038 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.918536+0000", 2024-04-26T03:38:50.038 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.9154999999999999e-05 2024-04-26T03:38:50.039 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.039 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.039 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:38:50.039 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.918578+0000", 2024-04-26T03:38:50.039 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.1721000000000003e-05 2024-04-26T03:38:50.039 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:50.039 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:38:50.039 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:50.039 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.039 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.039 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4213.0:162 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-04-26T03:38:50.039 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:34.911733+0000", 2024-04-26T03:38:50.039 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 15.087567811, 2024-04-26T03:38:50.039 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0070027350000000004, 2024-04-26T03:38:50.040 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:38:50.040 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:38:50.040 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:38:50.040 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:38:50.040 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4213", 2024-04-26T03:38:50.040 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/331201604", 2024-04-26T03:38:50.040 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 162 2024-04-26T03:38:50.040 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.040 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:38:50.040 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.040 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:38:50.040 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.911733+0000", 2024-04-26T03:38:50.040 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:50.040 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.040 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.041 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:38:50.041 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.911733+0000", 2024-04-26T03:38:50.041 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:50.041 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.041 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.041 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:38:50.041 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.911734+0000", 2024-04-26T03:38:50.041 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.1699999999999999e-07 2024-04-26T03:38:50.041 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.041 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.041 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:38:50.041 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.911736+0000", 2024-04-26T03:38:50.041 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.356e-06 2024-04-26T03:38:50.041 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.042 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.042 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:38:50.042 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.911739+0000", 2024-04-26T03:38:50.042 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.5099999999999999e-06 2024-04-26T03:38:50.042 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.042 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.042 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:38:50.042 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.911742+0000", 2024-04-26T03:38:50.042 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4990000000000001e-06 2024-04-26T03:38:50.042 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.042 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.042 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:38:50.042 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.916654+0000", 2024-04-26T03:38:50.042 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0049121939999999999 2024-04-26T03:38:50.043 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.043 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.043 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:38:50.043 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.916709+0000", 2024-04-26T03:38:50.043 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.4740000000000001e-05 2024-04-26T03:38:50.043 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.043 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.043 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 5", 2024-04-26T03:38:50.043 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.916783+0000", 2024-04-26T03:38:50.043 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3799000000000003e-05 2024-04-26T03:38:50.043 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.043 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.043 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-04-26T03:38:50.043 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.918589+0000", 2024-04-26T03:38:50.044 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0018063020000000001 2024-04-26T03:38:50.044 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.044 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.044 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-04-26T03:38:50.044 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.918643+0000", 2024-04-26T03:38:50.044 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.3919999999999999e-05 2024-04-26T03:38:50.044 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.044 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.044 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:38:50.044 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.918699+0000", 2024-04-26T03:38:50.044 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.6595999999999998e-05 2024-04-26T03:38:50.044 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.044 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.044 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:38:50.045 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.918736+0000", 2024-04-26T03:38:50.045 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.6501999999999997e-05 2024-04-26T03:38:50.045 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:50.045 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:38:50.045 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:50.045 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.045 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.045 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4213.0:163 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-04-26T03:38:50.045 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:34.911746+0000", 2024-04-26T03:38:50.045 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 15.087555289999999, 2024-04-26T03:38:50.045 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0075587980000000003, 2024-04-26T03:38:50.045 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:38:50.045 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:38:50.045 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:38:50.046 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:38:50.046 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4213", 2024-04-26T03:38:50.046 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/331201604", 2024-04-26T03:38:50.046 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 163 2024-04-26T03:38:50.046 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.046 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:38:50.046 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.046 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:38:50.046 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.911746+0000", 2024-04-26T03:38:50.046 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:50.046 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.046 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.046 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:38:50.046 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.911746+0000", 2024-04-26T03:38:50.047 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:50.047 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.047 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.047 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:38:50.047 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.911746+0000", 2024-04-26T03:38:50.047 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0900000000000001e-07 2024-04-26T03:38:50.047 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.047 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.047 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:38:50.047 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.911748+0000", 2024-04-26T03:38:50.047 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9719999999999999e-06 2024-04-26T03:38:50.047 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.047 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.047 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:38:50.048 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.911751+0000", 2024-04-26T03:38:50.048 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.0180000000000002e-06 2024-04-26T03:38:50.048 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.048 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.048 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:38:50.048 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.911753+0000", 2024-04-26T03:38:50.048 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.407e-06 2024-04-26T03:38:50.048 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.048 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.048 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:38:50.048 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.917226+0000", 2024-04-26T03:38:50.048 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0054730070000000002 2024-04-26T03:38:50.048 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.048 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.048 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:38:50.049 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.917272+0000", 2024-04-26T03:38:50.049 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.5952000000000003e-05 2024-04-26T03:38:50.049 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.049 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.049 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 5", 2024-04-26T03:38:50.049 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.917374+0000", 2024-04-26T03:38:50.049 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00010121899999999999 2024-04-26T03:38:50.049 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.049 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.049 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-04-26T03:38:50.049 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.919165+0000", 2024-04-26T03:38:50.049 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0017912959999999999 2024-04-26T03:38:50.049 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.049 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.049 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-04-26T03:38:50.050 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.919213+0000", 2024-04-26T03:38:50.050 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.8483000000000003e-05 2024-04-26T03:38:50.050 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.050 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.050 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:38:50.050 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.919267+0000", 2024-04-26T03:38:50.050 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.3535000000000002e-05 2024-04-26T03:38:50.050 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.050 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.050 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:38:50.050 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.919305+0000", 2024-04-26T03:38:50.050 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7700000000000002e-05 2024-04-26T03:38:50.050 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:50.050 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:38:50.051 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:50.051 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.051 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.051 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4213.0:164 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-04-26T03:38:50.051 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:34.911757+0000", 2024-04-26T03:38:50.051 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 15.087543964, 2024-04-26T03:38:50.051 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0077095430000000001, 2024-04-26T03:38:50.051 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:38:50.051 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:38:50.051 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:38:50.051 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:38:50.051 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4213", 2024-04-26T03:38:50.051 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/331201604", 2024-04-26T03:38:50.051 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 164 2024-04-26T03:38:50.051 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.052 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:38:50.052 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.052 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:38:50.052 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.911757+0000", 2024-04-26T03:38:50.052 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:50.052 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.052 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.052 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:38:50.052 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.911757+0000", 2024-04-26T03:38:50.052 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:50.052 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.052 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.052 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:38:50.052 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.911757+0000", 2024-04-26T03:38:50.052 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1299999999999999e-07 2024-04-26T03:38:50.053 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.053 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.053 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:38:50.053 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.911798+0000", 2024-04-26T03:38:50.053 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.1145e-05 2024-04-26T03:38:50.053 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.053 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.053 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:38:50.053 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.911804+0000", 2024-04-26T03:38:50.053 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.1100000000000002e-06 2024-04-26T03:38:50.053 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.053 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.053 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:38:50.053 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.911806+0000", 2024-04-26T03:38:50.054 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8109999999999999e-06 2024-04-26T03:38:50.054 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.054 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.054 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:38:50.054 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.917618+0000", 2024-04-26T03:38:50.054 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0058119729999999998 2024-04-26T03:38:50.054 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.054 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.054 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:38:50.054 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.917699+0000", 2024-04-26T03:38:50.054 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.0147000000000004e-05 2024-04-26T03:38:50.054 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.054 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.054 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 5", 2024-04-26T03:38:50.055 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.917826+0000", 2024-04-26T03:38:50.055 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000127563 2024-04-26T03:38:50.055 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.055 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.055 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-04-26T03:38:50.055 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.919363+0000", 2024-04-26T03:38:50.055 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0015366990000000001 2024-04-26T03:38:50.055 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.055 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.055 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-04-26T03:38:50.055 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.919376+0000", 2024-04-26T03:38:50.055 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3012e-05 2024-04-26T03:38:50.055 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.055 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.055 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:38:50.056 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.919430+0000", 2024-04-26T03:38:50.056 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.3848999999999999e-05 2024-04-26T03:38:50.056 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.056 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.056 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:38:50.056 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.919467+0000", 2024-04-26T03:38:50.056 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7020999999999997e-05 2024-04-26T03:38:50.056 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:50.056 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:38:50.056 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:50.056 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.056 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.056 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4213.0:165 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-04-26T03:38:50.056 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:34.911813+0000", 2024-04-26T03:38:50.057 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 15.087488262999999, 2024-04-26T03:38:50.057 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.007940875, 2024-04-26T03:38:50.057 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:38:50.057 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:38:50.057 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:38:50.057 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:38:50.057 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4213", 2024-04-26T03:38:50.057 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/331201604", 2024-04-26T03:38:50.057 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 165 2024-04-26T03:38:50.057 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.057 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:38:50.057 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.057 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:38:50.058 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.911813+0000", 2024-04-26T03:38:50.058 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:50.058 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.058 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.058 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:38:50.058 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.911813+0000", 2024-04-26T03:38:50.058 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:50.058 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.058 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.058 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:38:50.058 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.911813+0000", 2024-04-26T03:38:50.058 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.9700000000000003e-07 2024-04-26T03:38:50.058 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.058 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.059 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:38:50.059 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.911817+0000", 2024-04-26T03:38:50.059 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9570000000000004e-06 2024-04-26T03:38:50.059 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.059 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.059 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:38:50.059 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.911820+0000", 2024-04-26T03:38:50.059 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.1480000000000002e-06 2024-04-26T03:38:50.059 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.059 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.059 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:38:50.059 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.911823+0000", 2024-04-26T03:38:50.059 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4360000000000001e-06 2024-04-26T03:38:50.059 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.060 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.060 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:38:50.060 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.918780+0000", 2024-04-26T03:38:50.060 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.006957006 2024-04-26T03:38:50.060 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.060 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.060 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:38:50.060 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.918849+0000", 2024-04-26T03:38:50.060 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.9521000000000001e-05 2024-04-26T03:38:50.060 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.060 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.060 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 5", 2024-04-26T03:38:50.060 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.918921+0000", 2024-04-26T03:38:50.060 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.1729000000000004e-05 2024-04-26T03:38:50.061 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.061 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.061 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-04-26T03:38:50.061 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.919486+0000", 2024-04-26T03:38:50.061 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00056472200000000003 2024-04-26T03:38:50.061 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.061 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.061 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-04-26T03:38:50.061 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.919663+0000", 2024-04-26T03:38:50.061 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00017785899999999999 2024-04-26T03:38:50.061 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.061 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.061 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:38:50.061 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.919714+0000", 2024-04-26T03:38:50.061 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.0676000000000001e-05 2024-04-26T03:38:50.062 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.062 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.062 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:38:50.062 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.919754+0000", 2024-04-26T03:38:50.062 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9524000000000003e-05 2024-04-26T03:38:50.062 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:50.062 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:38:50.062 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:50.062 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.062 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.062 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4213.0:186 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-04-26T03:38:50.062 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:35.026739+0000", 2024-04-26T03:38:50.062 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 14.972561602000001, 2024-04-26T03:38:50.062 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0076306680000000002, 2024-04-26T03:38:50.063 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:38:50.063 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:38:50.063 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:38:50.063 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:38:50.063 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4213", 2024-04-26T03:38:50.063 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/331201604", 2024-04-26T03:38:50.063 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 186 2024-04-26T03:38:50.063 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.063 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:38:50.063 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.063 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:38:50.063 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.026739+0000", 2024-04-26T03:38:50.063 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:50.063 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.064 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.064 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:38:50.064 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.026739+0000", 2024-04-26T03:38:50.064 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:50.064 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.064 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.064 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:38:50.064 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.026740+0000", 2024-04-26T03:38:50.064 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8000000000000001e-07 2024-04-26T03:38:50.064 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.064 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.064 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:38:50.064 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.026742+0000", 2024-04-26T03:38:50.064 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4140000000000001e-06 2024-04-26T03:38:50.064 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.065 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.065 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:38:50.065 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.026746+0000", 2024-04-26T03:38:50.065 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.788e-06 2024-04-26T03:38:50.065 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.065 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.065 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:38:50.065 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.026750+0000", 2024-04-26T03:38:50.065 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.5499999999999999e-06 2024-04-26T03:38:50.065 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.065 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.065 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:38:50.065 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.031935+0000", 2024-04-26T03:38:50.065 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.005185556 2024-04-26T03:38:50.066 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.066 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.066 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:38:50.066 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.032020+0000", 2024-04-26T03:38:50.066 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.4858000000000006e-05 2024-04-26T03:38:50.066 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.066 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.066 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 5", 2024-04-26T03:38:50.066 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.032143+0000", 2024-04-26T03:38:50.066 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000123409 2024-04-26T03:38:50.066 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.066 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.066 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-04-26T03:38:50.066 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.033897+0000", 2024-04-26T03:38:50.066 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001753182 2024-04-26T03:38:50.067 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.067 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.067 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-04-26T03:38:50.067 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.034319+0000", 2024-04-26T03:38:50.067 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00042264499999999998 2024-04-26T03:38:50.067 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.067 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.067 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:38:50.067 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.034346+0000", 2024-04-26T03:38:50.067 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6614e-05 2024-04-26T03:38:50.067 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.067 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.067 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:38:50.068 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.034370+0000", 2024-04-26T03:38:50.068 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4272000000000001e-05 2024-04-26T03:38:50.068 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:50.068 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:38:50.068 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:50.068 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.068 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.068 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4213.0:187 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-04-26T03:38:50.068 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:35.026754+0000", 2024-04-26T03:38:50.068 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 14.97254686, 2024-04-26T03:38:50.068 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0077293240000000001, 2024-04-26T03:38:50.068 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:38:50.068 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:38:50.068 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:38:50.068 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:38:50.069 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4213", 2024-04-26T03:38:50.069 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/331201604", 2024-04-26T03:38:50.069 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 187 2024-04-26T03:38:50.069 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.069 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:38:50.069 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.069 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:38:50.069 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.026754+0000", 2024-04-26T03:38:50.069 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:50.069 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.069 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.069 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:38:50.069 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.026754+0000", 2024-04-26T03:38:50.069 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:50.070 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.070 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.070 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:38:50.070 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.026754+0000", 2024-04-26T03:38:50.070 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4699999999999998e-07 2024-04-26T03:38:50.070 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.070 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.070 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:38:50.070 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.026757+0000", 2024-04-26T03:38:50.070 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0600000000000002e-06 2024-04-26T03:38:50.070 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.070 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.070 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:38:50.071 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.026760+0000", 2024-04-26T03:38:50.071 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.1820000000000002e-06 2024-04-26T03:38:50.071 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.071 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.071 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:38:50.071 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.026762+0000", 2024-04-26T03:38:50.071 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4140000000000001e-06 2024-04-26T03:38:50.071 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.071 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.071 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:38:50.071 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.032580+0000", 2024-04-26T03:38:50.071 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0058182019999999997 2024-04-26T03:38:50.071 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.072 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.072 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:38:50.072 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.032629+0000", 2024-04-26T03:38:50.072 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.8667e-05 2024-04-26T03:38:50.072 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.072 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.072 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 5", 2024-04-26T03:38:50.072 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.032700+0000", 2024-04-26T03:38:50.072 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.0862e-05 2024-04-26T03:38:50.072 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.072 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.072 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-04-26T03:38:50.072 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.033948+0000", 2024-04-26T03:38:50.072 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0012480289999999999 2024-04-26T03:38:50.072 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.073 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.073 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-04-26T03:38:50.073 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.034429+0000", 2024-04-26T03:38:50.073 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00048097 2024-04-26T03:38:50.073 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.073 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.073 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:38:50.073 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.034454+0000", 2024-04-26T03:38:50.073 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5160999999999998e-05 2024-04-26T03:38:50.073 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.073 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.073 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:38:50.073 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.034484+0000", 2024-04-26T03:38:50.073 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.953e-05 2024-04-26T03:38:50.074 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:50.074 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:38:50.074 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:50.074 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.074 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.074 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4213.0:188 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-04-26T03:38:50.074 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:35.026766+0000", 2024-04-26T03:38:50.074 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 14.972535063, 2024-04-26T03:38:50.074 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.007768268, 2024-04-26T03:38:50.074 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:38:50.074 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:38:50.074 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:38:50.074 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:38:50.075 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4213", 2024-04-26T03:38:50.075 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/331201604", 2024-04-26T03:38:50.075 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 188 2024-04-26T03:38:50.075 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.075 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:38:50.075 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.075 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:38:50.075 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.026766+0000", 2024-04-26T03:38:50.075 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:50.075 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.075 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.075 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:38:50.075 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.026766+0000", 2024-04-26T03:38:50.075 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:50.076 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.076 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.076 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:38:50.076 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.026766+0000", 2024-04-26T03:38:50.076 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.23e-07 2024-04-26T03:38:50.076 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.076 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.076 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:38:50.076 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.026801+0000", 2024-04-26T03:38:50.076 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.5247999999999999e-05 2024-04-26T03:38:50.076 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.076 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.076 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:38:50.076 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.026809+0000", 2024-04-26T03:38:50.076 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.6620000000000005e-06 2024-04-26T03:38:50.076 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.077 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.077 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:38:50.077 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.026813+0000", 2024-04-26T03:38:50.077 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.45e-06 2024-04-26T03:38:50.077 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.077 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.077 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:38:50.077 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.033013+0000", 2024-04-26T03:38:50.077 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0062003270000000003 2024-04-26T03:38:50.077 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.077 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.077 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:38:50.077 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.033084+0000", 2024-04-26T03:38:50.077 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.0893e-05 2024-04-26T03:38:50.078 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.078 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.078 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 5", 2024-04-26T03:38:50.078 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.033214+0000", 2024-04-26T03:38:50.078 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012982099999999999 2024-04-26T03:38:50.078 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.078 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.078 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-04-26T03:38:50.078 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.034414+0000", 2024-04-26T03:38:50.078 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0012002429999999999 2024-04-26T03:38:50.078 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.078 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.078 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-04-26T03:38:50.078 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.034488+0000", 2024-04-26T03:38:50.079 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.4388999999999995e-05 2024-04-26T03:38:50.079 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.079 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.079 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:38:50.079 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.034512+0000", 2024-04-26T03:38:50.079 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3900000000000002e-05 2024-04-26T03:38:50.079 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.079 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.079 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:38:50.079 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.034534+0000", 2024-04-26T03:38:50.079 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2112000000000001e-05 2024-04-26T03:38:50.079 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:50.079 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:38:50.079 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:50.079 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.080 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.080 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4213.0:189 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-04-26T03:38:50.080 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:35.026818+0000", 2024-04-26T03:38:50.080 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 14.97248302, 2024-04-26T03:38:50.080 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0082990249999999998, 2024-04-26T03:38:50.080 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:38:50.080 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:38:50.080 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:38:50.080 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:38:50.080 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4213", 2024-04-26T03:38:50.080 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/331201604", 2024-04-26T03:38:50.080 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 189 2024-04-26T03:38:50.080 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.081 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:38:50.081 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.081 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:38:50.081 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.026818+0000", 2024-04-26T03:38:50.081 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:50.081 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.081 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.081 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:38:50.081 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.026818+0000", 2024-04-26T03:38:50.081 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:50.081 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.081 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.081 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:38:50.081 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.026818+0000", 2024-04-26T03:38:50.081 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.0800000000000001e-07 2024-04-26T03:38:50.082 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.082 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.082 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:38:50.082 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.026821+0000", 2024-04-26T03:38:50.082 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.181e-06 2024-04-26T03:38:50.082 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.082 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.082 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:38:50.082 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.026824+0000", 2024-04-26T03:38:50.082 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.2169999999999999e-06 2024-04-26T03:38:50.082 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.082 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.082 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:38:50.082 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.026826+0000", 2024-04-26T03:38:50.083 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5069999999999999e-06 2024-04-26T03:38:50.083 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.083 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.083 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:38:50.083 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.033975+0000", 2024-04-26T03:38:50.083 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0071489789999999998 2024-04-26T03:38:50.083 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.083 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.083 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:38:50.083 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.034018+0000", 2024-04-26T03:38:50.083 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.2894e-05 2024-04-26T03:38:50.083 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.083 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.083 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 5", 2024-04-26T03:38:50.084 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.034088+0000", 2024-04-26T03:38:50.084 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.9801999999999993e-05 2024-04-26T03:38:50.084 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.084 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.084 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-04-26T03:38:50.084 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.034753+0000", 2024-04-26T03:38:50.084 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00066494999999999998 2024-04-26T03:38:50.084 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.084 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.084 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-04-26T03:38:50.084 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.035072+0000", 2024-04-26T03:38:50.084 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000318814 2024-04-26T03:38:50.084 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.084 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.085 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:38:50.085 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.035097+0000", 2024-04-26T03:38:50.085 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5664999999999999e-05 2024-04-26T03:38:50.085 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.085 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.085 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:38:50.085 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:35.035117+0000", 2024-04-26T03:38:50.085 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9707999999999998e-05 2024-04-26T03:38:50.085 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:50.085 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:38:50.085 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:50.085 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.085 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.085 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4213.0:196 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [call lock.lock in=8b,cmpxattr striper.excl op 1 mode 1 in=12b,setxattr striper.excl (24) in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-26T03:38:50.085 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:36.409120+0000", 2024-04-26T03:38:50.086 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 13.590180875, 2024-04-26T03:38:50.086 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0071594149999999997, 2024-04-26T03:38:50.086 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:38:50.086 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:38:50.086 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:38:50.086 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:38:50.086 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4213", 2024-04-26T03:38:50.086 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/331201604", 2024-04-26T03:38:50.086 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 196 2024-04-26T03:38:50.086 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.086 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:38:50.086 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.086 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:38:50.086 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.409120+0000", 2024-04-26T03:38:50.087 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:50.087 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.087 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.087 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:38:50.087 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.409120+0000", 2024-04-26T03:38:50.087 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:50.087 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.087 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.087 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:38:50.087 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.409127+0000", 2024-04-26T03:38:50.087 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.2359999999999998e-06 2024-04-26T03:38:50.087 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.087 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.087 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:38:50.088 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.409147+0000", 2024-04-26T03:38:50.088 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0032999999999999e-05 2024-04-26T03:38:50.088 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.088 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.088 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:38:50.088 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.409174+0000", 2024-04-26T03:38:50.088 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6974999999999998e-05 2024-04-26T03:38:50.088 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.088 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.088 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:38:50.088 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.409202+0000", 2024-04-26T03:38:50.088 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7778999999999999e-05 2024-04-26T03:38:50.088 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.088 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.089 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:38:50.089 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.409287+0000", 2024-04-26T03:38:50.089 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.4468999999999994e-05 2024-04-26T03:38:50.089 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.089 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.089 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op must wait for map", 2024-04-26T03:38:50.089 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.409291+0000", 2024-04-26T03:38:50.089 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.7319999999999998e-06 2024-04-26T03:38:50.089 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.089 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.089 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:38:50.089 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.412593+0000", 2024-04-26T03:38:50.089 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.003301277 2024-04-26T03:38:50.089 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.090 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.090 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:38:50.090 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.412705+0000", 2024-04-26T03:38:50.090 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000112391 2024-04-26T03:38:50.090 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.090 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.090 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 5", 2024-04-26T03:38:50.090 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.412949+0000", 2024-04-26T03:38:50.090 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00024440600000000002 2024-04-26T03:38:50.090 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.090 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.090 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-04-26T03:38:50.090 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.414016+0000", 2024-04-26T03:38:50.090 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001066733 2024-04-26T03:38:50.091 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.091 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.091 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-04-26T03:38:50.091 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.416219+0000", 2024-04-26T03:38:50.091 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0022029160000000001 2024-04-26T03:38:50.091 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.091 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.091 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:38:50.091 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.416254+0000", 2024-04-26T03:38:50.091 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.4897999999999997e-05 2024-04-26T03:38:50.091 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.091 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.091 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:38:50.091 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.416280+0000", 2024-04-26T03:38:50.092 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5570000000000001e-05 2024-04-26T03:38:50.092 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:50.092 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:38:50.092 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:50.092 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.092 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.092 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4213.0:203 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-26T03:38:50.092 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:36.419000+0000", 2024-04-26T03:38:50.092 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 13.580300737, 2024-04-26T03:38:50.092 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0078783970000000005, 2024-04-26T03:38:50.092 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:38:50.092 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:38:50.092 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:38:50.092 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:38:50.092 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4213", 2024-04-26T03:38:50.093 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/331201604", 2024-04-26T03:38:50.093 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 203 2024-04-26T03:38:50.093 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.093 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:38:50.093 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.093 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:38:50.093 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.419000+0000", 2024-04-26T03:38:50.093 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:50.093 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.093 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.093 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:38:50.093 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.419000+0000", 2024-04-26T03:38:50.093 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:50.093 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.093 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.094 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:38:50.094 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.419001+0000", 2024-04-26T03:38:50.094 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.3300000000000002e-07 2024-04-26T03:38:50.094 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.094 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.094 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:38:50.094 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.419165+0000", 2024-04-26T03:38:50.094 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00016440899999999999 2024-04-26T03:38:50.094 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.094 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.094 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:38:50.098 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.419174+0000", 2024-04-26T03:38:50.098 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.3480000000000004e-06 2024-04-26T03:38:50.098 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.098 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.098 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:38:50.098 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.419178+0000", 2024-04-26T03:38:50.098 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.3080000000000001e-06 2024-04-26T03:38:50.098 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.098 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.098 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:38:50.098 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.419750+0000", 2024-04-26T03:38:50.098 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000572432 2024-04-26T03:38:50.098 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.099 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.099 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:38:50.099 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.419828+0000", 2024-04-26T03:38:50.099 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.7065999999999997e-05 2024-04-26T03:38:50.099 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.099 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.099 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 5", 2024-04-26T03:38:50.099 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.419948+0000", 2024-04-26T03:38:50.099 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000120072 2024-04-26T03:38:50.099 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.099 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.099 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-04-26T03:38:50.099 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.423947+0000", 2024-04-26T03:38:50.099 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0039997540000000003 2024-04-26T03:38:50.100 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.100 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.100 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-04-26T03:38:50.100 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.426777+0000", 2024-04-26T03:38:50.100 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0028293250000000002 2024-04-26T03:38:50.100 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.100 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.100 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:38:50.100 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.426835+0000", 2024-04-26T03:38:50.100 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.8158000000000001e-05 2024-04-26T03:38:50.100 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.100 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.100 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:38:50.100 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.426879+0000", 2024-04-26T03:38:50.100 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.3891999999999999e-05 2024-04-26T03:38:50.101 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:50.101 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:38:50.101 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:50.101 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.101 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.101 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4213.0:204 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-26T03:38:50.101 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:36.419186+0000", 2024-04-26T03:38:50.101 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 13.580114663, 2024-04-26T03:38:50.101 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0077545069999999999, 2024-04-26T03:38:50.101 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:38:50.101 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:38:50.101 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:38:50.101 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:38:50.102 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4213", 2024-04-26T03:38:50.102 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/331201604", 2024-04-26T03:38:50.102 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 204 2024-04-26T03:38:50.102 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.102 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:38:50.102 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.102 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:38:50.102 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.419186+0000", 2024-04-26T03:38:50.102 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:50.102 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.102 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.102 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:38:50.102 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.419186+0000", 2024-04-26T03:38:50.102 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:50.103 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.103 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.103 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:38:50.103 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.419187+0000", 2024-04-26T03:38:50.103 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.4000000000000002e-07 2024-04-26T03:38:50.103 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.103 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.103 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:38:50.103 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.419189+0000", 2024-04-26T03:38:50.103 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6460000000000002e-06 2024-04-26T03:38:50.103 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.103 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.103 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:38:50.103 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.419196+0000", 2024-04-26T03:38:50.104 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.0800000000000002e-06 2024-04-26T03:38:50.104 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.104 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.104 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:38:50.104 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.419200+0000", 2024-04-26T03:38:50.104 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.741e-06 2024-04-26T03:38:50.104 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.104 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.104 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:38:50.104 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.422829+0000", 2024-04-26T03:38:50.104 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0036288359999999999 2024-04-26T03:38:50.104 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.104 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.104 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:38:50.104 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.422914+0000", 2024-04-26T03:38:50.105 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.4401999999999996e-05 2024-04-26T03:38:50.105 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.105 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.105 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 5", 2024-04-26T03:38:50.105 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.423052+0000", 2024-04-26T03:38:50.105 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013814099999999999 2024-04-26T03:38:50.105 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.105 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.105 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-04-26T03:38:50.105 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.423977+0000", 2024-04-26T03:38:50.105 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00092557999999999996 2024-04-26T03:38:50.105 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.105 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.105 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-04-26T03:38:50.105 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.426883+0000", 2024-04-26T03:38:50.106 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.002905936 2024-04-26T03:38:50.106 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.106 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.106 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:38:50.106 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.426914+0000", 2024-04-26T03:38:50.106 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.0973999999999999e-05 2024-04-26T03:38:50.106 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.106 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.106 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:38:50.106 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.426941+0000", 2024-04-26T03:38:50.106 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6730999999999999e-05 2024-04-26T03:38:50.106 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:50.106 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:38:50.106 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:50.107 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.107 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.107 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4213.0:205 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-26T03:38:50.107 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:36.419206+0000", 2024-04-26T03:38:50.107 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 13.580094925999999, 2024-04-26T03:38:50.107 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0078178550000000003, 2024-04-26T03:38:50.107 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:38:50.107 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:38:50.107 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:38:50.107 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:38:50.107 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4213", 2024-04-26T03:38:50.107 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/331201604", 2024-04-26T03:38:50.107 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 205 2024-04-26T03:38:50.107 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.107 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:38:50.108 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.108 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:38:50.108 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.419206+0000", 2024-04-26T03:38:50.108 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:50.108 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.108 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.108 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:38:50.108 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.419206+0000", 2024-04-26T03:38:50.108 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:50.108 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.108 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.108 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:38:50.108 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.419206+0000", 2024-04-26T03:38:50.108 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3999999999999998e-07 2024-04-26T03:38:50.108 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.109 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.109 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:38:50.109 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.419210+0000", 2024-04-26T03:38:50.109 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.4640000000000002e-06 2024-04-26T03:38:50.109 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.109 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.109 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:38:50.109 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.419213+0000", 2024-04-26T03:38:50.109 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.2449999999999998e-06 2024-04-26T03:38:50.109 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.109 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.109 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:38:50.109 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.419223+0000", 2024-04-26T03:38:50.109 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0399e-05 2024-04-26T03:38:50.109 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.110 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.110 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:38:50.110 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.424024+0000", 2024-04-26T03:38:50.110 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.004800075 2024-04-26T03:38:50.110 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.110 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.110 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:38:50.110 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.424076+0000", 2024-04-26T03:38:50.110 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.2899000000000003e-05 2024-04-26T03:38:50.110 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.110 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.110 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 5", 2024-04-26T03:38:50.110 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.424149+0000", 2024-04-26T03:38:50.110 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.2716e-05 2024-04-26T03:38:50.111 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.111 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.111 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-04-26T03:38:50.111 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.426594+0000", 2024-04-26T03:38:50.111 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0024449129999999999 2024-04-26T03:38:50.111 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.111 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.111 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-04-26T03:38:50.111 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.426947+0000", 2024-04-26T03:38:50.111 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00035264399999999999 2024-04-26T03:38:50.111 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.111 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.111 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:38:50.111 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.426990+0000", 2024-04-26T03:38:50.111 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.3495000000000003e-05 2024-04-26T03:38:50.112 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.112 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.112 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:38:50.112 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.427024+0000", 2024-04-26T03:38:50.112 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.3764999999999999e-05 2024-04-26T03:38:50.112 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:50.112 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:38:50.112 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:50.112 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.112 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.112 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4213.0:206 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-26T03:38:50.112 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:36.419230+0000", 2024-04-26T03:38:50.112 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 13.580071346, 2024-04-26T03:38:50.112 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0080294219999999996, 2024-04-26T03:38:50.112 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:38:50.113 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:38:50.113 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:38:50.113 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:38:50.113 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4213", 2024-04-26T03:38:50.113 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/331201604", 2024-04-26T03:38:50.113 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 206 2024-04-26T03:38:50.113 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.113 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:38:50.113 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.113 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:38:50.113 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.419230+0000", 2024-04-26T03:38:50.113 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:50.113 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.113 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.114 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:38:50.114 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.419230+0000", 2024-04-26T03:38:50.114 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:50.114 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.114 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.114 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:38:50.114 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.419230+0000", 2024-04-26T03:38:50.114 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.2800000000000003e-07 2024-04-26T03:38:50.114 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.114 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.114 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:38:50.114 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.419279+0000", 2024-04-26T03:38:50.114 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.9101000000000003e-05 2024-04-26T03:38:50.114 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.115 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.115 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:38:50.115 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.419284+0000", 2024-04-26T03:38:50.115 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.7010000000000004e-06 2024-04-26T03:38:50.115 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.115 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.115 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:38:50.115 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.419287+0000", 2024-04-26T03:38:50.115 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.9270000000000001e-06 2024-04-26T03:38:50.115 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.115 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.115 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:38:50.115 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.424419+0000", 2024-04-26T03:38:50.115 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0051324580000000003 2024-04-26T03:38:50.115 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.116 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.116 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:38:50.116 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.424466+0000", 2024-04-26T03:38:50.116 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6791e-05 2024-04-26T03:38:50.116 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.116 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.116 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 5", 2024-04-26T03:38:50.116 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.424538+0000", 2024-04-26T03:38:50.116 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.1958000000000005e-05 2024-04-26T03:38:50.116 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.116 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.116 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-04-26T03:38:50.116 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.426763+0000", 2024-04-26T03:38:50.116 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0022254900000000001 2024-04-26T03:38:50.116 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.117 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.117 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-04-26T03:38:50.117 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.427206+0000", 2024-04-26T03:38:50.117 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00044210700000000001 2024-04-26T03:38:50.117 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.117 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.117 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:38:50.117 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.427235+0000", 2024-04-26T03:38:50.117 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.9688000000000001e-05 2024-04-26T03:38:50.117 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.117 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.117 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:38:50.117 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.427259+0000", 2024-04-26T03:38:50.117 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3873e-05 2024-04-26T03:38:50.118 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:50.118 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:38:50.118 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:50.118 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.118 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.118 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4213.0:207 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-04-26T03:38:50.118 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:36.419292+0000", 2024-04-26T03:38:50.118 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 13.580008775, 2024-04-26T03:38:50.118 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0082031559999999996, 2024-04-26T03:38:50.118 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:38:50.118 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:38:50.118 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:38:50.118 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:38:50.118 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4213", 2024-04-26T03:38:50.119 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/331201604", 2024-04-26T03:38:50.119 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 207 2024-04-26T03:38:50.119 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.119 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:38:50.119 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.119 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:38:50.119 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.419292+0000", 2024-04-26T03:38:50.119 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:50.119 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.119 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.119 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:38:50.119 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.419292+0000", 2024-04-26T03:38:50.119 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:50.119 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.119 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.120 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:38:50.120 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.419293+0000", 2024-04-26T03:38:50.120 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7599999999999998e-07 2024-04-26T03:38:50.120 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.120 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.120 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:38:50.120 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.419295+0000", 2024-04-26T03:38:50.120 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3439999999999999e-06 2024-04-26T03:38:50.120 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.120 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.120 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:38:50.120 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.419303+0000", 2024-04-26T03:38:50.120 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8949999999999998e-06 2024-04-26T03:38:50.120 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.121 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.121 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:38:50.121 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.419305+0000", 2024-04-26T03:38:50.121 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4820000000000001e-06 2024-04-26T03:38:50.121 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.121 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.121 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:38:50.121 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.426015+0000", 2024-04-26T03:38:50.121 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.006709329 2024-04-26T03:38:50.121 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.121 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.121 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:38:50.121 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.426095+0000", 2024-04-26T03:38:50.121 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.0632999999999999e-05 2024-04-26T03:38:50.122 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.122 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.122 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 5", 2024-04-26T03:38:50.122 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.426213+0000", 2024-04-26T03:38:50.122 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000118127 2024-04-26T03:38:50.122 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.122 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.122 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-04-26T03:38:50.122 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.427192+0000", 2024-04-26T03:38:50.122 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00097844399999999997 2024-04-26T03:38:50.122 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.122 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.122 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-04-26T03:38:50.122 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.427450+0000", 2024-04-26T03:38:50.123 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00025866600000000001 2024-04-26T03:38:50.123 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.123 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.123 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:38:50.123 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.427476+0000", 2024-04-26T03:38:50.123 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5418e-05 2024-04-26T03:38:50.123 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:50.123 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:50.123 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:38:50.123 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:36.427495+0000", 2024-04-26T03:38:50.123 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9542e-05 2024-04-26T03:38:50.123 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:50.123 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:38:50.123 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:50.123 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:50.124 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:38:50.124 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:38:50.125 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-04-26T03:38:50.126 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-04-26T03:38:50.123+0000 7fee562fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31759) UID: 0 2024-04-26T03:38:50.226 INFO:tasks.ceph.osd.4.smithi028.stderr:2024-04-26T03:38:50.224+0000 7fd9511b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31963) UID: 0 2024-04-26T03:38:50.273 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:38:50.273 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-04-26T03:38:50.274 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 0 2024-04-26T03:38:50.274 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:38:50.282 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-04-26T03:38:50.326 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:38:50.324+0000 7f60d3c4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31757) UID: 0 2024-04-26T03:38:50.389 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:38:50.389 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-04-26T03:38:50.389 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-04-26T03:38:50.389 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-04-26T03:38:50.389 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:38:50.398 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-04-26T03:38:50.439 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-04-26T03:38:50.437+0000 7fee562fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31759) UID: 0 2024-04-26T03:38:50.476 INFO:teuthology.orchestra.run.smithi028.stderr:nodeep-scrub is set 2024-04-26T03:38:50.493 DEBUG:teuthology.orchestra.run.smithi092:> CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no-omap --ec-pool --max-ops 400 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 0 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op setattr 25 --op rmattr 25 --op copy_from 50 --op append 100 --pool unique_pool_0 2024-04-26T03:38:50.526 INFO:tasks.rados.rados.0.smithi092.stdout:adding op weight read -> 100 2024-04-26T03:38:50.526 INFO:tasks.rados.rados.0.smithi092.stdout:adding op weight delete -> 50 2024-04-26T03:38:50.526 INFO:tasks.rados.rados.0.smithi092.stdout:adding op weight snap_create -> 50 2024-04-26T03:38:50.526 INFO:tasks.rados.rados.0.smithi092.stdout:adding op weight snap_remove -> 50 2024-04-26T03:38:50.526 INFO:tasks.rados.rados.0.smithi092.stdout:adding op weight rollback -> 50 2024-04-26T03:38:50.527 INFO:tasks.rados.rados.0.smithi092.stdout:adding op weight setattr -> 25 2024-04-26T03:38:50.527 INFO:tasks.rados.rados.0.smithi092.stdout:adding op weight rmattr -> 25 2024-04-26T03:38:50.527 INFO:tasks.rados.rados.0.smithi092.stdout:adding op weight copy_from -> 50 2024-04-26T03:38:50.527 INFO:tasks.rados.rados.0.smithi092.stdout:adding op weight append -> 100 2024-04-26T03:38:50.527 INFO:tasks.rados.rados.0.smithi092.stdout:ceph version 19.0.0-2457-gddeaa917 (ddeaa91797339a19172c3036ff48cca58c12f448) squid (dev) 2024-04-26T03:38:50.527 INFO:tasks.rados.rados.0.smithi092.stdout:Configuration: 2024-04-26T03:38:50.527 INFO:tasks.rados.rados.0.smithi092.stdout: Number of operations: 400 2024-04-26T03:38:50.527 INFO:tasks.rados.rados.0.smithi092.stdout: Number of objects: 50 2024-04-26T03:38:50.527 INFO:tasks.rados.rados.0.smithi092.stdout: Max in flight operations: 16 2024-04-26T03:38:50.527 INFO:tasks.rados.rados.0.smithi092.stdout: Object size (in bytes): 4000000 2024-04-26T03:38:50.527 INFO:tasks.rados.rados.0.smithi092.stdout: Write stride min: 400000 2024-04-26T03:38:50.528 INFO:tasks.rados.rados.0.smithi092.stdout: Write stride max: 800000 2024-04-26T03:38:50.534 INFO:tasks.ceph.osd.8.smithi028.stderr:2024-04-26T03:38:50.532+0000 7f5bb75d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 31968) UID: 0 2024-04-26T03:38:50.546 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:38:50.546 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-04-26T03:38:50.546 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-04-26T03:38:50.547 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [] 2024-04-26T03:38:50.547 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:38:50.555 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-04-26T03:38:50.630 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:38:50.628+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:38:50.664 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:38:50.665 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-04-26T03:38:50.665 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 0 2024-04-26T03:38:50.665 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:38:50.673 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-04-26T03:38:50.730 INFO:tasks.ceph.osd.8.smithi028.stderr:2024-04-26T03:38:50.729+0000 7f5bb75d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 31968) UID: 0 2024-04-26T03:38:50.780 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:38:50.780 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-04-26T03:38:50.780 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-04-26T03:38:50.780 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-04-26T03:38:50.780 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:38:50.789 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-04-26T03:38:50.849 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-04-26T03:38:50.847+0000 7f745118d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 31821) UID: 0 2024-04-26T03:38:50.949 INFO:tasks.ceph.osd.15.smithi092.stderr:2024-04-26T03:38:50.947+0000 7f6d12dec640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 31824) UID: 0 2024-04-26T03:38:50.955 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:38:50.955 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-04-26T03:38:50.955 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-04-26T03:38:50.955 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [] 2024-04-26T03:38:50.955 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:38:50.964 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-26T03:38:51.050 INFO:tasks.ceph.osd.15.smithi092.stderr:2024-04-26T03:38:51.048+0000 7f6d12dec640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 31824) UID: 0 2024-04-26T03:38:51.081 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:38:51.081 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-26T03:38:51.081 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-26T03:38:51.082 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:38:51.091 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-26T03:38:51.151 INFO:tasks.ceph.osd.8.smithi028.stderr:2024-04-26T03:38:51.149+0000 7f5bb75d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 31968) UID: 0 2024-04-26T03:38:51.239 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:38:51.239 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-26T03:38:51.239 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-26T03:38:51.239 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-26T03:38:51.239 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:38:51.247 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-26T03:38:51.252 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:38:51.250+0000 7fb78c544640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31956) UID: 0 2024-04-26T03:38:51.350 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:38:51.351 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-26T03:38:51.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-26T03:38:51.351 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [] 2024-04-26T03:38:51.351 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:38:51.353 INFO:tasks.ceph.osd.14.smithi066.stderr:2024-04-26T03:38:51.352+0000 7f5395f08640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 31765) UID: 0 2024-04-26T03:38:51.359 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-26T03:38:51.431 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-26T03:38:51.454 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:38:51.452+0000 7f60d3c4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31757) UID: 0 2024-04-26T03:38:51.461 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:38:51.461 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-26T03:38:51.461 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-26T03:38:51.461 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:38:51.470 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-26T03:38:51.481 INFO:tasks.rados.rados.0.smithi092.stdout:1: write initial oid 1 2024-04-26T03:38:51.481 INFO:tasks.rados.rados.0.smithi092.stdout:1: seq_num 0 ranges {0=32768} 2024-04-26T03:38:51.481 INFO:tasks.rados.rados.0.smithi092.stdout:1: writing smithi09235267-1 from 0 to 32768 tid 1 2024-04-26T03:38:51.481 INFO:tasks.rados.rados.0.smithi092.stdout:2: write initial oid 2 2024-04-26T03:38:51.481 INFO:tasks.rados.rados.0.smithi092.stdout:2: seq_num 1 ranges {0=32768} 2024-04-26T03:38:51.481 INFO:tasks.rados.rados.0.smithi092.stdout:2: writing smithi09235267-2 from 0 to 32768 tid 1 2024-04-26T03:38:51.481 INFO:tasks.rados.rados.0.smithi092.stdout:3: write initial oid 3 2024-04-26T03:38:51.481 INFO:tasks.rados.rados.0.smithi092.stdout:3: seq_num 2 ranges {0=49152} 2024-04-26T03:38:51.481 INFO:tasks.rados.rados.0.smithi092.stdout:3: writing smithi09235267-3 from 0 to 49152 tid 1 2024-04-26T03:38:51.481 INFO:tasks.rados.rados.0.smithi092.stdout:4: write initial oid 4 2024-04-26T03:38:51.481 INFO:tasks.rados.rados.0.smithi092.stdout:4: seq_num 3 ranges {0=65536} 2024-04-26T03:38:51.482 INFO:tasks.rados.rados.0.smithi092.stdout:4: writing smithi09235267-4 from 0 to 65536 tid 1 2024-04-26T03:38:51.482 INFO:tasks.rados.rados.0.smithi092.stdout:5: write initial oid 5 2024-04-26T03:38:51.482 INFO:tasks.rados.rados.0.smithi092.stdout:5: seq_num 4 ranges {0=81920} 2024-04-26T03:38:51.482 INFO:tasks.rados.rados.0.smithi092.stdout:5: writing smithi09235267-5 from 0 to 81920 tid 1 2024-04-26T03:38:51.482 INFO:tasks.rados.rados.0.smithi092.stdout:6: write initial oid 6 2024-04-26T03:38:51.482 INFO:tasks.rados.rados.0.smithi092.stdout:6: seq_num 5 ranges {0=98304} 2024-04-26T03:38:51.483 INFO:tasks.rados.rados.0.smithi092.stdout:6: writing smithi09235267-6 from 0 to 98304 tid 1 2024-04-26T03:38:51.483 INFO:tasks.rados.rados.0.smithi092.stdout:7: write initial oid 7 2024-04-26T03:38:51.483 INFO:tasks.rados.rados.0.smithi092.stdout:7: seq_num 6 ranges {0=114688} 2024-04-26T03:38:51.484 INFO:tasks.rados.rados.0.smithi092.stdout:7: writing smithi09235267-7 from 0 to 114688 tid 1 2024-04-26T03:38:51.484 INFO:tasks.rados.rados.0.smithi092.stdout:8: write initial oid 8 2024-04-26T03:38:51.484 INFO:tasks.rados.rados.0.smithi092.stdout:8: seq_num 7 ranges {0=131072} 2024-04-26T03:38:51.485 INFO:tasks.rados.rados.0.smithi092.stdout:8: writing smithi09235267-8 from 0 to 131072 tid 1 2024-04-26T03:38:51.485 INFO:tasks.rados.rados.0.smithi092.stdout:9: write initial oid 9 2024-04-26T03:38:51.485 INFO:tasks.rados.rados.0.smithi092.stdout:9: seq_num 8 ranges {0=147456} 2024-04-26T03:38:51.487 INFO:tasks.rados.rados.0.smithi092.stdout:9: writing smithi09235267-9 from 0 to 147456 tid 1 2024-04-26T03:38:51.487 INFO:tasks.rados.rados.0.smithi092.stdout:10: write initial oid 10 2024-04-26T03:38:51.487 INFO:tasks.rados.rados.0.smithi092.stdout:10: seq_num 9 ranges {0=163840} 2024-04-26T03:38:51.488 INFO:tasks.rados.rados.0.smithi092.stdout:10: writing smithi09235267-10 from 0 to 163840 tid 1 2024-04-26T03:38:51.488 INFO:tasks.rados.rados.0.smithi092.stdout:11: write initial oid 11 2024-04-26T03:38:51.488 INFO:tasks.rados.rados.0.smithi092.stdout:11: seq_num 10 ranges {0=180224} 2024-04-26T03:38:51.493 INFO:tasks.rados.rados.0.smithi092.stdout:11: writing smithi09235267-11 from 0 to 180224 tid 1 2024-04-26T03:38:51.493 INFO:tasks.rados.rados.0.smithi092.stdout:12: write initial oid 12 2024-04-26T03:38:51.493 INFO:tasks.rados.rados.0.smithi092.stdout:12: seq_num 11 ranges {0=196608} 2024-04-26T03:38:51.493 INFO:tasks.rados.rados.0.smithi092.stdout:12: writing smithi09235267-12 from 0 to 196608 tid 1 2024-04-26T03:38:51.493 INFO:tasks.rados.rados.0.smithi092.stdout:13: write initial oid 13 2024-04-26T03:38:51.494 INFO:tasks.rados.rados.0.smithi092.stdout:13: seq_num 12 ranges {0=212992} 2024-04-26T03:38:51.494 INFO:tasks.rados.rados.0.smithi092.stdout:13: writing smithi09235267-13 from 0 to 212992 tid 1 2024-04-26T03:38:51.494 INFO:tasks.rados.rados.0.smithi092.stdout:14: write initial oid 14 2024-04-26T03:38:51.494 INFO:tasks.rados.rados.0.smithi092.stdout:14: seq_num 13 ranges {0=229376} 2024-04-26T03:38:51.498 INFO:tasks.rados.rados.0.smithi092.stdout:14: writing smithi09235267-14 from 0 to 229376 tid 1 2024-04-26T03:38:51.498 INFO:tasks.rados.rados.0.smithi092.stdout:1: finishing write tid 1 to smithi09235267-1 2024-04-26T03:38:51.498 INFO:tasks.rados.rados.0.smithi092.stdout:2: finishing write tid 1 to smithi09235267-2 2024-04-26T03:38:51.498 INFO:tasks.rados.rados.0.smithi092.stdout:1: finishing write tid 2 to smithi09235267-1 2024-04-26T03:38:51.498 INFO:tasks.rados.rados.0.smithi092.stdout:2: finishing write tid 2 to smithi09235267-2 2024-04-26T03:38:51.498 INFO:tasks.rados.rados.0.smithi092.stdout:1: finishing write tid 3 to smithi09235267-1 2024-04-26T03:38:51.498 INFO:tasks.rados.rados.0.smithi092.stdout:1: oid 1 updating version 0 to 1 2024-04-26T03:38:51.498 INFO:tasks.rados.rados.0.smithi092.stdout:1: oid 1 updating version 1 to 2 2024-04-26T03:38:51.498 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 1 v 2 (ObjNum 0 snap 0 seq_num 0) dirty exists 2024-04-26T03:38:51.498 INFO:tasks.rados.rados.0.smithi092.stdout:1: left oid 1 (ObjNum 0 snap 0 seq_num 0) 2024-04-26T03:38:51.498 INFO:tasks.rados.rados.0.smithi092.stdout:4: finishing write tid 1 to smithi09235267-4 2024-04-26T03:38:51.499 INFO:tasks.rados.rados.0.smithi092.stdout:3: finishing write tid 1 to smithi09235267-3 2024-04-26T03:38:51.499 INFO:tasks.rados.rados.0.smithi092.stdout:3: finishing write tid 2 to smithi09235267-3 2024-04-26T03:38:51.499 INFO:tasks.rados.rados.0.smithi092.stdout:2: finishing write tid 3 to smithi09235267-2 2024-04-26T03:38:51.499 INFO:tasks.rados.rados.0.smithi092.stdout:2: oid 2 updating version 0 to 1 2024-04-26T03:38:51.499 INFO:tasks.rados.rados.0.smithi092.stdout:2: oid 2 updating version 1 to 2 2024-04-26T03:38:51.499 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 2 v 2 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-04-26T03:38:51.499 INFO:tasks.rados.rados.0.smithi092.stdout:2: left oid 2 (ObjNum 1 snap 0 seq_num 1) 2024-04-26T03:38:51.499 INFO:tasks.rados.rados.0.smithi092.stdout:7: finishing write tid 1 to smithi09235267-7 2024-04-26T03:38:51.499 INFO:tasks.rados.rados.0.smithi092.stdout:1: done (13 left) 2024-04-26T03:38:51.499 INFO:tasks.rados.rados.0.smithi092.stdout:2: done (12 left) 2024-04-26T03:38:51.499 INFO:tasks.rados.rados.0.smithi092.stdout:15: write initial oid 15 2024-04-26T03:38:51.499 INFO:tasks.rados.rados.0.smithi092.stdout:15: seq_num 14 ranges {0=245760} 2024-04-26T03:38:51.500 INFO:tasks.rados.rados.0.smithi092.stdout:15: writing smithi09235267-15 from 0 to 245760 tid 1 2024-04-26T03:38:51.500 INFO:tasks.rados.rados.0.smithi092.stdout:16: write initial oid 16 2024-04-26T03:38:51.500 INFO:tasks.rados.rados.0.smithi092.stdout:16: seq_num 15 ranges {0=262144} 2024-04-26T03:38:51.501 INFO:tasks.rados.rados.0.smithi092.stdout:16: writing smithi09235267-16 from 0 to 262144 tid 1 2024-04-26T03:38:51.501 INFO:tasks.rados.rados.0.smithi092.stdout:8: finishing write tid 1 to smithi09235267-8 2024-04-26T03:38:51.501 INFO:tasks.rados.rados.0.smithi092.stdout:6: finishing write tid 1 to smithi09235267-6 2024-04-26T03:38:51.501 INFO:tasks.rados.rados.0.smithi092.stdout:8: finishing write tid 2 to smithi09235267-8 2024-04-26T03:38:51.501 INFO:tasks.rados.rados.0.smithi092.stdout:4: finishing write tid 2 to smithi09235267-4 2024-04-26T03:38:51.501 INFO:tasks.rados.rados.0.smithi092.stdout:6: finishing write tid 2 to smithi09235267-6 2024-04-26T03:38:51.501 INFO:tasks.rados.rados.0.smithi092.stdout:10: finishing write tid 1 to smithi09235267-10 2024-04-26T03:38:51.501 INFO:tasks.rados.rados.0.smithi092.stdout:10: finishing write tid 2 to smithi09235267-10 2024-04-26T03:38:51.501 INFO:tasks.rados.rados.0.smithi092.stdout:17: write initial oid 17 2024-04-26T03:38:51.501 INFO:tasks.rados.rados.0.smithi092.stdout:17: seq_num 16 ranges {0=278528} 2024-04-26T03:38:51.503 INFO:tasks.rados.rados.0.smithi092.stdout:17: writing smithi09235267-17 from 0 to 278528 tid 1 2024-04-26T03:38:51.504 INFO:tasks.rados.rados.0.smithi092.stdout:9: finishing write tid 1 to smithi09235267-9 2024-04-26T03:38:51.504 INFO:tasks.rados.rados.0.smithi092.stdout:9: finishing write tid 2 to smithi09235267-9 2024-04-26T03:38:51.504 INFO:tasks.rados.rados.0.smithi092.stdout:5: finishing write tid 1 to smithi09235267-5 2024-04-26T03:38:51.505 INFO:tasks.rados.rados.0.smithi092.stdout:5: finishing write tid 2 to smithi09235267-5 2024-04-26T03:38:51.505 INFO:tasks.rados.rados.0.smithi092.stdout:8: finishing write tid 3 to smithi09235267-8 2024-04-26T03:38:51.505 INFO:tasks.rados.rados.0.smithi092.stdout:8: oid 8 updating version 0 to 2 2024-04-26T03:38:51.505 INFO:tasks.rados.rados.0.smithi092.stdout:8: oid 8 version 2 is already newer than 1 2024-04-26T03:38:51.505 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 8 v 2 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-04-26T03:38:51.505 INFO:tasks.rados.rados.0.smithi092.stdout:8: left oid 8 (ObjNum 7 snap 0 seq_num 7) 2024-04-26T03:38:51.505 INFO:tasks.rados.rados.0.smithi092.stdout:7: finishing write tid 2 to smithi09235267-7 2024-04-26T03:38:51.505 INFO:tasks.rados.rados.0.smithi092.stdout:8: done (14 left) 2024-04-26T03:38:51.505 INFO:tasks.rados.rados.0.smithi092.stdout:18: write initial oid 18 2024-04-26T03:38:51.505 INFO:tasks.rados.rados.0.smithi092.stdout:18: seq_num 17 ranges {0=294912} 2024-04-26T03:38:51.506 INFO:tasks.rados.rados.0.smithi092.stdout:18: writing smithi09235267-18 from 0 to 294912 tid 1 2024-04-26T03:38:51.506 INFO:tasks.rados.rados.0.smithi092.stdout:19: write initial oid 19 2024-04-26T03:38:51.506 INFO:tasks.rados.rados.0.smithi092.stdout:19: seq_num 18 ranges {0=311296} 2024-04-26T03:38:51.508 INFO:tasks.rados.rados.0.smithi092.stdout:19: writing smithi09235267-19 from 0 to 311296 tid 1 2024-04-26T03:38:51.509 INFO:tasks.rados.rados.0.smithi092.stdout: waiting on 16 2024-04-26T03:38:51.509 INFO:tasks.rados.rados.0.smithi092.stdout:9: finishing write tid 3 to smithi09235267-9 2024-04-26T03:38:51.509 INFO:tasks.rados.rados.0.smithi092.stdout:9: oid 9 updating version 0 to 2 2024-04-26T03:38:51.509 INFO:tasks.rados.rados.0.smithi092.stdout:9: oid 9 version 2 is already newer than 1 2024-04-26T03:38:51.509 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 9 v 2 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-04-26T03:38:51.509 INFO:tasks.rados.rados.0.smithi092.stdout:9: left oid 9 (ObjNum 8 snap 0 seq_num 8) 2024-04-26T03:38:51.509 INFO:tasks.rados.rados.0.smithi092.stdout:3: finishing write tid 3 to smithi09235267-3 2024-04-26T03:38:51.509 INFO:tasks.rados.rados.0.smithi092.stdout:3: oid 3 updating version 0 to 1 2024-04-26T03:38:51.509 INFO:tasks.rados.rados.0.smithi092.stdout:3: oid 3 updating version 1 to 2 2024-04-26T03:38:51.509 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 3 v 2 (ObjNum 2 snap 0 seq_num 2) dirty exists 2024-04-26T03:38:51.509 INFO:tasks.rados.rados.0.smithi092.stdout:3: left oid 3 (ObjNum 2 snap 0 seq_num 2) 2024-04-26T03:38:51.509 INFO:tasks.rados.rados.0.smithi092.stdout:3: done (15 left) 2024-04-26T03:38:51.509 INFO:tasks.rados.rados.0.smithi092.stdout:9: done (14 left) 2024-04-26T03:38:51.510 INFO:tasks.rados.rados.0.smithi092.stdout:20: write initial oid 20 2024-04-26T03:38:51.510 INFO:tasks.rados.rados.0.smithi092.stdout:20: seq_num 19 ranges {0=327680} 2024-04-26T03:38:51.510 INFO:tasks.rados.rados.0.smithi092.stdout:20: writing smithi09235267-20 from 0 to 327680 tid 1 2024-04-26T03:38:51.511 INFO:tasks.rados.rados.0.smithi092.stdout:21: write initial oid 21 2024-04-26T03:38:51.511 INFO:tasks.rados.rados.0.smithi092.stdout:21: seq_num 20 ranges {0=344064} 2024-04-26T03:38:51.514 INFO:tasks.rados.rados.0.smithi092.stdout:21: writing smithi09235267-21 from 0 to 344064 tid 1 2024-04-26T03:38:51.514 INFO:tasks.rados.rados.0.smithi092.stdout: waiting on 16 2024-04-26T03:38:51.514 INFO:tasks.rados.rados.0.smithi092.stdout:5: finishing write tid 3 to smithi09235267-5 2024-04-26T03:38:51.515 INFO:tasks.rados.rados.0.smithi092.stdout:5: oid 5 updating version 0 to 1 2024-04-26T03:38:51.515 INFO:tasks.rados.rados.0.smithi092.stdout:5: oid 5 updating version 1 to 2 2024-04-26T03:38:51.515 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 5 v 2 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-04-26T03:38:51.515 INFO:tasks.rados.rados.0.smithi092.stdout:5: left oid 5 (ObjNum 4 snap 0 seq_num 4) 2024-04-26T03:38:51.515 INFO:tasks.rados.rados.0.smithi092.stdout:14: finishing write tid 1 to smithi09235267-14 2024-04-26T03:38:51.515 INFO:tasks.rados.rados.0.smithi092.stdout:12: finishing write tid 1 to smithi09235267-12 2024-04-26T03:38:51.515 INFO:tasks.rados.rados.0.smithi092.stdout:12: finishing write tid 2 to smithi09235267-12 2024-04-26T03:38:51.515 INFO:tasks.rados.rados.0.smithi092.stdout:6: finishing write tid 3 to smithi09235267-6 2024-04-26T03:38:51.515 INFO:tasks.rados.rados.0.smithi092.stdout:6: oid 6 updating version 0 to 1 2024-04-26T03:38:51.515 INFO:tasks.rados.rados.0.smithi092.stdout:6: oid 6 updating version 1 to 2 2024-04-26T03:38:51.515 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 6 v 2 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-04-26T03:38:51.515 INFO:tasks.rados.rados.0.smithi092.stdout:6: left oid 6 (ObjNum 5 snap 0 seq_num 5) 2024-04-26T03:38:51.515 INFO:tasks.rados.rados.0.smithi092.stdout:7: finishing write tid 3 to smithi09235267-7 2024-04-26T03:38:51.515 INFO:tasks.rados.rados.0.smithi092.stdout:7: oid 7 updating version 0 to 1 2024-04-26T03:38:51.515 INFO:tasks.rados.rados.0.smithi092.stdout:7: oid 7 updating version 1 to 2 2024-04-26T03:38:51.516 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 7 v 2 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-04-26T03:38:51.516 INFO:tasks.rados.rados.0.smithi092.stdout:7: left oid 7 (ObjNum 6 snap 0 seq_num 6) 2024-04-26T03:38:51.516 INFO:tasks.rados.rados.0.smithi092.stdout:13: finishing write tid 1 to smithi09235267-13 2024-04-26T03:38:51.516 INFO:tasks.rados.rados.0.smithi092.stdout:5: done (15 left) 2024-04-26T03:38:51.516 INFO:tasks.rados.rados.0.smithi092.stdout:6: done (14 left) 2024-04-26T03:38:51.516 INFO:tasks.rados.rados.0.smithi092.stdout:7: done (13 left) 2024-04-26T03:38:51.516 INFO:tasks.rados.rados.0.smithi092.stdout:22: write initial oid 22 2024-04-26T03:38:51.516 INFO:tasks.rados.rados.0.smithi092.stdout:22: seq_num 21 ranges {0=360448} 2024-04-26T03:38:51.517 INFO:tasks.rados.rados.0.smithi092.stdout:22: writing smithi09235267-22 from 0 to 360448 tid 1 2024-04-26T03:38:51.517 INFO:tasks.rados.rados.0.smithi092.stdout:23: write initial oid 23 2024-04-26T03:38:51.517 INFO:tasks.rados.rados.0.smithi092.stdout:23: seq_num 22 ranges {0=376832} 2024-04-26T03:38:51.519 INFO:tasks.rados.rados.0.smithi092.stdout:23: writing smithi09235267-23 from 0 to 376832 tid 1 2024-04-26T03:38:51.519 INFO:tasks.rados.rados.0.smithi092.stdout:24: write initial oid 24 2024-04-26T03:38:51.519 INFO:tasks.rados.rados.0.smithi092.stdout:24: seq_num 23 ranges {0=393216} 2024-04-26T03:38:51.522 INFO:tasks.rados.rados.0.smithi092.stdout:24: writing smithi09235267-24 from 0 to 393216 tid 1 2024-04-26T03:38:51.522 INFO:tasks.rados.rados.0.smithi092.stdout: waiting on 16 2024-04-26T03:38:51.522 INFO:tasks.rados.rados.0.smithi092.stdout:15: finishing write tid 1 to smithi09235267-15 2024-04-26T03:38:51.522 INFO:tasks.rados.rados.0.smithi092.stdout:17: finishing write tid 1 to smithi09235267-17 2024-04-26T03:38:51.522 INFO:tasks.rados.rados.0.smithi092.stdout:15: finishing write tid 2 to smithi09235267-15 2024-04-26T03:38:51.522 INFO:tasks.rados.rados.0.smithi092.stdout:17: finishing write tid 2 to smithi09235267-17 2024-04-26T03:38:51.522 INFO:tasks.rados.rados.0.smithi092.stdout:16: finishing write tid 1 to smithi09235267-16 2024-04-26T03:38:51.522 INFO:tasks.rados.rados.0.smithi092.stdout:12: finishing write tid 3 to smithi09235267-12 2024-04-26T03:38:51.522 INFO:tasks.rados.rados.0.smithi092.stdout:12: oid 12 updating version 0 to 1 2024-04-26T03:38:51.523 INFO:tasks.rados.rados.0.smithi092.stdout:12: oid 12 updating version 1 to 2 2024-04-26T03:38:51.523 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 12 v 2 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-04-26T03:38:51.523 INFO:tasks.rados.rados.0.smithi092.stdout:12: left oid 12 (ObjNum 11 snap 0 seq_num 11) 2024-04-26T03:38:51.523 INFO:tasks.rados.rados.0.smithi092.stdout:14: finishing write tid 2 to smithi09235267-14 2024-04-26T03:38:51.523 INFO:tasks.rados.rados.0.smithi092.stdout:19: finishing write tid 1 to smithi09235267-19 2024-04-26T03:38:51.523 INFO:tasks.rados.rados.0.smithi092.stdout:19: finishing write tid 2 to smithi09235267-19 2024-04-26T03:38:51.523 INFO:tasks.rados.rados.0.smithi092.stdout:12: done (15 left) 2024-04-26T03:38:51.523 INFO:tasks.rados.rados.0.smithi092.stdout:25: write initial oid 25 2024-04-26T03:38:51.523 INFO:tasks.rados.rados.0.smithi092.stdout:25: seq_num 24 ranges {0=409600} 2024-04-26T03:38:51.525 INFO:tasks.rados.rados.0.smithi092.stdout:25: writing smithi09235267-25 from 0 to 409600 tid 1 2024-04-26T03:38:51.526 INFO:tasks.rados.rados.0.smithi092.stdout: waiting on 16 2024-04-26T03:38:51.526 INFO:tasks.rados.rados.0.smithi092.stdout:10: finishing write tid 3 to smithi09235267-10 2024-04-26T03:38:51.526 INFO:tasks.rados.rados.0.smithi092.stdout:10: oid 10 updating version 0 to 1 2024-04-26T03:38:51.526 INFO:tasks.rados.rados.0.smithi092.stdout:10: oid 10 updating version 1 to 2 2024-04-26T03:38:51.526 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 10 v 2 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-04-26T03:38:51.526 INFO:tasks.rados.rados.0.smithi092.stdout:10: left oid 10 (ObjNum 9 snap 0 seq_num 9) 2024-04-26T03:38:51.526 INFO:tasks.rados.rados.0.smithi092.stdout:16: finishing write tid 2 to smithi09235267-16 2024-04-26T03:38:51.526 INFO:tasks.rados.rados.0.smithi092.stdout:17: finishing write tid 3 to smithi09235267-17 2024-04-26T03:38:51.526 INFO:tasks.rados.rados.0.smithi092.stdout:17: oid 17 updating version 0 to 4 2024-04-26T03:38:51.526 INFO:tasks.rados.rados.0.smithi092.stdout:17: oid 17 version 4 is already newer than 3 2024-04-26T03:38:51.526 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 17 v 4 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-04-26T03:38:51.526 INFO:tasks.rados.rados.0.smithi092.stdout:17: left oid 17 (ObjNum 16 snap 0 seq_num 16) 2024-04-26T03:38:51.526 INFO:tasks.rados.rados.0.smithi092.stdout:14: finishing write tid 3 to smithi09235267-14 2024-04-26T03:38:51.527 INFO:tasks.rados.rados.0.smithi092.stdout:14: oid 14 updating version 0 to 3 2024-04-26T03:38:51.527 INFO:tasks.rados.rados.0.smithi092.stdout:14: oid 14 updating version 3 to 4 2024-04-26T03:38:51.527 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 14 v 4 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-04-26T03:38:51.527 INFO:tasks.rados.rados.0.smithi092.stdout:14: left oid 14 (ObjNum 13 snap 0 seq_num 13) 2024-04-26T03:38:51.527 INFO:tasks.rados.rados.0.smithi092.stdout:19: finishing write tid 3 to smithi09235267-19 2024-04-26T03:38:51.527 INFO:tasks.rados.rados.0.smithi092.stdout:19: oid 19 updating version 0 to 4 2024-04-26T03:38:51.527 INFO:tasks.rados.rados.0.smithi092.stdout:19: oid 19 version 4 is already newer than 3 2024-04-26T03:38:51.527 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 19 v 4 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-04-26T03:38:51.527 INFO:tasks.rados.rados.0.smithi092.stdout:19: left oid 19 (ObjNum 18 snap 0 seq_num 18) 2024-04-26T03:38:51.527 INFO:tasks.rados.rados.0.smithi092.stdout:16: finishing write tid 3 to smithi09235267-16 2024-04-26T03:38:51.527 INFO:tasks.rados.rados.0.smithi092.stdout:16: oid 16 updating version 0 to 1 2024-04-26T03:38:51.527 INFO:tasks.rados.rados.0.smithi092.stdout:16: oid 16 updating version 1 to 2 2024-04-26T03:38:51.527 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 16 v 2 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-04-26T03:38:51.527 INFO:tasks.rados.rados.0.smithi092.stdout:16: left oid 16 (ObjNum 15 snap 0 seq_num 15) 2024-04-26T03:38:51.528 INFO:tasks.rados.rados.0.smithi092.stdout:11: finishing write tid 1 to smithi09235267-11 2024-04-26T03:38:51.529 INFO:tasks.rados.rados.0.smithi092.stdout:13: finishing write tid 2 to smithi09235267-13 2024-04-26T03:38:51.530 INFO:tasks.rados.rados.0.smithi092.stdout:4: finishing write tid 3 to smithi09235267-4 2024-04-26T03:38:51.530 INFO:tasks.rados.rados.0.smithi092.stdout:4: oid 4 updating version 0 to 1 2024-04-26T03:38:51.530 INFO:tasks.rados.rados.0.smithi092.stdout:4: oid 4 updating version 1 to 2 2024-04-26T03:38:51.530 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 4 v 2 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-04-26T03:38:51.530 INFO:tasks.rados.rados.0.smithi092.stdout:4: left oid 4 (ObjNum 3 snap 0 seq_num 3) 2024-04-26T03:38:51.530 INFO:tasks.rados.rados.0.smithi092.stdout:20: finishing write tid 1 to smithi09235267-20 2024-04-26T03:38:51.530 INFO:tasks.rados.rados.0.smithi092.stdout:20: finishing write tid 2 to smithi09235267-20 2024-04-26T03:38:51.530 INFO:tasks.rados.rados.0.smithi092.stdout:18: finishing write tid 1 to smithi09235267-18 2024-04-26T03:38:51.530 INFO:tasks.rados.rados.0.smithi092.stdout:18: finishing write tid 2 to smithi09235267-18 2024-04-26T03:38:51.530 INFO:tasks.rados.rados.0.smithi092.stdout:11: finishing write tid 2 to smithi09235267-11 2024-04-26T03:38:51.530 INFO:tasks.rados.rados.0.smithi092.stdout:21: finishing write tid 1 to smithi09235267-21 2024-04-26T03:38:51.531 INFO:tasks.rados.rados.0.smithi092.stdout:21: finishing write tid 2 to smithi09235267-21 2024-04-26T03:38:51.531 INFO:tasks.rados.rados.0.smithi092.stdout:4: done (15 left) 2024-04-26T03:38:51.531 INFO:tasks.rados.rados.0.smithi092.stdout:10: done (14 left) 2024-04-26T03:38:51.531 INFO:tasks.rados.rados.0.smithi092.stdout:14: done (13 left) 2024-04-26T03:38:51.531 INFO:tasks.rados.rados.0.smithi092.stdout:16: done (12 left) 2024-04-26T03:38:51.531 INFO:tasks.rados.rados.0.smithi092.stdout:17: done (11 left) 2024-04-26T03:38:51.531 INFO:tasks.rados.rados.0.smithi092.stdout:19: done (10 left) 2024-04-26T03:38:51.531 INFO:tasks.rados.rados.0.smithi092.stdout:26: write initial oid 26 2024-04-26T03:38:51.531 INFO:tasks.rados.rados.0.smithi092.stdout:26: seq_num 25 ranges {0=425984} 2024-04-26T03:38:51.532 INFO:tasks.rados.rados.0.smithi092.stdout:26: writing smithi09235267-26 from 0 to 425984 tid 1 2024-04-26T03:38:51.532 INFO:tasks.rados.rados.0.smithi092.stdout:27: write initial oid 27 2024-04-26T03:38:51.532 INFO:tasks.rados.rados.0.smithi092.stdout:27: seq_num 26 ranges {0=442368} 2024-04-26T03:38:51.532 INFO:tasks.rados.rados.0.smithi092.stdout:27: writing smithi09235267-27 from 0 to 442368 tid 1 2024-04-26T03:38:51.532 INFO:tasks.rados.rados.0.smithi092.stdout:28: write initial oid 28 2024-04-26T03:38:51.532 INFO:tasks.rados.rados.0.smithi092.stdout:28: seq_num 27 ranges {0=458752} 2024-04-26T03:38:51.539 INFO:tasks.rados.rados.0.smithi092.stdout:28: writing smithi09235267-28 from 0 to 458752 tid 1 2024-04-26T03:38:51.539 INFO:tasks.rados.rados.0.smithi092.stdout:13: finishing write tid 3 to smithi09235267-13 2024-04-26T03:38:51.539 INFO:tasks.rados.rados.0.smithi092.stdout:13: oid 13 updating version 0 to 3 2024-04-26T03:38:51.540 INFO:tasks.rados.rados.0.smithi092.stdout:13: oid 13 updating version 3 to 4 2024-04-26T03:38:51.540 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 13 v 4 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-04-26T03:38:51.540 INFO:tasks.rados.rados.0.smithi092.stdout:13: left oid 13 (ObjNum 12 snap 0 seq_num 12) 2024-04-26T03:38:51.540 INFO:tasks.rados.rados.0.smithi092.stdout:21: finishing write tid 3 to smithi09235267-21 2024-04-26T03:38:51.540 INFO:tasks.rados.rados.0.smithi092.stdout:21: oid 21 updating version 0 to 5 2024-04-26T03:38:51.540 INFO:tasks.rados.rados.0.smithi092.stdout:21: oid 21 updating version 5 to 6 2024-04-26T03:38:51.540 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 21 v 6 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-04-26T03:38:51.540 INFO:tasks.rados.rados.0.smithi092.stdout:21: left oid 21 (ObjNum 20 snap 0 seq_num 20) 2024-04-26T03:38:51.540 INFO:tasks.rados.rados.0.smithi092.stdout:20: finishing write tid 3 to smithi09235267-20 2024-04-26T03:38:51.540 INFO:tasks.rados.rados.0.smithi092.stdout:20: oid 20 updating version 0 to 2 2024-04-26T03:38:51.540 INFO:tasks.rados.rados.0.smithi092.stdout:20: oid 20 version 2 is already newer than 1 2024-04-26T03:38:51.540 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 20 v 2 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-04-26T03:38:51.540 INFO:tasks.rados.rados.0.smithi092.stdout:20: left oid 20 (ObjNum 19 snap 0 seq_num 19) 2024-04-26T03:38:51.540 INFO:tasks.rados.rados.0.smithi092.stdout:15: finishing write tid 3 to smithi09235267-15 2024-04-26T03:38:51.541 INFO:tasks.rados.rados.0.smithi092.stdout:15: oid 15 updating version 0 to 2 2024-04-26T03:38:51.541 INFO:tasks.rados.rados.0.smithi092.stdout:15: oid 15 version 2 is already newer than 1 2024-04-26T03:38:51.541 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 15 v 2 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-04-26T03:38:51.541 INFO:tasks.rados.rados.0.smithi092.stdout:15: left oid 15 (ObjNum 14 snap 0 seq_num 14) 2024-04-26T03:38:51.541 INFO:tasks.rados.rados.0.smithi092.stdout:18: finishing write tid 3 to smithi09235267-18 2024-04-26T03:38:51.541 INFO:tasks.rados.rados.0.smithi092.stdout:18: oid 18 updating version 0 to 3 2024-04-26T03:38:51.541 INFO:tasks.rados.rados.0.smithi092.stdout:18: oid 18 updating version 3 to 4 2024-04-26T03:38:51.541 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 18 v 4 (ObjNum 17 snap 0 seq_num 17) dirty exists 2024-04-26T03:38:51.541 INFO:tasks.rados.rados.0.smithi092.stdout:18: left oid 18 (ObjNum 17 snap 0 seq_num 17) 2024-04-26T03:38:51.541 INFO:tasks.rados.rados.0.smithi092.stdout:11: finishing write tid 3 to smithi09235267-11 2024-04-26T03:38:51.541 INFO:tasks.rados.rados.0.smithi092.stdout:11: oid 11 updating version 0 to 4 2024-04-26T03:38:51.541 INFO:tasks.rados.rados.0.smithi092.stdout:11: oid 11 version 4 is already newer than 3 2024-04-26T03:38:51.541 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 11 v 4 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-04-26T03:38:51.541 INFO:tasks.rados.rados.0.smithi092.stdout:11: left oid 11 (ObjNum 10 snap 0 seq_num 10) 2024-04-26T03:38:51.541 INFO:tasks.rados.rados.0.smithi092.stdout:24: finishing write tid 1 to smithi09235267-24 2024-04-26T03:38:51.542 INFO:tasks.rados.rados.0.smithi092.stdout:24: finishing write tid 2 to smithi09235267-24 2024-04-26T03:38:51.542 INFO:tasks.rados.rados.0.smithi092.stdout:11: done (12 left) 2024-04-26T03:38:51.542 INFO:tasks.rados.rados.0.smithi092.stdout:13: done (11 left) 2024-04-26T03:38:51.542 INFO:tasks.rados.rados.0.smithi092.stdout:15: done (10 left) 2024-04-26T03:38:51.542 INFO:tasks.rados.rados.0.smithi092.stdout:18: done (9 left) 2024-04-26T03:38:51.542 INFO:tasks.rados.rados.0.smithi092.stdout:20: done (8 left) 2024-04-26T03:38:51.542 INFO:tasks.rados.rados.0.smithi092.stdout:21: done (7 left) 2024-04-26T03:38:51.542 INFO:tasks.rados.rados.0.smithi092.stdout:29: write initial oid 29 2024-04-26T03:38:51.542 INFO:tasks.rados.rados.0.smithi092.stdout:29: seq_num 28 ranges {0=475136} 2024-04-26T03:38:51.543 INFO:tasks.rados.rados.0.smithi092.stdout:29: writing smithi09235267-29 from 0 to 475136 tid 1 2024-04-26T03:38:51.543 INFO:tasks.rados.rados.0.smithi092.stdout:30: write initial oid 30 2024-04-26T03:38:51.543 INFO:tasks.rados.rados.0.smithi092.stdout:30: seq_num 29 ranges {0=491520} 2024-04-26T03:38:51.544 INFO:tasks.rados.rados.0.smithi092.stdout:30: writing smithi09235267-30 from 0 to 491520 tid 1 2024-04-26T03:38:51.544 INFO:tasks.rados.rados.0.smithi092.stdout:31: write initial oid 31 2024-04-26T03:38:51.544 INFO:tasks.rados.rados.0.smithi092.stdout:31: seq_num 30 ranges {0=507904} 2024-04-26T03:38:51.547 INFO:tasks.rados.rados.0.smithi092.stdout:31: writing smithi09235267-31 from 0 to 507904 tid 1 2024-04-26T03:38:51.547 INFO:tasks.rados.rados.0.smithi092.stdout:32: write initial oid 32 2024-04-26T03:38:51.547 INFO:tasks.rados.rados.0.smithi092.stdout:32: seq_num 31 ranges {0=524288} 2024-04-26T03:38:51.551 INFO:tasks.rados.rados.0.smithi092.stdout:32: writing smithi09235267-32 from 0 to 524288 tid 1 2024-04-26T03:38:51.551 INFO:tasks.rados.rados.0.smithi092.stdout:33: write initial oid 33 2024-04-26T03:38:51.551 INFO:tasks.rados.rados.0.smithi092.stdout:33: seq_num 32 ranges {0=540672} 2024-04-26T03:38:51.554 INFO:tasks.ceph.osd.12.smithi028.stderr:2024-04-26T03:38:51.553+0000 7f114fe14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 31967) UID: 0 2024-04-26T03:38:51.555 INFO:tasks.rados.rados.0.smithi092.stdout:33: writing smithi09235267-33 from 0 to 540672 tid 1 2024-04-26T03:38:51.555 INFO:tasks.rados.rados.0.smithi092.stdout:34: write initial oid 34 2024-04-26T03:38:51.555 INFO:tasks.rados.rados.0.smithi092.stdout:34: seq_num 33 ranges {0=557056} 2024-04-26T03:38:51.559 INFO:tasks.rados.rados.0.smithi092.stdout:34: writing smithi09235267-34 from 0 to 557056 tid 1 2024-04-26T03:38:51.559 INFO:tasks.rados.rados.0.smithi092.stdout:35: write initial oid 35 2024-04-26T03:38:51.559 INFO:tasks.rados.rados.0.smithi092.stdout:35: seq_num 34 ranges {0=573440} 2024-04-26T03:38:51.562 INFO:tasks.rados.rados.0.smithi092.stdout:35: writing smithi09235267-35 from 0 to 573440 tid 1 2024-04-26T03:38:51.563 INFO:tasks.rados.rados.0.smithi092.stdout:36: write initial oid 36 2024-04-26T03:38:51.563 INFO:tasks.rados.rados.0.smithi092.stdout:36: seq_num 35 ranges {0=589824} 2024-04-26T03:38:51.566 INFO:tasks.rados.rados.0.smithi092.stdout:36: writing smithi09235267-36 from 0 to 589824 tid 1 2024-04-26T03:38:51.567 INFO:tasks.rados.rados.0.smithi092.stdout:37: write initial oid 37 2024-04-26T03:38:51.567 INFO:tasks.rados.rados.0.smithi092.stdout:37: seq_num 36 ranges {0=606208} 2024-04-26T03:38:51.570 INFO:tasks.rados.rados.0.smithi092.stdout:37: writing smithi09235267-37 from 0 to 606208 tid 1 2024-04-26T03:38:51.570 INFO:tasks.rados.rados.0.smithi092.stdout: waiting on 16 2024-04-26T03:38:51.571 INFO:tasks.rados.rados.0.smithi092.stdout:24: finishing write tid 3 to smithi09235267-24 2024-04-26T03:38:51.571 INFO:tasks.rados.rados.0.smithi092.stdout:24: oid 24 updating version 0 to 3 2024-04-26T03:38:51.571 INFO:tasks.rados.rados.0.smithi092.stdout:24: oid 24 updating version 3 to 4 2024-04-26T03:38:51.571 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 24 v 4 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-04-26T03:38:51.571 INFO:tasks.rados.rados.0.smithi092.stdout:24: left oid 24 (ObjNum 23 snap 0 seq_num 23) 2024-04-26T03:38:51.571 INFO:tasks.rados.rados.0.smithi092.stdout:24: done (15 left) 2024-04-26T03:38:51.571 INFO:tasks.rados.rados.0.smithi092.stdout:38: write initial oid 38 2024-04-26T03:38:51.571 INFO:tasks.rados.rados.0.smithi092.stdout:38: seq_num 37 ranges {0=622592} 2024-04-26T03:38:51.574 INFO:tasks.rados.rados.0.smithi092.stdout:38: writing smithi09235267-38 from 0 to 622592 tid 1 2024-04-26T03:38:51.574 INFO:tasks.rados.rados.0.smithi092.stdout: waiting on 16 2024-04-26T03:38:51.574 INFO:tasks.rados.rados.0.smithi092.stdout:23: finishing write tid 1 to smithi09235267-23 2024-04-26T03:38:51.574 INFO:tasks.rados.rados.0.smithi092.stdout:23: finishing write tid 2 to smithi09235267-23 2024-04-26T03:38:51.575 INFO:tasks.rados.rados.0.smithi092.stdout:26: finishing write tid 1 to smithi09235267-26 2024-04-26T03:38:51.575 INFO:tasks.rados.rados.0.smithi092.stdout:26: finishing write tid 2 to smithi09235267-26 2024-04-26T03:38:51.575 INFO:tasks.rados.rados.0.smithi092.stdout:23: finishing write tid 3 to smithi09235267-23 2024-04-26T03:38:51.575 INFO:tasks.rados.rados.0.smithi092.stdout:23: oid 23 updating version 0 to 4 2024-04-26T03:38:51.575 INFO:tasks.rados.rados.0.smithi092.stdout:23: oid 23 version 4 is already newer than 3 2024-04-26T03:38:51.575 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 23 v 4 (ObjNum 22 snap 0 seq_num 22) dirty exists 2024-04-26T03:38:51.575 INFO:tasks.rados.rados.0.smithi092.stdout:23: left oid 23 (ObjNum 22 snap 0 seq_num 22) 2024-04-26T03:38:51.575 INFO:tasks.rados.rados.0.smithi092.stdout:25: finishing write tid 1 to smithi09235267-25 2024-04-26T03:38:51.575 INFO:tasks.rados.rados.0.smithi092.stdout:25: finishing write tid 2 to smithi09235267-25 2024-04-26T03:38:51.576 INFO:tasks.rados.rados.0.smithi092.stdout:23: done (15 left) 2024-04-26T03:38:51.576 INFO:tasks.rados.rados.0.smithi092.stdout:39: write initial oid 39 2024-04-26T03:38:51.576 INFO:tasks.rados.rados.0.smithi092.stdout:39: seq_num 38 ranges {0=638976} 2024-04-26T03:38:51.580 INFO:tasks.rados.rados.0.smithi092.stdout:39: writing smithi09235267-39 from 0 to 638976 tid 1 2024-04-26T03:38:51.581 INFO:tasks.rados.rados.0.smithi092.stdout: waiting on 16 2024-04-26T03:38:51.581 INFO:tasks.rados.rados.0.smithi092.stdout:22: finishing write tid 1 to smithi09235267-22 2024-04-26T03:38:51.581 INFO:tasks.rados.rados.0.smithi092.stdout:22: finishing write tid 2 to smithi09235267-22 2024-04-26T03:38:51.581 INFO:tasks.rados.rados.0.smithi092.stdout:26: finishing write tid 3 to smithi09235267-26 2024-04-26T03:38:51.581 INFO:tasks.rados.rados.0.smithi092.stdout:26: oid 26 updating version 0 to 3 2024-04-26T03:38:51.581 INFO:tasks.rados.rados.0.smithi092.stdout:26: oid 26 updating version 3 to 4 2024-04-26T03:38:51.581 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 26 v 4 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-04-26T03:38:51.581 INFO:tasks.rados.rados.0.smithi092.stdout:26: left oid 26 (ObjNum 25 snap 0 seq_num 25) 2024-04-26T03:38:51.581 INFO:tasks.rados.rados.0.smithi092.stdout:22: finishing write tid 3 to smithi09235267-22 2024-04-26T03:38:51.581 INFO:tasks.rados.rados.0.smithi092.stdout:22: oid 22 updating version 0 to 5 2024-04-26T03:38:51.581 INFO:tasks.rados.rados.0.smithi092.stdout:22: oid 22 updating version 5 to 6 2024-04-26T03:38:51.581 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 22 v 6 (ObjNum 21 snap 0 seq_num 21) dirty exists 2024-04-26T03:38:51.582 INFO:tasks.rados.rados.0.smithi092.stdout:22: left oid 22 (ObjNum 21 snap 0 seq_num 21) 2024-04-26T03:38:51.582 INFO:tasks.rados.rados.0.smithi092.stdout:25: finishing write tid 3 to smithi09235267-25 2024-04-26T03:38:51.582 INFO:tasks.rados.rados.0.smithi092.stdout:25: oid 25 updating version 0 to 4 2024-04-26T03:38:51.582 INFO:tasks.rados.rados.0.smithi092.stdout:25: oid 25 version 4 is already newer than 3 2024-04-26T03:38:51.582 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 25 v 4 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-04-26T03:38:51.582 INFO:tasks.rados.rados.0.smithi092.stdout:25: left oid 25 (ObjNum 24 snap 0 seq_num 24) 2024-04-26T03:38:51.583 INFO:tasks.rados.rados.0.smithi092.stdout:27: finishing write tid 1 to smithi09235267-27 2024-04-26T03:38:51.583 INFO:tasks.rados.rados.0.smithi092.stdout:28: finishing write tid 1 to smithi09235267-28 2024-04-26T03:38:51.583 INFO:tasks.rados.rados.0.smithi092.stdout:27: finishing write tid 2 to smithi09235267-27 2024-04-26T03:38:51.583 INFO:tasks.rados.rados.0.smithi092.stdout:27: finishing write tid 3 to smithi09235267-27 2024-04-26T03:38:51.583 INFO:tasks.rados.rados.0.smithi092.stdout:27: oid 27 updating version 0 to 4 2024-04-26T03:38:51.583 INFO:tasks.rados.rados.0.smithi092.stdout:27: oid 27 version 4 is already newer than 3 2024-04-26T03:38:51.583 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 27 v 4 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-04-26T03:38:51.583 INFO:tasks.rados.rados.0.smithi092.stdout:27: left oid 27 (ObjNum 26 snap 0 seq_num 26) 2024-04-26T03:38:51.583 INFO:tasks.rados.rados.0.smithi092.stdout:28: finishing write tid 2 to smithi09235267-28 2024-04-26T03:38:51.583 INFO:tasks.rados.rados.0.smithi092.stdout:29: finishing write tid 1 to smithi09235267-29 2024-04-26T03:38:51.583 INFO:tasks.rados.rados.0.smithi092.stdout:29: finishing write tid 2 to smithi09235267-29 2024-04-26T03:38:51.583 INFO:tasks.rados.rados.0.smithi092.stdout:28: finishing write tid 3 to smithi09235267-28 2024-04-26T03:38:51.583 INFO:tasks.rados.rados.0.smithi092.stdout:28: oid 28 updating version 0 to 6 2024-04-26T03:38:51.584 INFO:tasks.rados.rados.0.smithi092.stdout:28: oid 28 version 6 is already newer than 5 2024-04-26T03:38:51.584 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 28 v 6 (ObjNum 27 snap 0 seq_num 27) dirty exists 2024-04-26T03:38:51.584 INFO:tasks.rados.rados.0.smithi092.stdout:28: left oid 28 (ObjNum 27 snap 0 seq_num 27) 2024-04-26T03:38:51.584 INFO:tasks.rados.rados.0.smithi092.stdout:29: finishing write tid 3 to smithi09235267-29 2024-04-26T03:38:51.584 INFO:tasks.rados.rados.0.smithi092.stdout:29: oid 29 updating version 0 to 6 2024-04-26T03:38:51.584 INFO:tasks.rados.rados.0.smithi092.stdout:29: oid 29 version 6 is already newer than 5 2024-04-26T03:38:51.584 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 29 v 6 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-04-26T03:38:51.584 INFO:tasks.rados.rados.0.smithi092.stdout:29: left oid 29 (ObjNum 28 snap 0 seq_num 28) 2024-04-26T03:38:51.584 INFO:tasks.rados.rados.0.smithi092.stdout:31: finishing write tid 1 to smithi09235267-31 2024-04-26T03:38:51.584 INFO:tasks.rados.rados.0.smithi092.stdout:31: finishing write tid 2 to smithi09235267-31 2024-04-26T03:38:51.584 INFO:tasks.rados.rados.0.smithi092.stdout:31: finishing write tid 3 to smithi09235267-31 2024-04-26T03:38:51.584 INFO:tasks.rados.rados.0.smithi092.stdout:31: oid 31 updating version 0 to 5 2024-04-26T03:38:51.585 INFO:tasks.rados.rados.0.smithi092.stdout:31: oid 31 updating version 5 to 6 2024-04-26T03:38:51.585 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 31 v 6 (ObjNum 30 snap 0 seq_num 30) dirty exists 2024-04-26T03:38:51.585 INFO:tasks.rados.rados.0.smithi092.stdout:31: left oid 31 (ObjNum 30 snap 0 seq_num 30) 2024-04-26T03:38:51.585 INFO:tasks.rados.rados.0.smithi092.stdout:30: finishing write tid 1 to smithi09235267-30 2024-04-26T03:38:51.585 INFO:tasks.rados.rados.0.smithi092.stdout:30: finishing write tid 2 to smithi09235267-30 2024-04-26T03:38:51.585 INFO:tasks.rados.rados.0.smithi092.stdout:32: finishing write tid 1 to smithi09235267-32 2024-04-26T03:38:51.585 INFO:tasks.rados.rados.0.smithi092.stdout:32: finishing write tid 2 to smithi09235267-32 2024-04-26T03:38:51.585 INFO:tasks.rados.rados.0.smithi092.stdout:30: finishing write tid 3 to smithi09235267-30 2024-04-26T03:38:51.585 INFO:tasks.rados.rados.0.smithi092.stdout:30: oid 30 updating version 0 to 3 2024-04-26T03:38:51.585 INFO:tasks.rados.rados.0.smithi092.stdout:30: oid 30 updating version 3 to 4 2024-04-26T03:38:51.585 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 30 v 4 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-04-26T03:38:51.586 INFO:tasks.rados.rados.0.smithi092.stdout:30: left oid 30 (ObjNum 29 snap 0 seq_num 29) 2024-04-26T03:38:51.586 INFO:tasks.rados.rados.0.smithi092.stdout:32: finishing write tid 3 to smithi09235267-32 2024-04-26T03:38:51.586 INFO:tasks.rados.rados.0.smithi092.stdout:32: oid 32 updating version 0 to 6 2024-04-26T03:38:51.586 INFO:tasks.rados.rados.0.smithi092.stdout:32: oid 32 version 6 is already newer than 5 2024-04-26T03:38:51.586 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 32 v 6 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-04-26T03:38:51.586 INFO:tasks.rados.rados.0.smithi092.stdout:32: left oid 32 (ObjNum 31 snap 0 seq_num 31) 2024-04-26T03:38:51.586 INFO:tasks.rados.rados.0.smithi092.stdout:33: finishing write tid 1 to smithi09235267-33 2024-04-26T03:38:51.586 INFO:tasks.rados.rados.0.smithi092.stdout:33: finishing write tid 2 to smithi09235267-33 2024-04-26T03:38:51.586 INFO:tasks.rados.rados.0.smithi092.stdout:33: finishing write tid 3 to smithi09235267-33 2024-04-26T03:38:51.586 INFO:tasks.rados.rados.0.smithi092.stdout:33: oid 33 updating version 0 to 6 2024-04-26T03:38:51.586 INFO:tasks.rados.rados.0.smithi092.stdout:33: oid 33 version 6 is already newer than 5 2024-04-26T03:38:51.586 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 33 v 6 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-04-26T03:38:51.586 INFO:tasks.rados.rados.0.smithi092.stdout:33: left oid 33 (ObjNum 32 snap 0 seq_num 32) 2024-04-26T03:38:51.587 INFO:tasks.rados.rados.0.smithi092.stdout:34: finishing write tid 1 to smithi09235267-34 2024-04-26T03:38:51.587 INFO:tasks.rados.rados.0.smithi092.stdout:34: finishing write tid 2 to smithi09235267-34 2024-04-26T03:38:51.587 INFO:tasks.rados.rados.0.smithi092.stdout:36: finishing write tid 1 to smithi09235267-36 2024-04-26T03:38:51.587 INFO:tasks.rados.rados.0.smithi092.stdout:36: finishing write tid 2 to smithi09235267-36 2024-04-26T03:38:51.587 INFO:tasks.rados.rados.0.smithi092.stdout:35: finishing write tid 1 to smithi09235267-35 2024-04-26T03:38:51.587 INFO:tasks.rados.rados.0.smithi092.stdout:35: finishing write tid 2 to smithi09235267-35 2024-04-26T03:38:51.587 INFO:tasks.rados.rados.0.smithi092.stdout:34: finishing write tid 3 to smithi09235267-34 2024-04-26T03:38:51.587 INFO:tasks.rados.rados.0.smithi092.stdout:34: oid 34 updating version 0 to 8 2024-04-26T03:38:51.587 INFO:tasks.rados.rados.0.smithi092.stdout:34: oid 34 version 8 is already newer than 7 2024-04-26T03:38:51.588 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 34 v 8 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-04-26T03:38:51.588 INFO:tasks.rados.rados.0.smithi092.stdout:34: left oid 34 (ObjNum 33 snap 0 seq_num 33) 2024-04-26T03:38:51.588 INFO:tasks.rados.rados.0.smithi092.stdout:36: finishing write tid 3 to smithi09235267-36 2024-04-26T03:38:51.588 INFO:tasks.rados.rados.0.smithi092.stdout:36: oid 36 updating version 0 to 10 2024-04-26T03:38:51.588 INFO:tasks.rados.rados.0.smithi092.stdout:36: oid 36 version 10 is already newer than 9 2024-04-26T03:38:51.588 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 36 v 10 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-04-26T03:38:51.588 INFO:tasks.rados.rados.0.smithi092.stdout:36: left oid 36 (ObjNum 35 snap 0 seq_num 35) 2024-04-26T03:38:51.588 INFO:tasks.rados.rados.0.smithi092.stdout:22: done (15 left) 2024-04-26T03:38:51.588 INFO:tasks.rados.rados.0.smithi092.stdout:25: done (14 left) 2024-04-26T03:38:51.588 INFO:tasks.rados.rados.0.smithi092.stdout:26: done (13 left) 2024-04-26T03:38:51.588 INFO:tasks.rados.rados.0.smithi092.stdout:27: done (12 left) 2024-04-26T03:38:51.588 INFO:tasks.rados.rados.0.smithi092.stdout:28: done (11 left) 2024-04-26T03:38:51.588 INFO:tasks.rados.rados.0.smithi092.stdout:29: done (10 left) 2024-04-26T03:38:51.588 INFO:tasks.rados.rados.0.smithi092.stdout:30: done (9 left) 2024-04-26T03:38:51.588 INFO:tasks.rados.rados.0.smithi092.stdout:31: done (8 left) 2024-04-26T03:38:51.589 INFO:tasks.rados.rados.0.smithi092.stdout:32: done (7 left) 2024-04-26T03:38:51.589 INFO:tasks.rados.rados.0.smithi092.stdout:33: done (6 left) 2024-04-26T03:38:51.589 INFO:tasks.rados.rados.0.smithi092.stdout:34: done (5 left) 2024-04-26T03:38:51.589 INFO:tasks.rados.rados.0.smithi092.stdout:36: done (4 left) 2024-04-26T03:38:51.589 INFO:tasks.rados.rados.0.smithi092.stdout:40: write initial oid 40 2024-04-26T03:38:51.589 INFO:tasks.rados.rados.0.smithi092.stdout:40: seq_num 39 ranges {0=671744} 2024-04-26T03:38:51.590 INFO:tasks.rados.rados.0.smithi092.stdout:40: writing smithi09235267-40 from 0 to 671744 tid 1 2024-04-26T03:38:51.590 INFO:tasks.rados.rados.0.smithi092.stdout:41: write initial oid 41 2024-04-26T03:38:51.590 INFO:tasks.rados.rados.0.smithi092.stdout:41: seq_num 40 ranges {0=688128} 2024-04-26T03:38:51.590 INFO:tasks.rados.rados.0.smithi092.stdout:41: writing smithi09235267-41 from 0 to 688128 tid 1 2024-04-26T03:38:51.590 INFO:tasks.rados.rados.0.smithi092.stdout:42: write initial oid 42 2024-04-26T03:38:51.590 INFO:tasks.rados.rados.0.smithi092.stdout:42: seq_num 41 ranges {0=704512} 2024-04-26T03:38:51.594 INFO:tasks.rados.rados.0.smithi092.stdout:42: writing smithi09235267-42 from 0 to 704512 tid 1 2024-04-26T03:38:51.594 INFO:tasks.rados.rados.0.smithi092.stdout:43: write initial oid 43 2024-04-26T03:38:51.594 INFO:tasks.rados.rados.0.smithi092.stdout:43: seq_num 42 ranges {0=720896} 2024-04-26T03:38:51.599 INFO:tasks.rados.rados.0.smithi092.stdout:43: writing smithi09235267-43 from 0 to 720896 tid 1 2024-04-26T03:38:51.600 INFO:tasks.rados.rados.0.smithi092.stdout:44: write initial oid 44 2024-04-26T03:38:51.600 INFO:tasks.rados.rados.0.smithi092.stdout:44: seq_num 43 ranges {0=737280} 2024-04-26T03:38:51.604 INFO:tasks.rados.rados.0.smithi092.stdout:44: writing smithi09235267-44 from 0 to 737280 tid 1 2024-04-26T03:38:51.604 INFO:tasks.rados.rados.0.smithi092.stdout:45: write initial oid 45 2024-04-26T03:38:51.604 INFO:tasks.rados.rados.0.smithi092.stdout:45: seq_num 44 ranges {0=753664} 2024-04-26T03:38:51.606 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0: {} 2024-04-26T03:38:51.607 INFO:teuthology.orchestra.run.smithi028.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T03:38:51.609 INFO:tasks.rados.rados.0.smithi092.stdout:45: writing smithi09235267-45 from 0 to 753664 tid 1 2024-04-26T03:38:51.609 INFO:tasks.rados.rados.0.smithi092.stdout:46: write initial oid 46 2024-04-26T03:38:51.609 INFO:tasks.rados.rados.0.smithi092.stdout:46: seq_num 45 ranges {0=770048} 2024-04-26T03:38:51.614 INFO:tasks.rados.rados.0.smithi092.stdout:46: writing smithi09235267-46 from 0 to 770048 tid 1 2024-04-26T03:38:51.614 INFO:tasks.rados.rados.0.smithi092.stdout:47: write initial oid 47 2024-04-26T03:38:51.615 INFO:tasks.rados.rados.0.smithi092.stdout:47: seq_num 46 ranges {0=786432} 2024-04-26T03:38:51.618 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:38:51.618 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-26T03:38:51.619 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-26T03:38:51.619 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-26T03:38:51.619 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:38:51.619 INFO:tasks.rados.rados.0.smithi092.stdout:47: writing smithi09235267-47 from 0 to 786432 tid 1 2024-04-26T03:38:51.619 INFO:tasks.rados.rados.0.smithi092.stdout:48: write initial oid 48 2024-04-26T03:38:51.619 INFO:tasks.rados.rados.0.smithi092.stdout:48: seq_num 47 ranges {0=425984,425984=376832} 2024-04-26T03:38:51.622 INFO:tasks.rados.rados.0.smithi092.stdout:48: writing smithi09235267-48 from 0 to 425984 tid 1 2024-04-26T03:38:51.625 INFO:tasks.rados.rados.0.smithi092.stdout:48: writing smithi09235267-48 from 425984 to 802816 tid 2 2024-04-26T03:38:51.625 INFO:tasks.rados.rados.0.smithi092.stdout:49: write initial oid 49 2024-04-26T03:38:51.625 INFO:tasks.rados.rados.0.smithi092.stdout:49: seq_num 48 ranges {0=442368,442368=376832} 2024-04-26T03:38:51.628 INFO:tasks.rados.rados.0.smithi092.stdout:49: writing smithi09235267-49 from 0 to 442368 tid 1 2024-04-26T03:38:51.631 INFO:tasks.rados.rados.0.smithi092.stdout:49: writing smithi09235267-49 from 442368 to 819200 tid 2 2024-04-26T03:38:51.631 INFO:tasks.rados.rados.0.smithi092.stdout:50: write initial oid 50 2024-04-26T03:38:51.632 INFO:tasks.rados.rados.0.smithi092.stdout:50: seq_num 49 ranges {0=458752,458752=376832} 2024-04-26T03:38:51.632 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-26T03:38:51.633 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1: {} 2024-04-26T03:38:51.633 INFO:teuthology.orchestra.run.smithi028.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T03:38:51.634 INFO:tasks.rados.rados.0.smithi092.stdout:50: writing smithi09235267-50 from 0 to 458752 tid 1 2024-04-26T03:38:51.638 INFO:tasks.rados.rados.0.smithi092.stdout:50: writing smithi09235267-50 from 458752 to 835584 tid 2 2024-04-26T03:38:51.638 INFO:tasks.rados.rados.0.smithi092.stdout:51: copy_from oid 14 from oid 8 current snap is 0 2024-04-26T03:38:51.639 INFO:tasks.rados.rados.0.smithi092.stdout:37: finishing write tid 1 to smithi09235267-37 2024-04-26T03:38:51.639 INFO:tasks.rados.rados.0.smithi092.stdout:37: finishing write tid 2 to smithi09235267-37 2024-04-26T03:38:51.639 INFO:tasks.rados.rados.0.smithi092.stdout:38: finishing write tid 1 to smithi09235267-38 2024-04-26T03:38:51.639 INFO:tasks.rados.rados.0.smithi092.stdout:38: finishing write tid 2 to smithi09235267-38 2024-04-26T03:38:51.639 INFO:tasks.rados.rados.0.smithi092.stdout:37: finishing write tid 3 to smithi09235267-37 2024-04-26T03:38:51.639 INFO:tasks.rados.rados.0.smithi092.stdout:37: oid 37 updating version 0 to 8 2024-04-26T03:38:51.639 INFO:tasks.rados.rados.0.smithi092.stdout:37: oid 37 version 8 is already newer than 7 2024-04-26T03:38:51.639 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 37 v 8 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-04-26T03:38:51.639 INFO:tasks.rados.rados.0.smithi092.stdout:37: left oid 37 (ObjNum 36 snap 0 seq_num 36) 2024-04-26T03:38:51.639 INFO:tasks.rados.rados.0.smithi092.stdout:35: finishing write tid 3 to smithi09235267-35 2024-04-26T03:38:51.639 INFO:tasks.rados.rados.0.smithi092.stdout:35: oid 35 updating version 0 to 6 2024-04-26T03:38:51.639 INFO:tasks.rados.rados.0.smithi092.stdout:35: oid 35 version 6 is already newer than 5 2024-04-26T03:38:51.639 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 35 v 6 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-04-26T03:38:51.639 INFO:tasks.rados.rados.0.smithi092.stdout:35: left oid 35 (ObjNum 34 snap 0 seq_num 34) 2024-04-26T03:38:51.639 INFO:tasks.rados.rados.0.smithi092.stdout:38: finishing write tid 3 to smithi09235267-38 2024-04-26T03:38:51.640 INFO:tasks.rados.rados.0.smithi092.stdout:38: oid 38 updating version 0 to 3 2024-04-26T03:38:51.640 INFO:tasks.rados.rados.0.smithi092.stdout:38: oid 38 updating version 3 to 4 2024-04-26T03:38:51.640 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 38 v 4 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-04-26T03:38:51.640 INFO:tasks.rados.rados.0.smithi092.stdout:38: left oid 38 (ObjNum 37 snap 0 seq_num 37) 2024-04-26T03:38:51.640 INFO:tasks.rados.rados.0.smithi092.stdout:39: finishing write tid 1 to smithi09235267-39 2024-04-26T03:38:51.640 INFO:tasks.rados.rados.0.smithi092.stdout:39: finishing write tid 2 to smithi09235267-39 2024-04-26T03:38:51.640 INFO:tasks.rados.rados.0.smithi092.stdout:40: finishing write tid 1 to smithi09235267-40 2024-04-26T03:38:51.640 INFO:tasks.rados.rados.0.smithi092.stdout:40: finishing write tid 2 to smithi09235267-40 2024-04-26T03:38:51.640 INFO:tasks.rados.rados.0.smithi092.stdout:40: finishing write tid 3 to smithi09235267-40 2024-04-26T03:38:51.640 INFO:tasks.rados.rados.0.smithi092.stdout:40: oid 40 updating version 0 to 2 2024-04-26T03:38:51.640 INFO:tasks.rados.rados.0.smithi092.stdout:40: oid 40 version 2 is already newer than 1 2024-04-26T03:38:51.640 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 40 v 2 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-04-26T03:38:51.640 INFO:tasks.rados.rados.0.smithi092.stdout:40: left oid 40 (ObjNum 39 snap 0 seq_num 39) 2024-04-26T03:38:51.640 INFO:tasks.rados.rados.0.smithi092.stdout:39: finishing write tid 3 to smithi09235267-39 2024-04-26T03:38:51.640 INFO:tasks.rados.rados.0.smithi092.stdout:39: oid 39 updating version 0 to 7 2024-04-26T03:38:51.641 INFO:tasks.rados.rados.0.smithi092.stdout:39: oid 39 updating version 7 to 8 2024-04-26T03:38:51.641 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 39 v 8 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-04-26T03:38:51.641 INFO:tasks.rados.rados.0.smithi092.stdout:39: left oid 39 (ObjNum 38 snap 0 seq_num 38) 2024-04-26T03:38:51.641 INFO:tasks.rados.rados.0.smithi092.stdout:41: finishing write tid 1 to smithi09235267-41 2024-04-26T03:38:51.641 INFO:tasks.rados.rados.0.smithi092.stdout:41: finishing write tid 2 to smithi09235267-41 2024-04-26T03:38:51.641 INFO:tasks.rados.rados.0.smithi092.stdout:41: finishing write tid 3 to smithi09235267-41 2024-04-26T03:38:51.641 INFO:tasks.rados.rados.0.smithi092.stdout:41: oid 41 updating version 0 to 9 2024-04-26T03:38:51.641 INFO:tasks.rados.rados.0.smithi092.stdout:41: oid 41 updating version 9 to 10 2024-04-26T03:38:51.641 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 41 v 10 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-04-26T03:38:51.641 INFO:tasks.rados.rados.0.smithi092.stdout:41: left oid 41 (ObjNum 40 snap 0 seq_num 40) 2024-04-26T03:38:51.641 INFO:tasks.rados.rados.0.smithi092.stdout:42: finishing write tid 1 to smithi09235267-42 2024-04-26T03:38:51.641 INFO:tasks.rados.rados.0.smithi092.stdout:42: finishing write tid 2 to smithi09235267-42 2024-04-26T03:38:51.641 INFO:tasks.rados.rados.0.smithi092.stdout:42: finishing write tid 3 to smithi09235267-42 2024-04-26T03:38:51.641 INFO:tasks.rados.rados.0.smithi092.stdout:42: oid 42 updating version 0 to 7 2024-04-26T03:38:51.641 INFO:tasks.rados.rados.0.smithi092.stdout:42: oid 42 updating version 7 to 8 2024-04-26T03:38:51.642 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 42 v 8 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-04-26T03:38:51.642 INFO:tasks.rados.rados.0.smithi092.stdout:42: left oid 42 (ObjNum 41 snap 0 seq_num 41) 2024-04-26T03:38:51.642 INFO:tasks.rados.rados.0.smithi092.stdout:43: finishing write tid 1 to smithi09235267-43 2024-04-26T03:38:51.642 INFO:tasks.rados.rados.0.smithi092.stdout:35: done (15 left) 2024-04-26T03:38:51.642 INFO:tasks.rados.rados.0.smithi092.stdout:37: done (14 left) 2024-04-26T03:38:51.642 INFO:tasks.rados.rados.0.smithi092.stdout:38: done (13 left) 2024-04-26T03:38:51.642 INFO:tasks.rados.rados.0.smithi092.stdout:39: done (12 left) 2024-04-26T03:38:51.642 INFO:tasks.rados.rados.0.smithi092.stdout:40: done (11 left) 2024-04-26T03:38:51.642 INFO:tasks.rados.rados.0.smithi092.stdout:41: done (10 left) 2024-04-26T03:38:51.642 INFO:tasks.rados.rados.0.smithi092.stdout:42: done (9 left) 2024-04-26T03:38:51.642 INFO:tasks.rados.rados.0.smithi092.stdout:52: read oid 42 snap -1 2024-04-26T03:38:51.642 INFO:tasks.rados.rados.0.smithi092.stdout:52: expect (ObjNum 41 snap 0 seq_num 41) 2024-04-26T03:38:51.642 INFO:tasks.rados.rados.0.smithi092.stdout:53: read oid 25 snap -1 2024-04-26T03:38:51.642 INFO:tasks.rados.rados.0.smithi092.stdout:53: expect (ObjNum 24 snap 0 seq_num 24) 2024-04-26T03:38:51.642 INFO:tasks.rados.rados.0.smithi092.stdout:54: read oid 29 snap -1 2024-04-26T03:38:51.643 INFO:tasks.rados.rados.0.smithi092.stdout:54: expect (ObjNum 28 snap 0 seq_num 28) 2024-04-26T03:38:51.643 INFO:tasks.rados.rados.0.smithi092.stdout:55: rollback oid 37 current snap is 0 2024-04-26T03:38:51.643 INFO:tasks.rados.rados.0.smithi092.stdout:55: done (12 left) 2024-04-26T03:38:51.643 INFO:tasks.rados.rados.0.smithi092.stdout:56: delete oid 18 current snap is 0 2024-04-26T03:38:51.643 INFO:tasks.rados.rados.0.smithi092.stdout:44: finishing write tid 1 to smithi09235267-44 2024-04-26T03:38:51.643 INFO:tasks.rados.rados.0.smithi092.stdout:44: finishing write tid 2 to smithi09235267-44 2024-04-26T03:38:51.643 INFO:tasks.rados.rados.0.smithi092.stdout:44: finishing write tid 3 to smithi09235267-44 2024-04-26T03:38:51.643 INFO:tasks.rados.rados.0.smithi092.stdout:44: oid 44 updating version 0 to 11 2024-04-26T03:38:51.643 INFO:tasks.rados.rados.0.smithi092.stdout:44: oid 44 updating version 11 to 12 2024-04-26T03:38:51.643 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 44 v 12 (ObjNum 43 snap 0 seq_num 43) dirty exists 2024-04-26T03:38:51.643 INFO:tasks.rados.rados.0.smithi092.stdout:44: left oid 44 (ObjNum 43 snap 0 seq_num 43) 2024-04-26T03:38:51.643 INFO:tasks.rados.rados.0.smithi092.stdout:43: finishing write tid 2 to smithi09235267-43 2024-04-26T03:38:51.643 INFO:tasks.rados.rados.0.smithi092.stdout:45: finishing write tid 1 to smithi09235267-45 2024-04-26T03:38:51.643 INFO:tasks.rados.rados.0.smithi092.stdout:45: finishing write tid 2 to smithi09235267-45 2024-04-26T03:38:51.644 INFO:tasks.rados.rados.0.smithi092.stdout:43: finishing write tid 3 to smithi09235267-43 2024-04-26T03:38:51.644 INFO:tasks.rados.rados.0.smithi092.stdout:43: oid 43 updating version 0 to 7 2024-04-26T03:38:51.644 INFO:tasks.rados.rados.0.smithi092.stdout:43: oid 43 updating version 7 to 8 2024-04-26T03:38:51.644 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 43 v 8 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-04-26T03:38:51.644 INFO:tasks.rados.rados.0.smithi092.stdout:43: left oid 43 (ObjNum 42 snap 0 seq_num 42) 2024-04-26T03:38:51.644 INFO:tasks.rados.rados.0.smithi092.stdout:45: finishing write tid 3 to smithi09235267-45 2024-04-26T03:38:51.644 INFO:tasks.rados.rados.0.smithi092.stdout:45: oid 45 updating version 0 to 9 2024-04-26T03:38:51.644 INFO:tasks.rados.rados.0.smithi092.stdout:45: oid 45 updating version 9 to 10 2024-04-26T03:38:51.644 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 45 v 10 (ObjNum 44 snap 0 seq_num 44) dirty exists 2024-04-26T03:38:51.644 INFO:tasks.rados.rados.0.smithi092.stdout:45: left oid 45 (ObjNum 44 snap 0 seq_num 44) 2024-04-26T03:38:51.644 INFO:tasks.rados.rados.0.smithi092.stdout:46: finishing write tid 1 to smithi09235267-46 2024-04-26T03:38:51.644 INFO:tasks.rados.rados.0.smithi092.stdout:46: finishing write tid 2 to smithi09235267-46 2024-04-26T03:38:51.644 INFO:tasks.rados.rados.0.smithi092.stdout:46: finishing write tid 3 to smithi09235267-46 2024-04-26T03:38:51.644 INFO:tasks.rados.rados.0.smithi092.stdout:46: oid 46 updating version 0 to 5 2024-04-26T03:38:51.644 INFO:tasks.rados.rados.0.smithi092.stdout:46: oid 46 updating version 5 to 6 2024-04-26T03:38:51.645 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 46 v 6 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-04-26T03:38:51.645 INFO:tasks.rados.rados.0.smithi092.stdout:46: left oid 46 (ObjNum 45 snap 0 seq_num 45) 2024-04-26T03:38:51.645 INFO:tasks.rados.rados.0.smithi092.stdout:48: finishing write tid 1 to smithi09235267-48 2024-04-26T03:38:51.645 INFO:tasks.rados.rados.0.smithi092.stdout:47: finishing write tid 1 to smithi09235267-47 2024-04-26T03:38:51.645 INFO:tasks.rados.rados.0.smithi092.stdout:47: finishing write tid 2 to smithi09235267-47 2024-04-26T03:38:51.645 INFO:tasks.rados.rados.0.smithi092.stdout:48: finishing write tid 2 to smithi09235267-48 2024-04-26T03:38:51.645 INFO:tasks.rados.rados.0.smithi092.stdout:47: finishing write tid 3 to smithi09235267-47 2024-04-26T03:38:51.645 INFO:tasks.rados.rados.0.smithi092.stdout:47: oid 47 updating version 0 to 5 2024-04-26T03:38:51.645 INFO:tasks.rados.rados.0.smithi092.stdout:47: oid 47 updating version 5 to 6 2024-04-26T03:38:51.645 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 47 v 6 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-04-26T03:38:51.645 INFO:tasks.rados.rados.0.smithi092.stdout:47: left oid 47 (ObjNum 46 snap 0 seq_num 46) 2024-04-26T03:38:51.647 INFO:tasks.rados.rados.0.smithi092.stdout:48: finishing write tid 3 to smithi09235267-48 2024-04-26T03:38:51.647 INFO:tasks.rados.rados.0.smithi092.stdout:49: finishing write tid 1 to smithi09235267-49 2024-04-26T03:38:51.647 INFO:tasks.rados.rados.0.smithi092.stdout:48: finishing write tid 4 to smithi09235267-48 2024-04-26T03:38:51.647 INFO:tasks.rados.rados.0.smithi092.stdout:48: oid 48 updating version 0 to 11 2024-04-26T03:38:51.647 INFO:tasks.rados.rados.0.smithi092.stdout:48: oid 48 version 11 is already newer than 9 2024-04-26T03:38:51.647 INFO:tasks.rados.rados.0.smithi092.stdout:48: oid 48 version 11 is already newer than 10 2024-04-26T03:38:51.647 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 48 v 11 (ObjNum 47 snap 0 seq_num 47) dirty exists 2024-04-26T03:38:51.647 INFO:tasks.rados.rados.0.smithi092.stdout:48: left oid 48 (ObjNum 47 snap 0 seq_num 47) 2024-04-26T03:38:51.647 INFO:tasks.rados.rados.0.smithi092.stdout:51: finishing copy_from racing read to smithi09235267-14 2024-04-26T03:38:51.647 INFO:tasks.rados.rados.0.smithi092.stdout:43: done (12 left) 2024-04-26T03:38:51.648 INFO:tasks.rados.rados.0.smithi092.stdout:44: done (11 left) 2024-04-26T03:38:51.648 INFO:tasks.rados.rados.0.smithi092.stdout:45: done (10 left) 2024-04-26T03:38:51.648 INFO:tasks.rados.rados.0.smithi092.stdout:46: done (9 left) 2024-04-26T03:38:51.648 INFO:tasks.rados.rados.0.smithi092.stdout:47: done (8 left) 2024-04-26T03:38:51.648 INFO:tasks.rados.rados.0.smithi092.stdout:48: done (7 left) 2024-04-26T03:38:51.648 INFO:tasks.rados.rados.0.smithi092.stdout:56: done (6 left) 2024-04-26T03:38:51.648 INFO:tasks.rados.rados.0.smithi092.stdout:57: read oid 19 snap -1 2024-04-26T03:38:51.648 INFO:tasks.rados.rados.0.smithi092.stdout:57: expect (ObjNum 18 snap 0 seq_num 18) 2024-04-26T03:38:51.648 INFO:tasks.rados.rados.0.smithi092.stdout:58: rollback oid 43 current snap is 0 2024-04-26T03:38:51.648 INFO:tasks.rados.rados.0.smithi092.stdout:58: done (7 left) 2024-04-26T03:38:51.648 INFO:tasks.rados.rados.0.smithi092.stdout:59: read oid 39 snap -1 2024-04-26T03:38:51.648 INFO:tasks.rados.rados.0.smithi092.stdout:59: expect (ObjNum 38 snap 0 seq_num 38) 2024-04-26T03:38:51.648 INFO:tasks.rados.rados.0.smithi092.stdout:60: rmattr oid 22 current snap is 0 2024-04-26T03:38:51.648 INFO:tasks.rados.rados.0.smithi092.stdout:60: done (8 left) 2024-04-26T03:38:51.648 INFO:tasks.rados.rados.0.smithi092.stdout:61: rollback oid 12 current snap is 0 2024-04-26T03:38:51.648 INFO:tasks.rados.rados.0.smithi092.stdout:61: done (8 left) 2024-04-26T03:38:51.649 INFO:tasks.rados.rados.0.smithi092.stdout:62: delete oid 47 current snap is 0 2024-04-26T03:38:51.649 INFO:tasks.rados.rados.0.smithi092.stdout:49: finishing write tid 2 to smithi09235267-49 2024-04-26T03:38:51.649 INFO:tasks.rados.rados.0.smithi092.stdout:49: finishing write tid 3 to smithi09235267-49 2024-04-26T03:38:51.649 INFO:tasks.rados.rados.0.smithi092.stdout:49: finishing write tid 4 to smithi09235267-49 2024-04-26T03:38:51.649 INFO:tasks.rados.rados.0.smithi092.stdout:49: oid 49 updating version 0 to 1 2024-04-26T03:38:51.649 INFO:tasks.rados.rados.0.smithi092.stdout:49: oid 49 updating version 1 to 2 2024-04-26T03:38:51.649 INFO:tasks.rados.rados.0.smithi092.stdout:49: oid 49 updating version 2 to 3 2024-04-26T03:38:51.649 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 49 v 3 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-04-26T03:38:51.649 INFO:tasks.rados.rados.0.smithi092.stdout:49: left oid 49 (ObjNum 48 snap 0 seq_num 48) 2024-04-26T03:38:51.649 INFO:tasks.rados.rados.0.smithi092.stdout:52: expect (ObjNum 41 snap 0 seq_num 41) 2024-04-26T03:38:51.654 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2: {} 2024-04-26T03:38:51.654 INFO:teuthology.orchestra.run.smithi028.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T03:38:51.655 INFO:tasks.ceph.osd.9.smithi042.stderr:2024-04-26T03:38:51.653+0000 7fdac63b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 31960) UID: 0 2024-04-26T03:38:51.669 INFO:tasks.rados.rados.0.smithi092.stdout:51: finishing copy_from to smithi09235267-14 2024-04-26T03:38:51.670 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 14 v 2 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-04-26T03:38:51.670 INFO:tasks.rados.rados.0.smithi092.stdout:54: expect (ObjNum 28 snap 0 seq_num 28) 2024-04-26T03:38:51.672 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3: {} 2024-04-26T03:38:51.672 INFO:teuthology.orchestra.run.smithi028.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T03:38:51.682 INFO:tasks.rados.rados.0.smithi092.stdout:57: expect (ObjNum 18 snap 0 seq_num 18) 2024-04-26T03:38:51.688 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4: {} 2024-04-26T03:38:51.689 INFO:teuthology.orchestra.run.smithi028.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T03:38:51.690 INFO:tasks.rados.rados.0.smithi092.stdout:50: finishing write tid 1 to smithi09235267-50 2024-04-26T03:38:51.690 INFO:tasks.rados.rados.0.smithi092.stdout:53: expect (ObjNum 24 snap 0 seq_num 24) 2024-04-26T03:38:51.701 INFO:tasks.rados.rados.0.smithi092.stdout:50: finishing write tid 2 to smithi09235267-50 2024-04-26T03:38:51.701 INFO:tasks.rados.rados.0.smithi092.stdout:50: finishing write tid 3 to smithi09235267-50 2024-04-26T03:38:51.701 INFO:tasks.rados.rados.0.smithi092.stdout:50: finishing write tid 4 to smithi09235267-50 2024-04-26T03:38:51.701 INFO:tasks.rados.rados.0.smithi092.stdout:50: oid 50 updating version 0 to 13 2024-04-26T03:38:51.702 INFO:tasks.rados.rados.0.smithi092.stdout:50: oid 50 updating version 13 to 14 2024-04-26T03:38:51.702 INFO:tasks.rados.rados.0.smithi092.stdout:50: oid 50 updating version 14 to 15 2024-04-26T03:38:51.702 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 50 v 15 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-04-26T03:38:51.702 INFO:tasks.rados.rados.0.smithi092.stdout:50: left oid 50 (ObjNum 49 snap 0 seq_num 49) 2024-04-26T03:38:51.702 INFO:tasks.rados.rados.0.smithi092.stdout:59: expect (ObjNum 38 snap 0 seq_num 38) 2024-04-26T03:38:51.706 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5: {} 2024-04-26T03:38:51.706 INFO:teuthology.orchestra.run.smithi028.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T03:38:51.718 INFO:tasks.rados.rados.0.smithi092.stdout:49: done (8 left) 2024-04-26T03:38:51.719 INFO:tasks.rados.rados.0.smithi092.stdout:50: done (7 left) 2024-04-26T03:38:51.719 INFO:tasks.rados.rados.0.smithi092.stdout:51: done (6 left) 2024-04-26T03:38:51.719 INFO:tasks.rados.rados.0.smithi092.stdout:52: done (5 left) 2024-04-26T03:38:51.719 INFO:tasks.rados.rados.0.smithi092.stdout:53: done (4 left) 2024-04-26T03:38:51.719 INFO:tasks.rados.rados.0.smithi092.stdout:54: done (3 left) 2024-04-26T03:38:51.719 INFO:tasks.rados.rados.0.smithi092.stdout:57: done (2 left) 2024-04-26T03:38:51.719 INFO:tasks.rados.rados.0.smithi092.stdout:59: done (1 left) 2024-04-26T03:38:51.719 INFO:tasks.rados.rados.0.smithi092.stdout:62: done (0 left) 2024-04-26T03:38:51.719 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 14 current snap is 0 2024-04-26T03:38:51.719 INFO:tasks.rados.rados.0.smithi092.stdout:63: seq_num 52 ranges {131072=507904,638976=376832} 2024-04-26T03:38:51.721 INFO:tasks.rados.rados.0.smithi092.stdout:63: writing smithi09235267-14 from 131072 to 638976 tid 1 2024-04-26T03:38:51.724 INFO:tasks.rados.rados.0.smithi092.stdout:63: writing smithi09235267-14 from 638976 to 1015808 tid 2 2024-04-26T03:38:51.724 INFO:tasks.rados.rados.0.smithi092.stdout:64: rollback oid 3 current snap is 0 2024-04-26T03:38:51.724 INFO:tasks.rados.rados.0.smithi092.stdout:64: done (1 left) 2024-04-26T03:38:51.724 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 42 current snap is 0 2024-04-26T03:38:51.724 INFO:tasks.rados.rados.0.smithi092.stdout:65: seq_num 53 ranges {704512=524288,1228800=376832} 2024-04-26T03:38:51.724 INFO:teuthology.orchestra.run.smithi028.stdout:osd.6: {} 2024-04-26T03:38:51.725 INFO:teuthology.orchestra.run.smithi028.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T03:38:51.734 INFO:tasks.rados.rados.0.smithi092.stdout:65: writing smithi09235267-42 from 704512 to 1228800 tid 1 2024-04-26T03:38:51.735 INFO:tasks.rados.rados.0.smithi092.stdout:65: writing smithi09235267-42 from 1228800 to 1605632 tid 2 2024-04-26T03:38:51.735 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 10 current snap is 0 2024-04-26T03:38:51.735 INFO:tasks.rados.rados.0.smithi092.stdout:63: finishing write tid 1 to smithi09235267-14 2024-04-26T03:38:51.735 INFO:tasks.rados.rados.0.smithi092.stdout:63: finishing write tid 2 to smithi09235267-14 2024-04-26T03:38:51.735 INFO:tasks.rados.rados.0.smithi092.stdout:63: finishing write tid 3 to smithi09235267-14 2024-04-26T03:38:51.735 INFO:tasks.rados.rados.0.smithi092.stdout:66: seq_num 54 ranges {163840=540672,704512=376832} 2024-04-26T03:38:51.736 INFO:tasks.rados.rados.0.smithi092.stdout:66: writing smithi09235267-10 from 163840 to 704512 tid 1 2024-04-26T03:38:51.740 INFO:tasks.rados.rados.0.smithi092.stdout:66: writing smithi09235267-10 from 704512 to 1081344 tid 2 2024-04-26T03:38:51.740 INFO:tasks.rados.rados.0.smithi092.stdout:67: rmattr oid 13 current snap is 0 2024-04-26T03:38:51.740 INFO:tasks.rados.rados.0.smithi092.stdout:67: done (3 left) 2024-04-26T03:38:51.740 INFO:tasks.rados.rados.0.smithi092.stdout:68: delete oid 5 current snap is 0 2024-04-26T03:38:51.740 INFO:tasks.rados.rados.0.smithi092.stdout:63: finishing write tid 4 to smithi09235267-14 2024-04-26T03:38:51.740 INFO:tasks.rados.rados.0.smithi092.stdout:63: oid 14 updating version 0 to 8 2024-04-26T03:38:51.740 INFO:tasks.rados.rados.0.smithi092.stdout:63: oid 14 version 8 is already newer than 7 2024-04-26T03:38:51.740 INFO:tasks.rados.rados.0.smithi092.stdout:63: oid 14 version 8 is already newer than 6 2024-04-26T03:38:51.740 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 14 v 8 (ObjNum 52 snap 0 seq_num 52) dirty exists 2024-04-26T03:38:51.740 INFO:tasks.rados.rados.0.smithi092.stdout:63: left oid 14 (ObjNum 52 snap 0 seq_num 52) 2024-04-26T03:38:51.741 INFO:tasks.rados.rados.0.smithi092.stdout:65: finishing write tid 1 to smithi09235267-42 2024-04-26T03:38:51.741 INFO:tasks.rados.rados.0.smithi092.stdout:65: finishing write tid 2 to smithi09235267-42 2024-04-26T03:38:51.741 INFO:tasks.rados.rados.0.smithi092.stdout:63: done (3 left) 2024-04-26T03:38:51.742 INFO:tasks.rados.rados.0.smithi092.stdout:68: done (2 left) 2024-04-26T03:38:51.742 INFO:tasks.rados.rados.0.smithi092.stdout:69: read oid 2 snap -1 2024-04-26T03:38:51.742 INFO:tasks.rados.rados.0.smithi092.stdout:69: expect (ObjNum 1 snap 0 seq_num 1) 2024-04-26T03:38:51.742 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 46 current snap is 0 2024-04-26T03:38:51.742 INFO:tasks.rados.rados.0.smithi092.stdout:65: finishing write tid 3 to smithi09235267-42 2024-04-26T03:38:51.742 INFO:tasks.rados.rados.0.smithi092.stdout:70: seq_num 56 ranges {770048=573440,1343488=376832} 2024-04-26T03:38:51.748 INFO:teuthology.orchestra.run.smithi028.stdout:osd.7: {} 2024-04-26T03:38:51.748 INFO:teuthology.orchestra.run.smithi028.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T03:38:51.749 INFO:tasks.rados.rados.0.smithi092.stdout:70: writing smithi09235267-46 from 770048 to 1343488 tid 1 2024-04-26T03:38:51.752 INFO:tasks.rados.rados.0.smithi092.stdout:70: writing smithi09235267-46 from 1343488 to 1720320 tid 2 2024-04-26T03:38:51.752 INFO:tasks.rados.rados.0.smithi092.stdout:71: read oid 23 snap -1 2024-04-26T03:38:51.752 INFO:tasks.rados.rados.0.smithi092.stdout:71: expect (ObjNum 22 snap 0 seq_num 22) 2024-04-26T03:38:51.752 INFO:tasks.rados.rados.0.smithi092.stdout:72: delete oid 17 current snap is 0 2024-04-26T03:38:51.752 INFO:tasks.rados.rados.0.smithi092.stdout:65: finishing write tid 4 to smithi09235267-42 2024-04-26T03:38:51.752 INFO:tasks.rados.rados.0.smithi092.stdout:65: oid 42 updating version 0 to 11 2024-04-26T03:38:51.752 INFO:tasks.rados.rados.0.smithi092.stdout:65: oid 42 version 11 is already newer than 9 2024-04-26T03:38:51.752 INFO:tasks.rados.rados.0.smithi092.stdout:65: oid 42 version 11 is already newer than 10 2024-04-26T03:38:51.753 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 42 v 11 (ObjNum 53 snap 0 seq_num 53) dirty exists 2024-04-26T03:38:51.753 INFO:tasks.rados.rados.0.smithi092.stdout:65: left oid 42 (ObjNum 53 snap 0 seq_num 53) 2024-04-26T03:38:51.753 INFO:tasks.rados.rados.0.smithi092.stdout:69: expect (ObjNum 1 snap 0 seq_num 1) 2024-04-26T03:38:51.753 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:51.753 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:51.754 INFO:tasks.rados.rados.0.smithi092.stdout:66: finishing write tid 1 to smithi09235267-10 2024-04-26T03:38:51.754 INFO:tasks.rados.rados.0.smithi092.stdout:66: finishing write tid 2 to smithi09235267-10 2024-04-26T03:38:51.754 INFO:tasks.rados.rados.0.smithi092.stdout:66: finishing write tid 3 to smithi09235267-10 2024-04-26T03:38:51.754 INFO:tasks.rados.rados.0.smithi092.stdout:66: finishing write tid 4 to smithi09235267-10 2024-04-26T03:38:51.754 INFO:tasks.rados.rados.0.smithi092.stdout:66: oid 10 updating version 0 to 7 2024-04-26T03:38:51.754 INFO:tasks.rados.rados.0.smithi092.stdout:66: oid 10 version 7 is already newer than 6 2024-04-26T03:38:51.754 INFO:tasks.rados.rados.0.smithi092.stdout:66: oid 10 version 7 is already newer than 5 2024-04-26T03:38:51.754 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 10 v 7 (ObjNum 54 snap 0 seq_num 54) dirty exists 2024-04-26T03:38:51.754 INFO:tasks.rados.rados.0.smithi092.stdout:66: left oid 10 (ObjNum 54 snap 0 seq_num 54) 2024-04-26T03:38:51.755 INFO:tasks.rados.rados.0.smithi092.stdout:65: done (5 left) 2024-04-26T03:38:51.755 INFO:tasks.rados.rados.0.smithi092.stdout:66: done (4 left) 2024-04-26T03:38:51.755 INFO:tasks.rados.rados.0.smithi092.stdout:69: done (3 left) 2024-04-26T03:38:51.755 INFO:tasks.rados.rados.0.smithi092.stdout:72: done (2 left) 2024-04-26T03:38:51.755 INFO:tasks.rados.rados.0.smithi092.stdout:73: copy_from oid 36 from oid 27 current snap is 0 2024-04-26T03:38:51.755 INFO:tasks.rados.rados.0.smithi092.stdout:74: read oid 43 snap -1 2024-04-26T03:38:51.755 INFO:tasks.rados.rados.0.smithi092.stdout:74: expect (ObjNum 42 snap 0 seq_num 42) 2024-04-26T03:38:51.755 INFO:tasks.rados.rados.0.smithi092.stdout:75: delete oid 13 current snap is 0 2024-04-26T03:38:51.756 INFO:tasks.rados.rados.0.smithi092.stdout:71: expect (ObjNum 22 snap 0 seq_num 22) 2024-04-26T03:38:51.756 INFO:tasks.ceph.osd.4.smithi028.stderr:2024-04-26T03:38:51.754+0000 7fd9511b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31963) UID: 0 2024-04-26T03:38:51.764 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-04-26T03:38:51.768 INFO:tasks.rados.rados.0.smithi092.stdout:74: expect (ObjNum 42 snap 0 seq_num 42) 2024-04-26T03:38:51.772 INFO:teuthology.orchestra.run.smithi028.stdout:osd.8: {} 2024-04-26T03:38:51.772 INFO:teuthology.orchestra.run.smithi028.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T03:38:51.787 INFO:tasks.rados.rados.0.smithi092.stdout:70: finishing write tid 1 to smithi09235267-46 2024-04-26T03:38:51.788 INFO:tasks.rados.rados.0.smithi092.stdout:70: finishing write tid 2 to smithi09235267-46 2024-04-26T03:38:51.788 INFO:tasks.rados.rados.0.smithi092.stdout:71: done (4 left) 2024-04-26T03:38:51.788 INFO:tasks.rados.rados.0.smithi092.stdout:74: done (3 left) 2024-04-26T03:38:51.788 INFO:tasks.rados.rados.0.smithi092.stdout:75: done (2 left) 2024-04-26T03:38:51.788 INFO:tasks.rados.rados.0.smithi092.stdout:76: rollback oid 10 current snap is 0 2024-04-26T03:38:51.788 INFO:tasks.rados.rados.0.smithi092.stdout:76: done (2 left) 2024-04-26T03:38:51.788 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 7 current snap is 0 2024-04-26T03:38:51.788 INFO:tasks.rados.rados.0.smithi092.stdout:70: finishing write tid 3 to smithi09235267-46 2024-04-26T03:38:51.788 INFO:tasks.rados.rados.0.smithi092.stdout:77: seq_num 59 ranges {114688=622592,737280=376832} 2024-04-26T03:38:51.789 INFO:teuthology.orchestra.run.smithi028.stdout:osd.9: {} 2024-04-26T03:38:51.789 INFO:teuthology.orchestra.run.smithi028.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T03:38:51.791 INFO:tasks.rados.rados.0.smithi092.stdout:77: writing smithi09235267-7 from 114688 to 737280 tid 1 2024-04-26T03:38:51.793 INFO:tasks.rados.rados.0.smithi092.stdout:77: writing smithi09235267-7 from 737280 to 1114112 tid 2 2024-04-26T03:38:51.794 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 42 current snap is 0 2024-04-26T03:38:51.794 INFO:tasks.rados.rados.0.smithi092.stdout:78: seq_num 60 ranges {1605632=638976,2244608=376832} 2024-04-26T03:38:51.806 INFO:tasks.rados.rados.0.smithi092.stdout:78: writing smithi09235267-42 from 1605632 to 2244608 tid 1 2024-04-26T03:38:51.807 INFO:teuthology.orchestra.run.smithi028.stdout:osd.10: {} 2024-04-26T03:38:51.807 INFO:teuthology.orchestra.run.smithi028.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T03:38:51.809 INFO:tasks.rados.rados.0.smithi092.stdout:78: writing smithi09235267-42 from 2244608 to 2621440 tid 2 2024-04-26T03:38:51.809 INFO:tasks.rados.rados.0.smithi092.stdout:79: read oid 19 snap -1 2024-04-26T03:38:51.809 INFO:tasks.rados.rados.0.smithi092.stdout:79: expect (ObjNum 18 snap 0 seq_num 18) 2024-04-26T03:38:51.809 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 30 current snap is 0 2024-04-26T03:38:51.809 INFO:tasks.rados.rados.0.smithi092.stdout:70: finishing write tid 4 to smithi09235267-46 2024-04-26T03:38:51.809 INFO:tasks.rados.rados.0.smithi092.stdout:70: oid 46 updating version 0 to 8 2024-04-26T03:38:51.809 INFO:tasks.rados.rados.0.smithi092.stdout:70: oid 46 updating version 8 to 9 2024-04-26T03:38:51.809 INFO:tasks.rados.rados.0.smithi092.stdout:70: oid 46 version 9 is already newer than 7 2024-04-26T03:38:51.809 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 46 v 9 (ObjNum 56 snap 0 seq_num 56) dirty exists 2024-04-26T03:38:51.809 INFO:tasks.rados.rados.0.smithi092.stdout:70: left oid 46 (ObjNum 56 snap 0 seq_num 56) 2024-04-26T03:38:51.810 INFO:tasks.rados.rados.0.smithi092.stdout:73: finishing copy_from racing read to smithi09235267-36 2024-04-26T03:38:51.810 INFO:tasks.rados.rados.0.smithi092.stdout:73: finishing copy_from to smithi09235267-36 2024-04-26T03:38:51.810 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 36 v 4 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-04-26T03:38:51.810 INFO:tasks.rados.rados.0.smithi092.stdout:80: seq_num 61 ranges {491520=655360,1146880=376832} 2024-04-26T03:38:51.815 INFO:tasks.rados.rados.0.smithi092.stdout:80: writing smithi09235267-30 from 491520 to 1146880 tid 1 2024-04-26T03:38:51.818 INFO:tasks.rados.rados.0.smithi092.stdout:80: writing smithi09235267-30 from 1146880 to 1523712 tid 2 2024-04-26T03:38:51.818 INFO:tasks.rados.rados.0.smithi092.stdout:70: done (5 left) 2024-04-26T03:38:51.818 INFO:tasks.rados.rados.0.smithi092.stdout:73: done (4 left) 2024-04-26T03:38:51.818 INFO:tasks.rados.rados.0.smithi092.stdout:81: delete oid 27 current snap is 0 2024-04-26T03:38:51.818 INFO:tasks.rados.rados.0.smithi092.stdout:77: finishing write tid 1 to smithi09235267-7 2024-04-26T03:38:51.819 INFO:tasks.rados.rados.0.smithi092.stdout:77: finishing write tid 2 to smithi09235267-7 2024-04-26T03:38:51.819 INFO:tasks.rados.rados.0.smithi092.stdout:77: finishing write tid 3 to smithi09235267-7 2024-04-26T03:38:51.819 INFO:tasks.rados.rados.0.smithi092.stdout:77: finishing write tid 4 to smithi09235267-7 2024-04-26T03:38:51.819 INFO:tasks.rados.rados.0.smithi092.stdout:77: oid 7 updating version 0 to 18 2024-04-26T03:38:51.819 INFO:tasks.rados.rados.0.smithi092.stdout:77: oid 7 version 18 is already newer than 16 2024-04-26T03:38:51.819 INFO:tasks.rados.rados.0.smithi092.stdout:77: oid 7 version 18 is already newer than 17 2024-04-26T03:38:51.819 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 7 v 18 (ObjNum 59 snap 0 seq_num 59) dirty exists 2024-04-26T03:38:51.819 INFO:tasks.rados.rados.0.smithi092.stdout:77: left oid 7 (ObjNum 59 snap 0 seq_num 59) 2024-04-26T03:38:51.819 INFO:tasks.rados.rados.0.smithi092.stdout:79: expect (ObjNum 18 snap 0 seq_num 18) 2024-04-26T03:38:51.829 INFO:teuthology.orchestra.run.smithi028.stdout:osd.11: {} 2024-04-26T03:38:51.829 INFO:teuthology.orchestra.run.smithi028.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T03:38:51.831 INFO:tasks.rados.rados.0.smithi092.stdout:78: finishing write tid 1 to smithi09235267-42 2024-04-26T03:38:51.832 INFO:tasks.rados.rados.0.smithi092.stdout:78: finishing write tid 2 to smithi09235267-42 2024-04-26T03:38:51.832 INFO:tasks.rados.rados.0.smithi092.stdout:78: finishing write tid 3 to smithi09235267-42 2024-04-26T03:38:51.832 INFO:tasks.rados.rados.0.smithi092.stdout:77: done (4 left) 2024-04-26T03:38:51.832 INFO:tasks.rados.rados.0.smithi092.stdout:79: done (3 left) 2024-04-26T03:38:51.832 INFO:tasks.rados.rados.0.smithi092.stdout:81: done (2 left) 2024-04-26T03:38:51.832 INFO:tasks.rados.rados.0.smithi092.stdout:82: setattr oid 21 current snap is 0 2024-04-26T03:38:51.832 INFO:tasks.rados.rados.0.smithi092.stdout:78: finishing write tid 4 to smithi09235267-42 2024-04-26T03:38:51.832 INFO:tasks.rados.rados.0.smithi092.stdout:78: oid 42 updating version 0 to 13 2024-04-26T03:38:51.833 INFO:tasks.rados.rados.0.smithi092.stdout:78: oid 42 version 13 is already newer than 12 2024-04-26T03:38:51.833 INFO:tasks.rados.rados.0.smithi092.stdout:78: oid 42 updating version 13 to 14 2024-04-26T03:38:51.833 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 42 v 14 (ObjNum 60 snap 0 seq_num 60) dirty exists 2024-04-26T03:38:51.833 INFO:tasks.rados.rados.0.smithi092.stdout:78: left oid 42 (ObjNum 60 snap 0 seq_num 60) 2024-04-26T03:38:51.833 INFO:tasks.rados.rados.0.smithi092.stdout:80: finishing write tid 1 to smithi09235267-30 2024-04-26T03:38:51.833 INFO:tasks.rados.rados.0.smithi092.stdout:80: finishing write tid 2 to smithi09235267-30 2024-04-26T03:38:51.833 INFO:tasks.rados.rados.0.smithi092.stdout:80: finishing write tid 3 to smithi09235267-30 2024-04-26T03:38:51.833 INFO:tasks.rados.rados.0.smithi092.stdout:80: finishing write tid 4 to smithi09235267-30 2024-04-26T03:38:51.834 INFO:tasks.rados.rados.0.smithi092.stdout:80: oid 30 updating version 0 to 13 2024-04-26T03:38:51.834 INFO:tasks.rados.rados.0.smithi092.stdout:80: oid 30 updating version 13 to 14 2024-04-26T03:38:51.834 INFO:tasks.rados.rados.0.smithi092.stdout:80: oid 30 version 14 is already newer than 12 2024-04-26T03:38:51.834 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 30 v 14 (ObjNum 61 snap 0 seq_num 61) dirty exists 2024-04-26T03:38:51.834 INFO:tasks.rados.rados.0.smithi092.stdout:80: left oid 30 (ObjNum 61 snap 0 seq_num 61) 2024-04-26T03:38:51.835 INFO:tasks.rados.rados.0.smithi092.stdout:78: done (2 left) 2024-04-26T03:38:51.835 INFO:tasks.rados.rados.0.smithi092.stdout:80: done (1 left) 2024-04-26T03:38:51.835 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 1 current snap is 0 2024-04-26T03:38:51.835 INFO:tasks.rados.rados.0.smithi092.stdout:83: seq_num 63 ranges {32768=688128,720896=376832} 2024-04-26T03:38:51.836 INFO:tasks.rados.rados.0.smithi092.stdout:83: writing smithi09235267-1 from 32768 to 720896 tid 1 2024-04-26T03:38:51.838 INFO:tasks.rados.rados.0.smithi092.stdout:83: writing smithi09235267-1 from 720896 to 1097728 tid 2 2024-04-26T03:38:51.839 INFO:tasks.rados.rados.0.smithi092.stdout:84: snap_create 2024-04-26T03:38:51.839 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 21 v 12 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-04-26T03:38:51.845 INFO:tasks.rados.rados.0.smithi092.stdout:83: finishing write tid 1 to smithi09235267-1 2024-04-26T03:38:51.847 INFO:teuthology.orchestra.run.smithi028.stdout:osd.12: {} 2024-04-26T03:38:51.847 INFO:teuthology.orchestra.run.smithi028.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T03:38:51.848 INFO:tasks.rados.rados.0.smithi092.stdout:83: finishing write tid 2 to smithi09235267-1 2024-04-26T03:38:51.850 INFO:tasks.rados.rados.0.smithi092.stdout:83: finishing write tid 3 to smithi09235267-1 2024-04-26T03:38:51.851 INFO:tasks.rados.rados.0.smithi092.stdout:83: finishing write tid 4 to smithi09235267-1 2024-04-26T03:38:51.851 INFO:tasks.rados.rados.0.smithi092.stdout:83: oid 1 updating version 0 to 9 2024-04-26T03:38:51.851 INFO:tasks.rados.rados.0.smithi092.stdout:83: oid 1 updating version 9 to 10 2024-04-26T03:38:51.852 INFO:tasks.rados.rados.0.smithi092.stdout:83: oid 1 updating version 10 to 11 2024-04-26T03:38:51.852 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 1 v 11 (ObjNum 63 snap 0 seq_num 63) dirty exists 2024-04-26T03:38:51.852 INFO:tasks.rados.rados.0.smithi092.stdout:83: left oid 1 (ObjNum 63 snap 0 seq_num 63) 2024-04-26T03:38:51.857 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-26T03:38:51.855+0000 7f610b3a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31816) UID: 0 2024-04-26T03:38:51.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:38:51.861+0000 7f857bfff640 0 -- 172.21.15.28:0/816437784 >> [v2:172.21.15.42:6817/301360893,v1:172.21.15.42:6819/301360893] conn(0x7f857c21a980 msgr2=0x7f857c21cd70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-26T03:38:51.865 INFO:teuthology.orchestra.run.smithi028.stdout:osd.13: {} 2024-04-26T03:38:51.865 INFO:teuthology.orchestra.run.smithi028.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T03:38:51.883 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:51.883 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:51.883 INFO:teuthology.orchestra.run.smithi028.stdout:osd.14: {} 2024-04-26T03:38:51.883 INFO:teuthology.orchestra.run.smithi028.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T03:38:51.892 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-04-26T03:38:51.902 INFO:teuthology.orchestra.run.smithi028.stdout:osd.15: {} 2024-04-26T03:38:51.902 INFO:teuthology.orchestra.run.smithi028.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T03:38:51.958 INFO:tasks.ceph.osd.14.smithi066.stderr:2024-04-26T03:38:51.957+0000 7f5395f08640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 31765) UID: 0 2024-04-26T03:38:52.033 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:52.033 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:52.042 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-04-26T03:38:52.085 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-26T03:38:52.084+0000 7f610b3a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31816) UID: 0 2024-04-26T03:38:52.146 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:52.147 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:52.153 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-04-26T03:38:52.160 INFO:tasks.ceph.osd.8.smithi028.stderr:2024-04-26T03:38:52.159+0000 7f5bb75d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 31968) UID: 0 2024-04-26T03:38:52.263 INFO:tasks.ceph.osd.9.smithi042.stderr:2024-04-26T03:38:52.261+0000 7fdac63b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 31960) UID: 0 2024-04-26T03:38:52.296 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:52.296 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:52.305 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-04-26T03:38:52.364 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-04-26T03:38:52.363+0000 7fe9223c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31954) UID: 0 2024-04-26T03:38:52.407 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:52.407 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:52.416 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-04-26T03:38:52.465 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-04-26T03:38:52.463+0000 7f745118d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 31821) UID: 0 2024-04-26T03:38:52.483 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-04-26T03:38:52.483 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-26T03:38:52.484 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_0 2024-04-26T03:38:52.484 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T03:38:52.488 INFO:tasks.rados.rados.0.smithi092.stdout:82: done (2 left) 2024-04-26T03:38:52.489 INFO:tasks.rados.rados.0.smithi092.stdout:83: done (1 left) 2024-04-26T03:38:52.489 INFO:tasks.rados.rados.0.smithi092.stdout:84: done (0 left) 2024-04-26T03:38:52.489 INFO:tasks.rados.rados.0.smithi092.stdout:85: snap_create 2024-04-26T03:38:52.493 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-26T03:38:52.520 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:52.520 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:52.529 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-26T03:38:52.566 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:38:52.565+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:38:52.651 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:52.651 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:52.661 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-26T03:38:52.667 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:38:52.666+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:38:52.768 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:38:52.766+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:38:52.769 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:52.770 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:52.778 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-26T03:38:52.804 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:38:52.805 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-04-26T03:38:52.817 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2024-04-26T03:38:52.485982+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":14860832,"num_objects":26,"num_object_clones":0,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":224,"num_read_kb":6638,"num_write":280,"num_write_kb":19732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":234,"ondisk_log_size":234,"up":95,"acting":95,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":14,"kb":1509949440,"kb_used":466612,"kb_used_data":17468,"kb_used_omap":24,"kb_used_meta":448743,"kb_avail":1509482828,"statfs":{"total":1546188226560,"available":1545710415872,"internally_reserved":0,"allocated":17887232,"data_stored":16287155,"data_compressed":315180,"data_compressed_allocated":708608,"data_compressed_original":2310144,"omap_allocated":25522,"internal_metadata":459512910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":14270464,"num_objects":24,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":88,"num_read_kb":6520,"num_write":77,"num_write_kb":16272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.019191"},"pg_stats":[{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T03:38:50.517859+0000","last_change":"2024-04-26T03:38:49.487382+0000","last_active":"2024-04-26T03:38:50.517859+0000","last_peered":"2024-04-26T03:38:50.517859+0000","last_clean":"2024-04-26T03:38:50.517859+0000","last_became_active":"2024-04-26T03:38:49.486850+0000","last_became_peered":"2024-04-26T03:38:49.486850+0000","last_unstale":"2024-04-26T03:38:50.517859+0000","last_undegraded":"2024-04-26T03:38:50.517859+0000","last_fullsized":"2024-04-26T03:38:50.517859+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:58.507295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,13,5,14,2,1,0],"acting":[11,13,5,14,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"35'14","reported_seq":46,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-26T03:38:51.819426+0000","last_change":"2024-04-26T03:38:49.942161+0000","last_active":"2024-04-26T03:38:51.819426+0000","last_peered":"2024-04-26T03:38:51.819426+0000","last_clean":"2024-04-26T03:38:51.819426+0000","last_became_active":"2024-04-26T03:38:49.941260+0000","last_became_peered":"2024-04-26T03:38:49.941260+0000","last_unstale":"2024-04-26T03:38:51.819426+0000","last_undegraded":"2024-04-26T03:38:51.819426+0000","last_fullsized":"2024-04-26T03:38:51.819426+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:09.815347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3817472,"num_objects":4,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":18,"num_read_kb":3464,"num_write":14,"num_write_kb":3728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,14,6,11,8,9,12],"acting":[13,14,6,11,8,9,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.d","version":"35'7","reported_seq":30,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-26T03:38:51.643451+0000","last_change":"2024-04-26T03:38:49.512510+0000","last_active":"2024-04-26T03:38:51.643451+0000","last_peered":"2024-04-26T03:38:51.643451+0000","last_clean":"2024-04-26T03:38:51.643451+0000","last_became_active":"2024-04-26T03:38:49.511783+0000","last_became_peered":"2024-04-26T03:38:49.511783+0000","last_unstale":"2024-04-26T03:38:51.643451+0000","last_undegraded":"2024-04-26T03:38:51.643451+0000","last_fullsized":"2024-04-26T03:38:51.643451+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:09.694455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":606208,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":3,"num_write":7,"num_write_kb":880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,9,1,5,8,4],"acting":[7,3,9,1,5,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T03:38:50.517648+0000","last_change":"2024-04-26T03:38:49.941997+0000","last_active":"2024-04-26T03:38:50.517648+0000","last_peered":"2024-04-26T03:38:50.517648+0000","last_clean":"2024-04-26T03:38:50.517648+0000","last_became_active":"2024-04-26T03:38:49.941017+0000","last_became_peered":"2024-04-26T03:38:49.941017+0000","last_unstale":"2024-04-26T03:38:50.517648+0000","last_undegraded":"2024-04-26T03:38:50.517648+0000","last_fullsized":"2024-04-26T03:38:50.517648+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:51.807456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,0,1,9,12,14],"acting":[11,8,0,1,9,12,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T03:38:48.463214+0000","last_change":"2024-04-26T03:38:48.463214+0000","last_active":"2024-04-26T03:38:48.463214+0000","last_peered":"2024-04-26T03:38:48.463214+0000","last_clean":"2024-04-26T03:38:48.463214+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T03:38:48.463214+0000","last_undegraded":"2024-04-26T03:38:48.463214+0000","last_fullsized":"2024-04-26T03:38:48.463214+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T03:38:48.476636+0000","last_change":"2024-04-26T03:38:48.476636+0000","last_active":"2024-04-26T03:38:48.463214+0000","last_peered":"2024-04-26T03:38:48.463214+0000","last_clean":"2024-04-26T03:38:48.463214+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T03:38:48.476636+0000","last_undegraded":"2024-04-26T03:38:48.476636+0000","last_fullsized":"2024-04-26T03:38:48.476636+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,10,13,1,11],"acting":[3,7,6,10,13,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"35'6","reported_seq":29,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-26T03:38:51.565572+0000","last_change":"2024-04-26T03:38:49.512705+0000","last_active":"2024-04-26T03:38:51.565572+0000","last_peered":"2024-04-26T03:38:51.565572+0000","last_clean":"2024-04-26T03:38:51.565572+0000","last_became_active":"2024-04-26T03:38:49.512126+0000","last_became_peered":"2024-04-26T03:38:49.512126+0000","last_unstale":"2024-04-26T03:38:51.565572+0000","last_undegraded":"2024-04-26T03:38:51.565572+0000","last_fullsized":"2024-04-26T03:38:51.565572+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:56.709656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1228800,"num_objects":3,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":6,"num_write_kb":1200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2,10,9,15,8,4],"acting":[13,2,10,9,15,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.8","version":"35'6","reported_seq":38,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-26T03:38:51.811129+0000","last_change":"2024-04-26T03:38:49.488040+0000","last_active":"2024-04-26T03:38:51.811129+0000","last_peered":"2024-04-26T03:38:51.811129+0000","last_clean":"2024-04-26T03:38:51.811129+0000","last_became_active":"2024-04-26T03:38:49.487363+0000","last_became_peered":"2024-04-26T03:38:49.487363+0000","last_unstale":"2024-04-26T03:38:51.811129+0000","last_undegraded":"2024-04-26T03:38:51.811129+0000","last_fullsized":"2024-04-26T03:38:51.811129+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:02.514408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":851968,"num_objects":3,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":1926,"num_write":6,"num_write_kb":832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,10,11,15,5],"acting":[6,0,1,10,11,15,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"27'157","reported_seq":227,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-26T03:38:51.485873+0000","last_change":"2024-04-26T03:38:34.390206+0000","last_active":"2024-04-26T03:38:51.485873+0000","last_peered":"2024-04-26T03:38:51.485873+0000","last_clean":"2024-04-26T03:38:51.485873+0000","last_became_active":"2024-04-26T03:38:34.389099+0000","last_became_peered":"2024-04-26T03:38:34.389099+0000","last_unstale":"2024-04-26T03:38:51.485873+0000","last_undegraded":"2024-04-26T03:38:51.485873+0000","last_fullsized":"2024-04-26T03:38:51.485873+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:32.306810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:32.306810+0000","last_clean_scrub_stamp":"2024-04-26T03:38:32.306810+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:40.216975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":26,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-26T03:38:51.489028+0000","last_change":"2024-04-26T03:38:36.431667+0000","last_active":"2024-04-26T03:38:51.489028+0000","last_peered":"2024-04-26T03:38:51.489028+0000","last_clean":"2024-04-26T03:38:51.489028+0000","last_became_active":"2024-04-26T03:38:36.431449+0000","last_became_peered":"2024-04-26T03:38:36.431449+0000","last_unstale":"2024-04-26T03:38:51.489028+0000","last_undegraded":"2024-04-26T03:38:51.489028+0000","last_fullsized":"2024-04-26T03:38:51.489028+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:47.562648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.2","version":"35'11","reported_seq":36,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-26T03:38:51.851106+0000","last_change":"2024-04-26T03:38:49.512480+0000","last_active":"2024-04-26T03:38:51.851106+0000","last_peered":"2024-04-26T03:38:51.851106+0000","last_clean":"2024-04-26T03:38:51.851106+0000","last_became_active":"2024-04-26T03:38:49.511901+0000","last_became_peered":"2024-04-26T03:38:49.511901+0000","last_unstale":"2024-04-26T03:38:51.851106+0000","last_undegraded":"2024-04-26T03:38:51.851106+0000","last_fullsized":"2024-04-26T03:38:51.851106+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:09.283003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1097728,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":4,"num_write":11,"num_write_kb":2272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,5,3,9,15,10,11],"acting":[14,5,3,9,15,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-26T03:38:48.477124+0000","last_change":"2024-04-26T03:38:36.431325+0000","last_active":"2024-04-26T03:38:48.477124+0000","last_peered":"2024-04-26T03:38:48.477124+0000","last_clean":"2024-04-26T03:38:48.477124+0000","last_became_active":"2024-04-26T03:38:36.431121+0000","last_became_peered":"2024-04-26T03:38:36.431121+0000","last_unstale":"2024-04-26T03:38:48.477124+0000","last_undegraded":"2024-04-26T03:38:48.477124+0000","last_fullsized":"2024-04-26T03:38:48.477124+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:53.294619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T03:38:48.463214+0000","last_change":"2024-04-26T03:38:48.463214+0000","last_active":"2024-04-26T03:38:48.463214+0000","last_peered":"2024-04-26T03:38:48.463214+0000","last_clean":"2024-04-26T03:38:48.463214+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T03:38:48.463214+0000","last_undegraded":"2024-04-26T03:38:48.463214+0000","last_fullsized":"2024-04-26T03:38:48.463214+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-26T03:38:51.488749+0000","last_change":"2024-04-26T03:38:37.255450+0000","last_active":"2024-04-26T03:38:51.488749+0000","last_peered":"2024-04-26T03:38:51.488749+0000","last_clean":"2024-04-26T03:38:51.488749+0000","last_became_active":"2024-04-26T03:38:37.255181+0000","last_became_peered":"2024-04-26T03:38:37.255181+0000","last_unstale":"2024-04-26T03:38:51.488749+0000","last_undegraded":"2024-04-26T03:38:51.488749+0000","last_fullsized":"2024-04-26T03:38:51.488749+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:00.449388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0],"acting":[14,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T03:38:48.463214+0000","last_change":"2024-04-26T03:38:48.463214+0000","last_active":"2024-04-26T03:38:48.463214+0000","last_peered":"2024-04-26T03:38:48.463214+0000","last_clean":"2024-04-26T03:38:48.463214+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T03:38:48.463214+0000","last_undegraded":"2024-04-26T03:38:48.463214+0000","last_fullsized":"2024-04-26T03:38:48.463214+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T03:38:50.517859+0000","last_change":"2024-04-26T03:38:48.475569+0000","last_active":"2024-04-26T03:38:50.517859+0000","last_peered":"2024-04-26T03:38:50.517859+0000","last_clean":"2024-04-26T03:38:50.517859+0000","last_became_active":"2024-04-26T03:38:48.475155+0000","last_became_peered":"2024-04-26T03:38:48.475155+0000","last_unstale":"2024-04-26T03:38:50.517859+0000","last_undegraded":"2024-04-26T03:38:50.517859+0000","last_fullsized":"2024-04-26T03:38:50.517859+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:38.214686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,12],"acting":[11,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3","version":"35'14","reported_seq":40,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-26T03:38:51.830240+0000","last_change":"2024-04-26T03:38:49.487622+0000","last_active":"2024-04-26T03:38:51.830240+0000","last_peered":"2024-04-26T03:38:51.830240+0000","last_clean":"2024-04-26T03:38:51.830240+0000","last_became_active":"2024-04-26T03:38:49.486695+0000","last_became_peered":"2024-04-26T03:38:49.486695+0000","last_unstale":"2024-04-26T03:38:51.830240+0000","last_undegraded":"2024-04-26T03:38:51.830240+0000","last_fullsized":"2024-04-26T03:38:51.830240+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:18.153195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3555328,"num_objects":5,"num_object_clones":0,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":11,"num_read_kb":6,"num_write":14,"num_write_kb":3472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,7,4,15,1,2,3],"acting":[10,7,4,15,1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.5","version":"35'7","reported_seq":34,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-26T03:38:51.754828+0000","last_change":"2024-04-26T03:38:49.486797+0000","last_active":"2024-04-26T03:38:51.754828+0000","last_peered":"2024-04-26T03:38:51.754828+0000","last_clean":"2024-04-26T03:38:51.754828+0000","last_became_active":"2024-04-26T03:38:49.485980+0000","last_became_peered":"2024-04-26T03:38:49.485980+0000","last_unstale":"2024-04-26T03:38:51.754828+0000","last_undegraded":"2024-04-26T03:38:51.754828+0000","last_fullsized":"2024-04-26T03:38:51.754828+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:05.053965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1458176,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":1109,"num_write":7,"num_write_kb":1424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,8,4,2,7,0,5],"acting":[15,8,4,2,7,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-26T03:38:46.764425+0000","last_change":"2024-04-26T03:38:37.255389+0000","last_active":"2024-04-26T03:38:46.764425+0000","last_peered":"2024-04-26T03:38:46.764425+0000","last_clean":"2024-04-26T03:38:46.764425+0000","last_became_active":"2024-04-26T03:38:37.255066+0000","last_became_peered":"2024-04-26T03:38:37.255066+0000","last_unstale":"2024-04-26T03:38:46.764425+0000","last_undegraded":"2024-04-26T03:38:46.764425+0000","last_fullsized":"2024-04-26T03:38:46.764425+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:43.860480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"35'12","reported_seq":38,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-26T03:38:51.763040+0000","last_change":"2024-04-26T03:38:49.488190+0000","last_active":"2024-04-26T03:38:51.763040+0000","last_peered":"2024-04-26T03:38:51.763040+0000","last_clean":"2024-04-26T03:38:51.763040+0000","last_became_active":"2024-04-26T03:38:49.487567+0000","last_became_peered":"2024-04-26T03:38:49.487567+0000","last_unstale":"2024-04-26T03:38:51.763040+0000","last_undegraded":"2024-04-26T03:38:51.763040+0000","last_fullsized":"2024-04-26T03:38:51.763040+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:49.294680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1654784,"num_objects":4,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":11,"num_read_kb":5,"num_write":12,"num_write_kb":2464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,13,4,2,6,10,5],"acting":[8,13,4,2,6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":25,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-26T03:38:51.479742+0000","last_change":"2024-04-26T03:38:38.187714+0000","last_active":"2024-04-26T03:38:51.479742+0000","last_peered":"2024-04-26T03:38:51.479742+0000","last_clean":"2024-04-26T03:38:51.479742+0000","last_became_active":"2024-04-26T03:38:38.187383+0000","last_became_peered":"2024-04-26T03:38:38.187383+0000","last_unstale":"2024-04-26T03:38:51.479742+0000","last_undegraded":"2024-04-26T03:38:51.479742+0000","last_fullsized":"2024-04-26T03:38:51.479742+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:59.062623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4],"acting":[9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T03:38:48.463214+0000","last_change":"2024-04-26T03:38:48.463214+0000","last_active":"2024-04-26T03:38:48.463214+0000","last_peered":"2024-04-26T03:38:48.463214+0000","last_clean":"2024-04-26T03:38:48.463214+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T03:38:48.463214+0000","last_undegraded":"2024-04-26T03:38:48.463214+0000","last_fullsized":"2024-04-26T03:38:48.463214+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":26,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-26T03:38:51.479436+0000","last_change":"2024-04-26T03:38:36.484782+0000","last_active":"2024-04-26T03:38:51.479436+0000","last_peered":"2024-04-26T03:38:51.479436+0000","last_clean":"2024-04-26T03:38:51.479436+0000","last_became_active":"2024-04-26T03:38:36.484388+0000","last_became_peered":"2024-04-26T03:38:36.484388+0000","last_unstale":"2024-04-26T03:38:51.479436+0000","last_undegraded":"2024-04-26T03:38:51.479436+0000","last_fullsized":"2024-04-26T03:38:51.479436+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:36.318287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,13],"acting":[8,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T03:38:48.463214+0000","last_change":"2024-04-26T03:38:48.463214+0000","last_active":"2024-04-26T03:38:48.463214+0000","last_peered":"2024-04-26T03:38:48.463214+0000","last_clean":"2024-04-26T03:38:48.463214+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T03:38:48.463214+0000","last_undegraded":"2024-04-26T03:38:48.463214+0000","last_fullsized":"2024-04-26T03:38:48.463214+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":26,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-26T03:38:51.514479+0000","last_change":"2024-04-26T03:38:37.359252+0000","last_active":"2024-04-26T03:38:51.514479+0000","last_peered":"2024-04-26T03:38:51.514479+0000","last_clean":"2024-04-26T03:38:51.514479+0000","last_became_active":"2024-04-26T03:38:37.358910+0000","last_became_peered":"2024-04-26T03:38:37.358910+0000","last_unstale":"2024-04-26T03:38:51.514479+0000","last_undegraded":"2024-04-26T03:38:51.514479+0000","last_fullsized":"2024-04-26T03:38:51.514479+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:04.186001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":14270464,"num_objects":24,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":88,"num_read_kb":6520,"num_write":77,"num_write_kb":16272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":25694208,"data_stored":25694208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77,"ondisk_log_size":77,"up":77,"acting":77,"num_store_stats":11},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":13852,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":24,"seq":103079215112,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31516,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340324,"statfs":{"total":96636764160,"available":96604491776,"internally_reserved":0,"allocated":344064,"data_stored":665289,"data_compressed":20127,"data_compressed_allocated":61440,"data_compressed_original":569344,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[4,6,7,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215112,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":315392,"data_stored":157788,"data_compressed":17196,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,4,5,7,8,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":23,"seq":98784247817,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31464,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":290816,"data_stored":136361,"data_compressed":13201,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,7,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280519,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30664,"kb_used_data":3640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341176,"statfs":{"total":96636764160,"available":96605364224,"internally_reserved":0,"allocated":3727360,"data_stored":4039394,"data_compressed":30137,"data_compressed_allocated":81920,"data_compressed_original":610304,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,4,5,7,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280519,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30620,"kb_used_data":3592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341220,"statfs":{"total":96636764160,"available":96605409280,"internally_reserved":0,"allocated":3678208,"data_stored":3514562,"data_compressed":23211,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[4,6,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":22,"seq":94489280520,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":339968,"data_stored":178368,"data_compressed":21207,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[2,4,5,6,7,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280518,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":339968,"data_stored":178368,"data_compressed":21207,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,2,4,7,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280519,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":339968,"data_stored":178368,"data_compressed":21207,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[1,2,3,4,7,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313224,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":327680,"data_stored":168126,"data_compressed":19203,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,8,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":12,"up_from":22,"seq":94489280518,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29020,"kb_used_data":1992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342820,"statfs":{"total":96636764160,"available":96607047680,"internally_reserved":0,"allocated":2039808,"data_stored":1876162,"data_compressed":23211,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,7,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280518,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30868,"kb_used_data":3840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340972,"statfs":{"total":96636764160,"available":96605155328,"internally_reserved":0,"allocated":3932160,"data_stored":3768514,"data_compressed":23211,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[2,4,6,7,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247815,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31464,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":290816,"data_stored":136361,"data_compressed":13201,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,4,6,7,8,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280518,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28132,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343708,"statfs":{"total":96636764160,"available":96607956992,"internally_reserved":0,"allocated":1011712,"data_stored":848066,"data_compressed":23211,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,3,5,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":23,"seq":98784247817,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31460,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":278528,"data_stored":126699,"data_compressed":11242,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,4,6,7,8,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280518,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":339968,"data_stored":178368,"data_compressed":21207,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,4,5,7,8,9,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247815,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":136361,"data_compressed":13201,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3321856,"data_stored":3321856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3325952,"data_stored":3325952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":2252800,"data_stored":2252800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":2170880,"data_stored":2170880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4014080,"data_stored":4014080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":1687552,"data_stored":1687552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":3579904,"data_stored":3579904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":3039232,"data_stored":3039232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":2301952,"data_stored":2301952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T03:38:52.869 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-04-26T03:38:52.867+0000 7fdc2d21c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31819) UID: 0 2024-04-26T03:38:52.886 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:52.886 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:52.895 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-26T03:38:52.970 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:38:52.968+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:38:53.001 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:53.001 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:53.010 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-26T03:38:53.071 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-04-26T03:38:53.069+0000 7f745118d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 31821) UID: 0 2024-04-26T03:38:53.117 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:53.117 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:53.126 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-26T03:38:53.173 INFO:tasks.ceph.osd.9.smithi042.stderr:2024-04-26T03:38:53.171+0000 7fdac63b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 31960) UID: 0 2024-04-26T03:38:53.233 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:53.233 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:53.242 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-04-26T03:38:53.273 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-04-26T03:38:53.272+0000 7fe9223c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31954) UID: 0 2024-04-26T03:38:53.349 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:53.350 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:53.359 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-04-26T03:38:53.375 INFO:tasks.ceph.osd.15.smithi092.stderr:2024-04-26T03:38:53.373+0000 7f6d12dec640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 31824) UID: 0 2024-04-26T03:38:53.467 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:53.467 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:53.476 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-04-26T03:38:53.477 INFO:tasks.ceph.osd.10.smithi066.stderr:2024-04-26T03:38:53.474+0000 7f130fcc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 31763) UID: 0 2024-04-26T03:38:53.487 INFO:teuthology.orchestra.run.smithi028.stderr:noscrub is unset 2024-04-26T03:38:53.491 INFO:tasks.rados.rados.0.smithi092.stdout:85: done (0 left) 2024-04-26T03:38:53.491 INFO:tasks.rados.rados.0.smithi092.stdout:86: setattr oid 4 current snap is 2 2024-04-26T03:38:53.493 INFO:tasks.rados.rados.0.smithi092.stdout:87: read oid 17 snap 0 2024-04-26T03:38:53.493 INFO:tasks.rados.rados.0.smithi092.stdout:87: expect deleted 2024-04-26T03:38:53.493 INFO:tasks.rados.rados.0.smithi092.stdout:88: copy_from oid 40 from oid 42 current snap is 2 2024-04-26T03:38:53.493 INFO:tasks.rados.rados.0.smithi092.stdout:89: read oid 48 snap -1 2024-04-26T03:38:53.493 INFO:tasks.rados.rados.0.smithi092.stdout:89: expect (ObjNum 47 snap 0 seq_num 47) 2024-04-26T03:38:53.493 INFO:tasks.rados.rados.0.smithi092.stdout:90: rollback oid 23 current snap is 2 2024-04-26T03:38:53.493 INFO:tasks.rados.rados.0.smithi092.stdout:rollback oid 23 to 0 2024-04-26T03:38:53.493 INFO:tasks.rados.rados.0.smithi092.stdout:91: setattr oid 46 current snap is 2 2024-04-26T03:38:53.496 INFO:tasks.rados.rados.0.smithi092.stdout:87: done (5 left) 2024-04-26T03:38:53.496 INFO:tasks.rados.rados.0.smithi092.stdout:92: setattr oid 10 current snap is 2 2024-04-26T03:38:53.499 INFO:tasks.rados.rados.0.smithi092.stdout:93: read oid 47 snap -1 2024-04-26T03:38:53.499 INFO:tasks.rados.rados.0.smithi092.stdout:93: expect deleted 2024-04-26T03:38:53.499 INFO:tasks.rados.rados.0.smithi092.stdout:94: snap_create 2024-04-26T03:38:53.499 INFO:tasks.rados.rados.0.smithi092.stdout:89: expect (ObjNum 47 snap 0 seq_num 47) 2024-04-26T03:38:53.523 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 46 v 11 (ObjNum 56 snap 0 seq_num 56) dirty exists 2024-04-26T03:38:53.523 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 4 v 6 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-04-26T03:38:53.523 INFO:tasks.rados.rados.0.smithi092.stdout:90: finishing rollback tid 0 to smithi09235267-23 2024-04-26T03:38:53.523 INFO:tasks.rados.rados.0.smithi092.stdout:90: finishing rollback tid 1 to smithi09235267-23 2024-04-26T03:38:53.524 INFO:tasks.rados.rados.0.smithi092.stdout:90: finishing rollback tid 2 to smithi09235267-23 2024-04-26T03:38:53.524 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 23 v 11 (ObjNum 22 snap 0 seq_num 22) dirty exists 2024-04-26T03:38:53.524 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 10 v 13 (ObjNum 54 snap 0 seq_num 54) dirty exists 2024-04-26T03:38:53.527 INFO:tasks.rados.rados.0.smithi092.stdout:88: finishing copy_from racing read to smithi09235267-40 2024-04-26T03:38:53.531 INFO:tasks.rados.rados.0.smithi092.stdout:88: finishing copy_from to smithi09235267-40 2024-04-26T03:38:53.532 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 40 v 14 (ObjNum 60 snap 0 seq_num 60) dirty exists 2024-04-26T03:38:53.579 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-04-26T03:38:53.578+0000 7fee562fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31759) UID: 0 2024-04-26T03:38:53.593 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:53.593 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:53.603 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-04-26T03:38:53.681 INFO:tasks.ceph.osd.14.smithi066.stderr:2024-04-26T03:38:53.680+0000 7f5395f08640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 31765) UID: 0 2024-04-26T03:38:53.709 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:53.710 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:53.718 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-04-26T03:38:53.782 INFO:tasks.ceph.osd.15.smithi092.stderr:2024-04-26T03:38:53.781+0000 7f6d12dec640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 31824) UID: 0 2024-04-26T03:38:53.824 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:53.824 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:53.833 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-04-26T03:38:53.883 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:38:53.881+0000 7fba0d2ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31961) UID: 0 2024-04-26T03:38:53.913 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-26T03:38:53.941 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:53.941 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:53.951 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-26T03:38:53.992 INFO:tasks.ceph.osd.8.smithi028.stderr:2024-04-26T03:38:53.990+0000 7f5bb75d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 31968) UID: 0 2024-04-26T03:38:54.038 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0: {} 2024-04-26T03:38:54.038 INFO:teuthology.orchestra.run.smithi028.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T03:38:54.057 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1: {} 2024-04-26T03:38:54.057 INFO:teuthology.orchestra.run.smithi028.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T03:38:54.072 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-26T03:38:54.072 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-04-26T03:38:54.072 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-04-26T03:38:54.072 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-26T03:38:54.075 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2: {} 2024-04-26T03:38:54.075 INFO:teuthology.orchestra.run.smithi028.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T03:38:54.081 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-26T03:38:54.084 INFO:tasks.ceph.osd.12.smithi028.stderr:2024-04-26T03:38:54.082+0000 7f114fe14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 31967) UID: 0 2024-04-26T03:38:54.093 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3: {} 2024-04-26T03:38:54.093 INFO:teuthology.orchestra.run.smithi028.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T03:38:54.110 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4: {} 2024-04-26T03:38:54.110 INFO:teuthology.orchestra.run.smithi028.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T03:38:54.128 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5: {} 2024-04-26T03:38:54.128 INFO:teuthology.orchestra.run.smithi028.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T03:38:54.146 INFO:teuthology.orchestra.run.smithi028.stdout:osd.6: {} 2024-04-26T03:38:54.147 INFO:teuthology.orchestra.run.smithi028.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T03:38:54.164 INFO:teuthology.orchestra.run.smithi028.stdout:osd.7: {} 2024-04-26T03:38:54.164 INFO:teuthology.orchestra.run.smithi028.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T03:38:54.181 INFO:teuthology.orchestra.run.smithi028.stdout:osd.8: {} 2024-04-26T03:38:54.181 INFO:teuthology.orchestra.run.smithi028.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T03:38:54.185 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-04-26T03:38:54.184+0000 7fdc2d21c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31819) UID: 0 2024-04-26T03:38:54.189 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-26T03:38:54.189 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-04-26T03:38:54.189 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-04-26T03:38:54.189 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-04-26T03:38:54.190 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-26T03:38:54.199 INFO:teuthology.orchestra.run.smithi028.stdout:osd.9: {} 2024-04-26T03:38:54.199 INFO:teuthology.orchestra.run.smithi028.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T03:38:54.200 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-26T03:38:54.216 INFO:teuthology.orchestra.run.smithi028.stdout:osd.10: {} 2024-04-26T03:38:54.216 INFO:teuthology.orchestra.run.smithi028.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T03:38:54.234 INFO:teuthology.orchestra.run.smithi028.stdout:osd.11: {} 2024-04-26T03:38:54.234 INFO:teuthology.orchestra.run.smithi028.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T03:38:54.251 INFO:teuthology.orchestra.run.smithi028.stdout:osd.12: {} 2024-04-26T03:38:54.251 INFO:teuthology.orchestra.run.smithi028.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T03:38:54.268 INFO:teuthology.orchestra.run.smithi028.stdout:osd.13: {} 2024-04-26T03:38:54.269 INFO:teuthology.orchestra.run.smithi028.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T03:38:54.286 INFO:teuthology.orchestra.run.smithi028.stdout:osd.14: {} 2024-04-26T03:38:54.286 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-26T03:38:54.284+0000 7f610b3a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31816) UID: 0 2024-04-26T03:38:54.287 INFO:teuthology.orchestra.run.smithi028.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T03:38:54.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:38:54.286+0000 7fbeda7d6640 0 -- 172.21.15.28:0/2380683910 >> [v2:172.21.15.92:6826/3982634323,v1:172.21.15.92:6827/3982634323] conn(0x7fbed4220c00 msgr2=0x7fbed4222ff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-26T03:38:54.306 INFO:teuthology.orchestra.run.smithi028.stdout:osd.15: {} 2024-04-26T03:38:54.306 INFO:teuthology.orchestra.run.smithi028.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T03:38:54.309 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-26T03:38:54.309 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-04-26T03:38:54.309 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-04-26T03:38:54.309 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-04-26T03:38:54.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.309 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:13 3.0s0 3:0a8403ef:::smithi09235267-5:head [append 0~81920,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:38:54.310 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.484308+0000", 2024-04-26T03:38:54.310 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 2.8217715170000002, 2024-04-26T03:38:54.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017722486999999999, 2024-04-26T03:38:54.310 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:54.310 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:54.310 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:38:54.310 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:38:54.310 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:38:54.310 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:38:54.310 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 13 2024-04-26T03:38:54.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.310 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:54.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:54.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484308+0000", 2024-04-26T03:38:54.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:54.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484308+0000", 2024-04-26T03:38:54.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:54.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484311+0000", 2024-04-26T03:38:54.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8890000000000002e-06 2024-04-26T03:38:54.312 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.312 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.312 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:54.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484792+0000", 2024-04-26T03:38:54.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00048166600000000001 2024-04-26T03:38:54.312 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.312 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.312 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:54.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484806+0000", 2024-04-26T03:38:54.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3709999999999999e-05 2024-04-26T03:38:54.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.313 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.313 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:54.313 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484816+0000", 2024-04-26T03:38:54.313 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7000000000000003e-06 2024-04-26T03:38:54.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.313 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.313 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:54.313 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484975+0000", 2024-04-26T03:38:54.313 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015943199999999999 2024-04-26T03:38:54.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.314 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:38:54.314 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.485147+0000", 2024-04-26T03:38:54.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000171888 2024-04-26T03:38:54.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.314 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:54.314 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.486224+0000", 2024-04-26T03:38:54.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010766720000000001 2024-04-26T03:38:54.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:54.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.489541+0000", 2024-04-26T03:38:54.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033176210000000002 2024-04-26T03:38:54.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:38:54.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.501934+0000", 2024-04-26T03:38:54.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012392633 2024-04-26T03:38:54.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:54.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.502030+0000", 2024-04-26T03:38:54.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6275999999999998e-05 2024-04-26T03:38:54.316 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.316 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:54.316 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.316 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:14 3.0s0 3:0a8403ef:::smithi09235267-5:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:38:54.316 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.484901+0000", 2024-04-26T03:38:54.316 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 2.8211784099999999, 2024-04-26T03:38:54.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017354086000000001, 2024-04-26T03:38:54.316 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:54.316 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:54.316 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:38:54.317 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:38:54.317 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:38:54.317 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:38:54.317 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 14 2024-04-26T03:38:54.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.317 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:54.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.317 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:54.317 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484901+0000", 2024-04-26T03:38:54.317 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.317 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:54.318 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484901+0000", 2024-04-26T03:38:54.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.318 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:54.318 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484902+0000", 2024-04-26T03:38:54.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1069999999999999e-06 2024-04-26T03:38:54.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.319 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:54.319 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484909+0000", 2024-04-26T03:38:54.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.323e-06 2024-04-26T03:38:54.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.319 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:54.319 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484919+0000", 2024-04-26T03:38:54.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0069e-05 2024-04-26T03:38:54.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.319 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:54.320 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484930+0000", 2024-04-26T03:38:54.320 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1092e-05 2024-04-26T03:38:54.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.320 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:54.320 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.487836+0000", 2024-04-26T03:38:54.320 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0029053569999999999 2024-04-26T03:38:54.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.320 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:38:54.320 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.487949+0000", 2024-04-26T03:38:54.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000112822 2024-04-26T03:38:54.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.321 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:54.321 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.488660+0000", 2024-04-26T03:38:54.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00071102899999999996 2024-04-26T03:38:54.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.321 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:54.322 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.490973+0000", 2024-04-26T03:38:54.322 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023132510000000001 2024-04-26T03:38:54.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.322 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:38:54.322 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.502166+0000", 2024-04-26T03:38:54.322 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011193639999999999 2024-04-26T03:38:54.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.322 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:54.322 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.502255+0000", 2024-04-26T03:38:54.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8395999999999999e-05 2024-04-26T03:38:54.323 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.323 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:54.323 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.323 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:15 3.0s0 3:0a8403ef:::smithi09235267-5:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:38:54.323 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.484947+0000", 2024-04-26T03:38:54.323 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 2.8211326369999998, 2024-04-26T03:38:54.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024468274000000002, 2024-04-26T03:38:54.324 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:54.324 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:54.324 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-04-26T03:38:54.324 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:38:54.324 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:38:54.324 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:38:54.324 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15 2024-04-26T03:38:54.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.324 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:54.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:54.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484947+0000", 2024-04-26T03:38:54.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:54.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484947+0000", 2024-04-26T03:38:54.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.326 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:54.326 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484947+0000", 2024-04-26T03:38:54.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0999999999999997e-07 2024-04-26T03:38:54.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.326 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:54.326 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484952+0000", 2024-04-26T03:38:54.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.549e-06 2024-04-26T03:38:54.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.327 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:54.327 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484962+0000", 2024-04-26T03:38:54.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0064999999999999e-05 2024-04-26T03:38:54.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.327 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:54.327 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484972+0000", 2024-04-26T03:38:54.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7529999999999996e-06 2024-04-26T03:38:54.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.328 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:54.328 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.489714+0000", 2024-04-26T03:38:54.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0047418649999999996 2024-04-26T03:38:54.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.328 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-04-26T03:38:54.328 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.489836+0000", 2024-04-26T03:38:54.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012253399999999999 2024-04-26T03:38:54.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:54.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.502300+0000", 2024-04-26T03:38:54.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012464098 2024-04-26T03:38:54.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:38:54.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.502390+0000", 2024-04-26T03:38:54.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9954e-05 2024-04-26T03:38:54.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:54.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.509415+0000", 2024-04-26T03:38:54.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0070245459999999996 2024-04-26T03:38:54.330 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.330 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:54.330 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.330 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:40 3.0s0 3:0e9b8628:::smithi09235267-14:head [append 0~229376,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:38:54.331 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.495863+0000", 2024-04-26T03:38:54.331 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 2.810216144, 2024-04-26T03:38:54.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013720777999999999, 2024-04-26T03:38:54.331 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:54.331 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:54.331 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:38:54.331 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:38:54.331 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:38:54.332 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:38:54.332 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 40 2024-04-26T03:38:54.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.332 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:54.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:54.332 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.495863+0000", 2024-04-26T03:38:54.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.333 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:54.333 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.495863+0000", 2024-04-26T03:38:54.333 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.333 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:54.334 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.495865+0000", 2024-04-26T03:38:54.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4090000000000001e-06 2024-04-26T03:38:54.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.334 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:54.334 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496307+0000", 2024-04-26T03:38:54.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00044159800000000002 2024-04-26T03:38:54.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.334 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:54.334 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496322+0000", 2024-04-26T03:38:54.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4935e-05 2024-04-26T03:38:54.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.335 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:54.335 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496338+0000", 2024-04-26T03:38:54.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6155e-05 2024-04-26T03:38:54.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.335 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:54.335 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496408+0000", 2024-04-26T03:38:54.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0183000000000002e-05 2024-04-26T03:38:54.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:38:54.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496716+0000", 2024-04-26T03:38:54.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000308024 2024-04-26T03:38:54.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:54.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.497681+0000", 2024-04-26T03:38:54.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00096483300000000001 2024-04-26T03:38:54.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.337 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:54.337 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.505243+0000", 2024-04-26T03:38:54.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0075613329999999999 2024-04-26T03:38:54.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.337 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:38:54.337 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.509516+0000", 2024-04-26T03:38:54.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0042732480000000003 2024-04-26T03:38:54.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.337 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:54.337 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.509584+0000", 2024-04-26T03:38:54.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8059999999999996e-05 2024-04-26T03:38:54.338 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.338 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:54.338 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.338 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:41 3.0s0 3:0e9b8628:::smithi09235267-14:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:38:54.338 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.496478+0000", 2024-04-26T03:38:54.338 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 2.809600884, 2024-04-26T03:38:54.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018225094000000001, 2024-04-26T03:38:54.338 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:54.338 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:54.339 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:38:54.339 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:38:54.339 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:38:54.339 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:38:54.339 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 41 2024-04-26T03:38:54.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.339 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:54.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.339 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:54.339 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496478+0000", 2024-04-26T03:38:54.339 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.340 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:54.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496478+0000", 2024-04-26T03:38:54.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.340 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:54.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496479+0000", 2024-04-26T03:38:54.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7700000000000003e-07 2024-04-26T03:38:54.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.340 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:54.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496563+0000", 2024-04-26T03:38:54.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3778999999999999e-05 2024-04-26T03:38:54.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:54.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496574+0000", 2024-04-26T03:38:54.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1144999999999999e-05 2024-04-26T03:38:54.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:54.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496586+0000", 2024-04-26T03:38:54.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1457999999999999e-05 2024-04-26T03:38:54.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.342 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:54.342 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.503975+0000", 2024-04-26T03:38:54.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0073896480000000004 2024-04-26T03:38:54.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.342 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:38:54.342 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504089+0000", 2024-04-26T03:38:54.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011359299999999999 2024-04-26T03:38:54.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.342 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:54.342 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504968+0000", 2024-04-26T03:38:54.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00087954500000000002 2024-04-26T03:38:54.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:54.343 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.508613+0000", 2024-04-26T03:38:54.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0036446500000000001 2024-04-26T03:38:54.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:38:54.343 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.514613+0000", 2024-04-26T03:38:54.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0060000000000000001 2024-04-26T03:38:54.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:54.343 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.514703+0000", 2024-04-26T03:38:54.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0399e-05 2024-04-26T03:38:54.344 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.344 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:54.344 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.344 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:42 3.0s0 3:0e9b8628:::smithi09235267-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:38:54.344 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.496943+0000", 2024-04-26T03:38:54.344 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 2.8091361080000001, 2024-04-26T03:38:54.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021178255999999999, 2024-04-26T03:38:54.344 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:54.344 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:54.344 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-04-26T03:38:54.344 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:38:54.344 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:38:54.345 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:38:54.345 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 42 2024-04-26T03:38:54.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.345 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:54.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:54.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496943+0000", 2024-04-26T03:38:54.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:54.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496943+0000", 2024-04-26T03:38:54.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:54.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496944+0000", 2024-04-26T03:38:54.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9000000000000005e-07 2024-04-26T03:38:54.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:54.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496951+0000", 2024-04-26T03:38:54.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6470000000000001e-06 2024-04-26T03:38:54.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:54.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496962+0000", 2024-04-26T03:38:54.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1618e-05 2024-04-26T03:38:54.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:54.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496971+0000", 2024-04-26T03:38:54.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6170000000000007e-06 2024-04-26T03:38:54.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:54.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.505538+0000", 2024-04-26T03:38:54.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0085666530000000005 2024-04-26T03:38:54.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-04-26T03:38:54.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.505620+0000", 2024-04-26T03:38:54.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2669000000000004e-05 2024-04-26T03:38:54.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:54.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.515261+0000", 2024-04-26T03:38:54.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0096405210000000009 2024-04-26T03:38:54.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:38:54.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.515342+0000", 2024-04-26T03:38:54.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1642000000000002e-05 2024-04-26T03:38:54.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:54.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.518121+0000", 2024-04-26T03:38:54.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002778899 2024-04-26T03:38:54.349 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.349 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:54.349 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.349 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 35/32 ECSubWriteReply(tid=1, last_complete=34'2, committed=1, applied=1))", 2024-04-26T03:38:54.349 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.497082+0000", 2024-04-26T03:38:54.349 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 2.8089975680000001, 2024-04-26T03:38:54.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0051841459999999997, 2024-04-26T03:38:54.350 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:54.350 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:54.350 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:54.350 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:54.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:54.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.497082+0000", 2024-04-26T03:38:54.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:54.351 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.497082+0000", 2024-04-26T03:38:54.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:54.351 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.497082+0000", 2024-04-26T03:38:54.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2099999999999998e-07 2024-04-26T03:38:54.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:54.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.497085+0000", 2024-04-26T03:38:54.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0249999999999998e-06 2024-04-26T03:38:54.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.352 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:54.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.497093+0000", 2024-04-26T03:38:54.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6450000000000005e-06 2024-04-26T03:38:54.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.352 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:54.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.497098+0000", 2024-04-26T03:38:54.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1780000000000002e-06 2024-04-26T03:38:54.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:54.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.502095+0000", 2024-04-26T03:38:54.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0049970919999999999 2024-04-26T03:38:54.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:54.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.502266+0000", 2024-04-26T03:38:54.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017088499999999999 2024-04-26T03:38:54.353 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.353 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:54.354 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.354 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds3 35/32 ECSubWrite(tid=0, reqid=client.4556.0:43, at_version=35'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:38:54.354 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.499996+0000", 2024-04-26T03:38:54.354 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 2.8060829419999997, 2024-04-26T03:38:54.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0057023960000000002, 2024-04-26T03:38:54.354 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:54.354 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:54.354 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:54.354 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:54.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.355 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:54.355 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.499996+0000", 2024-04-26T03:38:54.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.355 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:54.355 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.499996+0000", 2024-04-26T03:38:54.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.356 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.356 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:54.356 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.499997+0000", 2024-04-26T03:38:54.356 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.89e-07 2024-04-26T03:38:54.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.356 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.356 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:54.356 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.500102+0000", 2024-04-26T03:38:54.356 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000105369 2024-04-26T03:38:54.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.357 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.357 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:54.357 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.500120+0000", 2024-04-26T03:38:54.357 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7813e-05 2024-04-26T03:38:54.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.357 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.357 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:54.357 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.500130+0000", 2024-04-26T03:38:54.357 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6229999999999992e-06 2024-04-26T03:38:54.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:54.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.500195+0000", 2024-04-26T03:38:54.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5394000000000004e-05 2024-04-26T03:38:54.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:54.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.500203+0000", 2024-04-26T03:38:54.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2530000000000004e-06 2024-04-26T03:38:54.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:54.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.505663+0000", 2024-04-26T03:38:54.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0054600910000000002 2024-04-26T03:38:54.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:54.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.505699+0000", 2024-04-26T03:38:54.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5364000000000002e-05 2024-04-26T03:38:54.359 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.359 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:54.360 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.360 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.bs4 35/32 ECSubWrite(tid=6, reqid=client.4556.0:37, at_version=35'3, trim_to=0'0, roll_forward_to=34'2))", 2024-04-26T03:38:54.360 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.501702+0000", 2024-04-26T03:38:54.360 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 2.8043773089999999, 2024-04-26T03:38:54.360 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0052680410000000002, 2024-04-26T03:38:54.360 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:54.360 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:54.360 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:54.361 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:54.361 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.361 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:54.361 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.501702+0000", 2024-04-26T03:38:54.361 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.361 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.361 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:54.361 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.501702+0000", 2024-04-26T03:38:54.361 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.362 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:54.362 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.501703+0000", 2024-04-26T03:38:54.362 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2799999999999995e-07 2024-04-26T03:38:54.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.362 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:54.362 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.501876+0000", 2024-04-26T03:38:54.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000173318 2024-04-26T03:38:54.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:54.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.501888+0000", 2024-04-26T03:38:54.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2114e-05 2024-04-26T03:38:54.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:54.364 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.501894+0000", 2024-04-26T03:38:54.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.964e-06 2024-04-26T03:38:54.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:54.364 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.501951+0000", 2024-04-26T03:38:54.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6764000000000001e-05 2024-04-26T03:38:54.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:54.365 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.501961+0000", 2024-04-26T03:38:54.365 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7319999999999993e-06 2024-04-26T03:38:54.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.365 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.365 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:54.365 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.506931+0000", 2024-04-26T03:38:54.365 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0049704759999999997 2024-04-26T03:38:54.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.365 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.366 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:54.366 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.506970+0000", 2024-04-26T03:38:54.366 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8844999999999998e-05 2024-04-26T03:38:54.366 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.366 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:54.366 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.366 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.366 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.366 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds3 35/32 ECSubWrite(tid=1, reqid=client.4556.0:44, at_version=35'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:38:54.367 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.504056+0000", 2024-04-26T03:38:54.367 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 2.80202302, 2024-04-26T03:38:54.367 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0085269190000000009, 2024-04-26T03:38:54.367 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:54.367 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:54.367 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:54.367 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:54.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.367 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:54.367 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504056+0000", 2024-04-26T03:38:54.367 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.367 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:54.368 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504056+0000", 2024-04-26T03:38:54.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:54.368 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504057+0000", 2024-04-26T03:38:54.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2600000000000001e-07 2024-04-26T03:38:54.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:54.368 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504078+0000", 2024-04-26T03:38:54.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1250000000000002e-05 2024-04-26T03:38:54.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:54.369 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504092+0000", 2024-04-26T03:38:54.369 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3370000000000001e-05 2024-04-26T03:38:54.369 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.369 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.369 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:54.369 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504102+0000", 2024-04-26T03:38:54.369 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0047000000000001e-05 2024-04-26T03:38:54.369 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.369 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.369 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:54.370 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504408+0000", 2024-04-26T03:38:54.370 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030663799999999997 2024-04-26T03:38:54.370 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.370 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.370 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:54.370 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504417+0000", 2024-04-26T03:38:54.370 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8400000000000001e-06 2024-04-26T03:38:54.370 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.370 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.371 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:54.371 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.512551+0000", 2024-04-26T03:38:54.371 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0081339870000000005 2024-04-26T03:38:54.371 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.371 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.371 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:54.371 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.512583+0000", 2024-04-26T03:38:54.371 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1961000000000001e-05 2024-04-26T03:38:54.371 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.371 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:54.371 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.371 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.371 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.371 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds3 35/32 ECSubWrite(tid=2, reqid=client.4556.0:52, at_version=35'3, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:38:54.372 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.506681+0000", 2024-04-26T03:38:54.372 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 2.7993978369999999, 2024-04-26T03:38:54.372 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0070048899999999997, 2024-04-26T03:38:54.372 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:54.372 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:54.372 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:54.372 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:54.372 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.372 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:54.372 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.506681+0000", 2024-04-26T03:38:54.372 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.372 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.372 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.372 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:54.373 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.506681+0000", 2024-04-26T03:38:54.373 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.373 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:54.373 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.506682+0000", 2024-04-26T03:38:54.373 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.04e-06 2024-04-26T03:38:54.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.373 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:54.373 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.506853+0000", 2024-04-26T03:38:54.373 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017012599999999999 2024-04-26T03:38:54.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.374 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:54.374 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.506866+0000", 2024-04-26T03:38:54.374 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3244999999999999e-05 2024-04-26T03:38:54.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.374 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.374 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:54.374 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.506874+0000", 2024-04-26T03:38:54.374 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0229999999999992e-06 2024-04-26T03:38:54.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.374 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.374 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:54.374 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.508635+0000", 2024-04-26T03:38:54.374 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0017609380000000001 2024-04-26T03:38:54.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.375 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.375 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:54.375 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.508644+0000", 2024-04-26T03:38:54.375 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0569999999999999e-06 2024-04-26T03:38:54.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.375 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.375 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:54.375 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.513648+0000", 2024-04-26T03:38:54.375 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0050038920000000002 2024-04-26T03:38:54.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.375 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.375 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:54.375 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.513686+0000", 2024-04-26T03:38:54.376 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8569000000000003e-05 2024-04-26T03:38:54.376 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.376 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:54.376 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.376 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.376 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 35/32 ECSubWriteReply(tid=4, last_complete=35'4, committed=1, applied=1))", 2024-04-26T03:38:54.376 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.507927+0000", 2024-04-26T03:38:54.376 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 2.7981519750000001, 2024-04-26T03:38:54.376 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0053162010000000004, 2024-04-26T03:38:54.376 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:54.376 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:54.376 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:54.376 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:54.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.377 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:54.377 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.507927+0000", 2024-04-26T03:38:54.377 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.377 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:54.377 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.507927+0000", 2024-04-26T03:38:54.377 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.377 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:54.377 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.507928+0000", 2024-04-26T03:38:54.377 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.083e-06 2024-04-26T03:38:54.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.378 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.378 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:54.378 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.507938+0000", 2024-04-26T03:38:54.378 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7880000000000001e-06 2024-04-26T03:38:54.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.378 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.378 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:54.378 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.512598+0000", 2024-04-26T03:38:54.378 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0046601029999999996 2024-04-26T03:38:54.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.379 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.379 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:54.379 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.512607+0000", 2024-04-26T03:38:54.379 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.619e-06 2024-04-26T03:38:54.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.379 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.379 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:54.379 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.513211+0000", 2024-04-26T03:38:54.379 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00060419900000000003 2024-04-26T03:38:54.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.379 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.379 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:54.380 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.513243+0000", 2024-04-26T03:38:54.380 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2409000000000001e-05 2024-04-26T03:38:54.380 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.380 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:54.380 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.380 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.380 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.8s2 35/32 ECSubWrite(tid=161, reqid=client.4556.0:55, at_version=35'3, trim_to=0'0, roll_forward_to=34'2))", 2024-04-26T03:38:54.380 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.509799+0000", 2024-04-26T03:38:54.380 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 2.7962803649999999, 2024-04-26T03:38:54.380 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.005403705, 2024-04-26T03:38:54.380 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:54.380 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:54.380 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:54.381 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:54.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:54.381 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.509799+0000", 2024-04-26T03:38:54.381 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:54.381 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.509799+0000", 2024-04-26T03:38:54.381 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:54.382 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.509800+0000", 2024-04-26T03:38:54.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5300000000000003e-07 2024-04-26T03:38:54.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.382 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:54.382 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.509859+0000", 2024-04-26T03:38:54.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9777e-05 2024-04-26T03:38:54.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.382 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:54.382 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.509873+0000", 2024-04-26T03:38:54.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3259e-05 2024-04-26T03:38:54.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.383 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.383 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:54.383 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.509881+0000", 2024-04-26T03:38:54.383 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0239999999999997e-06 2024-04-26T03:38:54.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.383 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.383 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:54.383 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.510016+0000", 2024-04-26T03:38:54.383 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000135809 2024-04-26T03:38:54.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.383 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.383 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:54.383 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.510037+0000", 2024-04-26T03:38:54.383 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0395e-05 2024-04-26T03:38:54.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.384 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.384 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:54.384 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.515171+0000", 2024-04-26T03:38:54.384 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0051341199999999998 2024-04-26T03:38:54.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.384 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.384 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:54.384 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.515203+0000", 2024-04-26T03:38:54.384 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1568e-05 2024-04-26T03:38:54.384 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.384 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:54.384 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.385 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.385 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.bs4 35/32 ECSubWrite(tid=10, reqid=client.4556.0:61, at_version=35'5, trim_to=0'0, roll_forward_to=35'3))", 2024-04-26T03:38:54.385 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.514897+0000", 2024-04-26T03:38:54.385 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 2.7911823870000001, 2024-04-26T03:38:54.385 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0051308029999999998, 2024-04-26T03:38:54.385 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:54.385 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:54.385 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:54.385 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:54.385 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.385 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:54.385 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.514897+0000", 2024-04-26T03:38:54.385 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.385 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.386 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:54.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.514897+0000", 2024-04-26T03:38:54.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.386 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:54.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.514898+0000", 2024-04-26T03:38:54.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.111e-06 2024-04-26T03:38:54.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.386 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:54.387 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.514987+0000", 2024-04-26T03:38:54.387 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9265999999999995e-05 2024-04-26T03:38:54.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.387 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:54.387 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.515001+0000", 2024-04-26T03:38:54.387 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4280999999999999e-05 2024-04-26T03:38:54.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.387 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:54.387 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.515011+0000", 2024-04-26T03:38:54.387 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0420000000000008e-06 2024-04-26T03:38:54.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.388 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:54.388 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.515088+0000", 2024-04-26T03:38:54.388 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7919999999999999e-05 2024-04-26T03:38:54.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.388 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.388 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:54.388 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.515098+0000", 2024-04-26T03:38:54.388 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2809999999999997e-06 2024-04-26T03:38:54.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.388 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.388 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:54.388 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.519981+0000", 2024-04-26T03:38:54.389 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0048836410000000002 2024-04-26T03:38:54.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.389 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.389 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:54.389 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.520028+0000", 2024-04-26T03:38:54.389 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6261000000000003e-05 2024-04-26T03:38:54.389 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.389 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:54.389 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.389 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.389 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.as5 35/32 ECSubWrite(tid=4, reqid=client.4556.0:73, at_version=35'3, trim_to=0'0, roll_forward_to=34'2))", 2024-04-26T03:38:54.389 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.525773+0000", 2024-04-26T03:38:54.389 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 2.780306639, 2024-04-26T03:38:54.390 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0073138420000000001, 2024-04-26T03:38:54.390 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:54.390 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:54.390 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:54.390 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:54.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.390 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:54.390 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.525773+0000", 2024-04-26T03:38:54.390 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.390 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:54.390 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.525773+0000", 2024-04-26T03:38:54.390 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.391 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:54.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.525773+0000", 2024-04-26T03:38:54.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0699999999999996e-07 2024-04-26T03:38:54.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.391 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:54.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.525972+0000", 2024-04-26T03:38:54.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019857100000000001 2024-04-26T03:38:54.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.391 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:54.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.525984+0000", 2024-04-26T03:38:54.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2099999999999999e-05 2024-04-26T03:38:54.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.392 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.392 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:54.392 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.525990+0000", 2024-04-26T03:38:54.392 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7320000000000003e-06 2024-04-26T03:38:54.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.392 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.392 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:54.392 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.526063+0000", 2024-04-26T03:38:54.392 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3499999999999998e-05 2024-04-26T03:38:54.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.392 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.393 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:54.393 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.526074+0000", 2024-04-26T03:38:54.393 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0899e-05 2024-04-26T03:38:54.393 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.393 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.393 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:54.393 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.533045+0000", 2024-04-26T03:38:54.393 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0069703400000000002 2024-04-26T03:38:54.393 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.393 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.393 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:54.393 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.533086+0000", 2024-04-26T03:38:54.393 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1893e-05 2024-04-26T03:38:54.394 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.394 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:54.394 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.394 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds3 35/32 ECSubWrite(tid=7, reqid=client.4556.0:64, at_version=35'5, trim_to=0'0, roll_forward_to=35'4))", 2024-04-26T03:38:54.394 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.528525+0000", 2024-04-26T03:38:54.394 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 2.7775546759999998, 2024-04-26T03:38:54.394 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.006326946, 2024-04-26T03:38:54.394 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:54.394 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:54.394 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:54.395 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:54.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:54.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528525+0000", 2024-04-26T03:38:54.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:54.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528525+0000", 2024-04-26T03:38:54.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:54.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528526+0000", 2024-04-26T03:38:54.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9199999999999999e-07 2024-04-26T03:38:54.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:54.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528649+0000", 2024-04-26T03:38:54.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012333600000000001 2024-04-26T03:38:54.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:54.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.529481+0000", 2024-04-26T03:38:54.397 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00083257199999999996 2024-04-26T03:38:54.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.397 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.397 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:54.397 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.529492+0000", 2024-04-26T03:38:54.397 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0769e-05 2024-04-26T03:38:54.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.397 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.397 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:54.397 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.529566+0000", 2024-04-26T03:38:54.397 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3756000000000003e-05 2024-04-26T03:38:54.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.398 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:54.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.529578+0000", 2024-04-26T03:38:54.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2513e-05 2024-04-26T03:38:54.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.398 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:54.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.534815+0000", 2024-04-26T03:38:54.402 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0052369510000000001 2024-04-26T03:38:54.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.402 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:54.402 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.534851+0000", 2024-04-26T03:38:54.402 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6056999999999997e-05 2024-04-26T03:38:54.403 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.403 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:54.403 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.403 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.3s4 35/32 ECSubWrite(tid=4, reqid=client.4556.0:88, at_version=35'3, trim_to=0'0, roll_forward_to=34'2))", 2024-04-26T03:38:54.403 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.545174+0000", 2024-04-26T03:38:54.403 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 2.7609049699999999, 2024-04-26T03:38:54.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0052690339999999997, 2024-04-26T03:38:54.403 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:54.403 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:54.404 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:54.404 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:54.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:54.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.545174+0000", 2024-04-26T03:38:54.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:54.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.545174+0000", 2024-04-26T03:38:54.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:54.405 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.545176+0000", 2024-04-26T03:38:54.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3009999999999999e-06 2024-04-26T03:38:54.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:54.405 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.545582+0000", 2024-04-26T03:38:54.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00040631100000000002 2024-04-26T03:38:54.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.406 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:54.406 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.545596+0000", 2024-04-26T03:38:54.406 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4365000000000001e-05 2024-04-26T03:38:54.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.406 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:54.406 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.545607+0000", 2024-04-26T03:38:54.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0774999999999999e-05 2024-04-26T03:38:54.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.407 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:54.407 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.545680+0000", 2024-04-26T03:38:54.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2818999999999995e-05 2024-04-26T03:38:54.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.407 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:54.407 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.545692+0000", 2024-04-26T03:38:54.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1769e-05 2024-04-26T03:38:54.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:54.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.550405+0000", 2024-04-26T03:38:54.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0047133890000000001 2024-04-26T03:38:54.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:54.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.550443+0000", 2024-04-26T03:38:54.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8305e-05 2024-04-26T03:38:54.408 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.408 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:54.408 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.408 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.as5 35/32 ECSubWrite(tid=12, reqid=client.4556.0:115, at_version=35'7, trim_to=0'0, roll_forward_to=35'6))", 2024-04-26T03:38:54.409 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.579868+0000", 2024-04-26T03:38:54.409 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 2.7262110110000002, 2024-04-26T03:38:54.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0052991979999999998, 2024-04-26T03:38:54.409 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:54.409 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:54.409 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:54.409 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:54.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:54.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.579868+0000", 2024-04-26T03:38:54.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:54.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.579868+0000", 2024-04-26T03:38:54.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.410 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:54.410 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.579869+0000", 2024-04-26T03:38:54.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4600000000000003e-07 2024-04-26T03:38:54.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.410 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:54.410 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.580319+0000", 2024-04-26T03:38:54.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00045016799999999998 2024-04-26T03:38:54.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.410 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:54.411 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.580332+0000", 2024-04-26T03:38:54.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2639e-05 2024-04-26T03:38:54.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.411 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:54.411 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.580340+0000", 2024-04-26T03:38:54.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8480000000000001e-06 2024-04-26T03:38:54.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.411 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:54.411 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.580399+0000", 2024-04-26T03:38:54.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9122000000000001e-05 2024-04-26T03:38:54.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.412 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:54.412 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.580404+0000", 2024-04-26T03:38:54.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6699999999999999e-06 2024-04-26T03:38:54.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.412 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:54.412 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.585141+0000", 2024-04-26T03:38:54.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0047369270000000002 2024-04-26T03:38:54.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.412 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:54.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.585167+0000", 2024-04-26T03:38:54.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5978000000000001e-05 2024-04-26T03:38:54.413 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.413 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:54.413 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.413 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.bs4 35/32 ECSubWrite(tid=20, reqid=client.4556.0:127, at_version=35'7, trim_to=0'0, roll_forward_to=35'6))", 2024-04-26T03:38:54.413 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.601259+0000", 2024-04-26T03:38:54.413 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 2.7048199579999999, 2024-04-26T03:38:54.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0054673129999999997, 2024-04-26T03:38:54.413 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:54.413 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:54.413 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:54.414 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:54.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:54.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.601259+0000", 2024-04-26T03:38:54.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:54.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.601259+0000", 2024-04-26T03:38:54.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:54.415 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.601260+0000", 2024-04-26T03:38:54.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3900000000000004e-07 2024-04-26T03:38:54.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.415 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:54.415 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.601863+0000", 2024-04-26T03:38:54.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00060349500000000001 2024-04-26T03:38:54.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.415 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:54.415 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.601879+0000", 2024-04-26T03:38:54.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5471000000000001e-05 2024-04-26T03:38:54.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.416 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:54.416 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.601888+0000", 2024-04-26T03:38:54.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8540000000000003e-06 2024-04-26T03:38:54.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.416 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:54.416 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.601961+0000", 2024-04-26T03:38:54.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3249000000000003e-05 2024-04-26T03:38:54.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.416 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:54.416 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.601971+0000", 2024-04-26T03:38:54.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0197000000000001e-05 2024-04-26T03:38:54.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.417 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:54.417 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.606694+0000", 2024-04-26T03:38:54.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0047231010000000004 2024-04-26T03:38:54.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.417 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:54.417 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.606727+0000", 2024-04-26T03:38:54.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2206999999999998e-05 2024-04-26T03:38:54.417 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.417 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:54.417 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.418 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.fs5 35/32 ECSubWrite(tid=15, reqid=client.4556.0:136, at_version=35'5, trim_to=0'0, roll_forward_to=35'4))", 2024-04-26T03:38:54.418 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.616287+0000", 2024-04-26T03:38:54.418 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 2.689792486, 2024-04-26T03:38:54.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0056710830000000004, 2024-04-26T03:38:54.418 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:54.418 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:54.418 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:54.418 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:54.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:54.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.616287+0000", 2024-04-26T03:38:54.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.419 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:54.419 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.616287+0000", 2024-04-26T03:38:54.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.419 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:54.419 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.616288+0000", 2024-04-26T03:38:54.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3600000000000002e-07 2024-04-26T03:38:54.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.419 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:54.419 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.616564+0000", 2024-04-26T03:38:54.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00027615700000000002 2024-04-26T03:38:54.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.420 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:54.420 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.616578+0000", 2024-04-26T03:38:54.420 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3895e-05 2024-04-26T03:38:54.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.420 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:54.420 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.616585+0000", 2024-04-26T03:38:54.420 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8960000000000003e-06 2024-04-26T03:38:54.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.420 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:54.420 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.616649+0000", 2024-04-26T03:38:54.420 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3744000000000005e-05 2024-04-26T03:38:54.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.421 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:54.421 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.616658+0000", 2024-04-26T03:38:54.421 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9570000000000008e-06 2024-04-26T03:38:54.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.421 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:54.421 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.621924+0000", 2024-04-26T03:38:54.421 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0052660459999999999 2024-04-26T03:38:54.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.421 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:54.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.621958+0000", 2024-04-26T03:38:54.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3552e-05 2024-04-26T03:38:54.422 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.422 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:54.422 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.422 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.422 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:54.422 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-26T03:38:54.424 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-26T03:38:54.425 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-04-26T03:38:54.422+0000 7fe9223c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31954) UID: 0 2024-04-26T03:38:54.500 INFO:tasks.rados.rados.0.smithi092.stdout:86: done (7 left) 2024-04-26T03:38:54.500 INFO:tasks.rados.rados.0.smithi092.stdout:88: done (6 left) 2024-04-26T03:38:54.500 INFO:tasks.rados.rados.0.smithi092.stdout:89: done (5 left) 2024-04-26T03:38:54.500 INFO:tasks.rados.rados.0.smithi092.stdout:90: done (4 left) 2024-04-26T03:38:54.500 INFO:tasks.rados.rados.0.smithi092.stdout:91: done (3 left) 2024-04-26T03:38:54.500 INFO:tasks.rados.rados.0.smithi092.stdout:92: done (2 left) 2024-04-26T03:38:54.500 INFO:tasks.rados.rados.0.smithi092.stdout:93: done (1 left) 2024-04-26T03:38:54.501 INFO:tasks.rados.rados.0.smithi092.stdout:94: done (0 left) 2024-04-26T03:38:54.501 INFO:tasks.rados.rados.0.smithi092.stdout:95: read oid 38 snap -1 2024-04-26T03:38:54.501 INFO:tasks.rados.rados.0.smithi092.stdout:95: expect (ObjNum 37 snap 0 seq_num 37) 2024-04-26T03:38:54.501 INFO:tasks.rados.rados.0.smithi092.stdout:96: snap_remove snap 2 2024-04-26T03:38:54.524 INFO:tasks.ceph.osd.12.smithi028.stderr:2024-04-26T03:38:54.523+0000 7f114fe14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 31967) UID: 0 2024-04-26T03:38:54.575 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-26T03:38:54.575 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-04-26T03:38:54.576 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-04-26T03:38:54.576 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-26T03:38:54.584 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-26T03:38:54.626 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-04-26T03:38:54.624+0000 7f745118d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 31821) UID: 0 2024-04-26T03:38:54.691 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-26T03:38:54.692 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-04-26T03:38:54.692 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-04-26T03:38:54.692 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-04-26T03:38:54.692 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-26T03:38:54.701 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-26T03:38:54.727 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-04-26T03:38:54.726+0000 7fdc2d21c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31819) UID: 0 2024-04-26T03:38:54.811 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-26T03:38:54.811 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-04-26T03:38:54.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-04-26T03:38:54.812 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-04-26T03:38:54.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.812 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.2s1 34/32 ECSubWrite(tid=0, reqid=client.4556.0:1, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:38:54.812 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.481623+0000", 2024-04-26T03:38:54.812 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.325809789, 2024-04-26T03:38:54.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0026324389999999999, 2024-04-26T03:38:54.812 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:54.812 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:54.812 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:54.812 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:54.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:54.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.481623+0000", 2024-04-26T03:38:54.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:54.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.481623+0000", 2024-04-26T03:38:54.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:54.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.481625+0000", 2024-04-26T03:38:54.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1170000000000001e-06 2024-04-26T03:38:54.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:54.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.481661+0000", 2024-04-26T03:38:54.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6359999999999997e-05 2024-04-26T03:38:54.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:54.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.481714+0000", 2024-04-26T03:38:54.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3329999999999999e-05 2024-04-26T03:38:54.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:54.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.481729+0000", 2024-04-26T03:38:54.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5166e-05 2024-04-26T03:38:54.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:54.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.481906+0000", 2024-04-26T03:38:54.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017697500000000001 2024-04-26T03:38:54.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:54.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.481940+0000", 2024-04-26T03:38:54.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3769000000000001e-05 2024-04-26T03:38:54.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:54.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484207+0000", 2024-04-26T03:38:54.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022672450000000002 2024-04-26T03:38:54.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:54.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484256+0000", 2024-04-26T03:38:54.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8476999999999997e-05 2024-04-26T03:38:54.816 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.816 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:54.816 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.816 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.8s6 34/32 ECSubWrite(tid=157, reqid=client.4556.0:4, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:38:54.816 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.481740+0000", 2024-04-26T03:38:54.816 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.3256927649999999, 2024-04-26T03:38:54.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0021668569999999999, 2024-04-26T03:38:54.816 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:54.816 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:54.816 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:54.817 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:54.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:54.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.481740+0000", 2024-04-26T03:38:54.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:54.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.481740+0000", 2024-04-26T03:38:54.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:54.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.481741+0000", 2024-04-26T03:38:54.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6e-07 2024-04-26T03:38:54.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:54.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.481794+0000", 2024-04-26T03:38:54.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3171999999999998e-05 2024-04-26T03:38:54.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:54.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.481808+0000", 2024-04-26T03:38:54.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4551e-05 2024-04-26T03:38:54.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:54.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.481818+0000", 2024-04-26T03:38:54.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2979999999999997e-06 2024-04-26T03:38:54.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:54.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.481946+0000", 2024-04-26T03:38:54.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000128059 2024-04-26T03:38:54.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:54.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.481964+0000", 2024-04-26T03:38:54.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8012e-05 2024-04-26T03:38:54.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:54.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.483864+0000", 2024-04-26T03:38:54.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00190043 2024-04-26T03:38:54.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:54.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.483907+0000", 2024-04-26T03:38:54.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3074999999999997e-05 2024-04-26T03:38:54.820 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.820 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:54.820 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.821 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.fs2 34/32 ECSubWrite(tid=2, reqid=client.4556.0:25, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:38:54.821 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.489563+0000", 2024-04-26T03:38:54.821 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.3178706529999999, 2024-04-26T03:38:54.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0028516330000000001, 2024-04-26T03:38:54.821 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:54.821 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:54.821 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:54.821 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:54.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:54.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.489563+0000", 2024-04-26T03:38:54.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:54.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.489563+0000", 2024-04-26T03:38:54.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:54.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.489563+0000", 2024-04-26T03:38:54.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6899999999999998e-07 2024-04-26T03:38:54.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:54.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.489605+0000", 2024-04-26T03:38:54.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2030000000000002e-05 2024-04-26T03:38:54.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:54.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.489620+0000", 2024-04-26T03:38:54.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4732000000000001e-05 2024-04-26T03:38:54.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:54.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.489631+0000", 2024-04-26T03:38:54.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1418e-05 2024-04-26T03:38:54.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:54.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.489790+0000", 2024-04-26T03:38:54.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000159121 2024-04-26T03:38:54.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:54.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.489808+0000", 2024-04-26T03:38:54.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7533e-05 2024-04-26T03:38:54.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:54.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.492376+0000", 2024-04-26T03:38:54.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025682579999999999 2024-04-26T03:38:54.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:54.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.492414+0000", 2024-04-26T03:38:54.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8172e-05 2024-04-26T03:38:54.825 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.825 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:54.825 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.825 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.5s6 34/32 ECSubWrite(tid=0, reqid=client.4556.0:28, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:38:54.825 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.492527+0000", 2024-04-26T03:38:54.825 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.3149059269999999, 2024-04-26T03:38:54.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025979789999999998, 2024-04-26T03:38:54.826 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:54.826 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:54.826 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:54.826 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:54.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:54.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.492527+0000", 2024-04-26T03:38:54.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:54.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.492527+0000", 2024-04-26T03:38:54.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:54.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.492528+0000", 2024-04-26T03:38:54.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2e-07 2024-04-26T03:38:54.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:54.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.492573+0000", 2024-04-26T03:38:54.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4972000000000002e-05 2024-04-26T03:38:54.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:54.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.492584+0000", 2024-04-26T03:38:54.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1358999999999999e-05 2024-04-26T03:38:54.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:54.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.492594+0000", 2024-04-26T03:38:54.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9590000000000007e-06 2024-04-26T03:38:54.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:54.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.492727+0000", 2024-04-26T03:38:54.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013292499999999999 2024-04-26T03:38:54.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:54.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.492738+0000", 2024-04-26T03:38:54.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0997e-05 2024-04-26T03:38:54.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:54.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.495087+0000", 2024-04-26T03:38:54.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023485820000000001 2024-04-26T03:38:54.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:54.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.495125+0000", 2024-04-26T03:38:54.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8665000000000003e-05 2024-04-26T03:38:54.829 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.829 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:54.829 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.830 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.4s6 35/32 ECSubWrite(tid=0, reqid=client.4556.0:34, at_version=35'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:38:54.830 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.496017+0000", 2024-04-26T03:38:54.830 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.3114163850000002, 2024-04-26T03:38:54.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037380109999999999, 2024-04-26T03:38:54.830 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:54.830 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:54.830 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:54.830 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:54.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:54.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496017+0000", 2024-04-26T03:38:54.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:54.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496017+0000", 2024-04-26T03:38:54.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:54.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496035+0000", 2024-04-26T03:38:54.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8490000000000001e-05 2024-04-26T03:38:54.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:54.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496177+0000", 2024-04-26T03:38:54.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000142137 2024-04-26T03:38:54.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:54.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496199+0000", 2024-04-26T03:38:54.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1531000000000001e-05 2024-04-26T03:38:54.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:54.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496209+0000", 2024-04-26T03:38:54.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0231000000000001e-05 2024-04-26T03:38:54.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:54.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496276+0000", 2024-04-26T03:38:54.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7089999999999996e-05 2024-04-26T03:38:54.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:54.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496286+0000", 2024-04-26T03:38:54.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.961e-06 2024-04-26T03:38:54.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:54.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.499720+0000", 2024-04-26T03:38:54.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0034336800000000002 2024-04-26T03:38:54.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:54.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.499755+0000", 2024-04-26T03:38:54.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4891000000000002e-05 2024-04-26T03:38:54.834 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.834 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:54.834 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.834 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds4 35/32 ECSubWrite(tid=0, reqid=client.4556.0:43, at_version=35'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:38:54.834 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.500392+0000", 2024-04-26T03:38:54.834 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.3070415359999998, 2024-04-26T03:38:54.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0036998230000000001, 2024-04-26T03:38:54.835 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:54.835 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:54.835 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:54.835 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:54.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:54.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.500392+0000", 2024-04-26T03:38:54.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:54.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.500392+0000", 2024-04-26T03:38:54.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:54.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.500392+0000", 2024-04-26T03:38:54.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9200000000000002e-07 2024-04-26T03:38:54.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:54.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.500442+0000", 2024-04-26T03:38:54.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0031000000000003e-05 2024-04-26T03:38:54.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:54.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.500456+0000", 2024-04-26T03:38:54.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3767e-05 2024-04-26T03:38:54.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:54.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.500465+0000", 2024-04-26T03:38:54.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5000000000000005e-06 2024-04-26T03:38:54.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:54.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.500521+0000", 2024-04-26T03:38:54.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5551999999999999e-05 2024-04-26T03:38:54.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:54.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.500529+0000", 2024-04-26T03:38:54.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2139999999999996e-06 2024-04-26T03:38:54.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:54.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504052+0000", 2024-04-26T03:38:54.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035233460000000001 2024-04-26T03:38:54.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:54.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504091+0000", 2024-04-26T03:38:54.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9020999999999998e-05 2024-04-26T03:38:54.839 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.840 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:54.840 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.840 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.4s6 35/32 ECSubWrite(tid=2, reqid=client.4556.0:49, at_version=35'3, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:38:54.840 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.505687+0000", 2024-04-26T03:38:54.840 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.3017462279999998, 2024-04-26T03:38:54.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0039336620000000001, 2024-04-26T03:38:54.840 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:54.840 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:54.840 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:54.840 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:54.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:54.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.505687+0000", 2024-04-26T03:38:54.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:54.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.505687+0000", 2024-04-26T03:38:54.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:54.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.505688+0000", 2024-04-26T03:38:54.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0449999999999999e-06 2024-04-26T03:38:54.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:54.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.505851+0000", 2024-04-26T03:38:54.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016254799999999999 2024-04-26T03:38:54.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:54.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.505869+0000", 2024-04-26T03:38:54.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8771999999999999e-05 2024-04-26T03:38:54.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:54.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.505879+0000", 2024-04-26T03:38:54.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3619999999999995e-06 2024-04-26T03:38:54.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:54.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.505960+0000", 2024-04-26T03:38:54.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1354999999999996e-05 2024-04-26T03:38:54.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:54.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.505969+0000", 2024-04-26T03:38:54.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3170000000000008e-06 2024-04-26T03:38:54.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:54.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.509544+0000", 2024-04-26T03:38:54.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035749250000000001 2024-04-26T03:38:54.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:54.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.509621+0000", 2024-04-26T03:38:54.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6338e-05 2024-04-26T03:38:54.844 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.844 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:54.844 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.844 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds4 35/32 ECSubWrite(tid=2, reqid=client.4556.0:52, at_version=35'3, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:38:54.844 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.506924+0000", 2024-04-26T03:38:54.844 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.3005089019999998, 2024-04-26T03:38:54.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0043484229999999997, 2024-04-26T03:38:54.845 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:54.845 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:54.845 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:54.845 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:54.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:54.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.506924+0000", 2024-04-26T03:38:54.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:54.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.506924+0000", 2024-04-26T03:38:54.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:54.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.506925+0000", 2024-04-26T03:38:54.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1310000000000001e-06 2024-04-26T03:38:54.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:54.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.507016+0000", 2024-04-26T03:38:54.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0923999999999999e-05 2024-04-26T03:38:54.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:54.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.507045+0000", 2024-04-26T03:38:54.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8336000000000001e-05 2024-04-26T03:38:54.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:54.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.507056+0000", 2024-04-26T03:38:54.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1141999999999999e-05 2024-04-26T03:38:54.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:54.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.507119+0000", 2024-04-26T03:38:54.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3008000000000004e-05 2024-04-26T03:38:54.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:54.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.507128+0000", 2024-04-26T03:38:54.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1339999999999993e-06 2024-04-26T03:38:54.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:54.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.511221+0000", 2024-04-26T03:38:54.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0040928600000000002 2024-04-26T03:38:54.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:54.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.511273+0000", 2024-04-26T03:38:54.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1888000000000002e-05 2024-04-26T03:38:54.848 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.848 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:54.848 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.849 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.8s6 35/32 ECSubWrite(tid=161, reqid=client.4556.0:55, at_version=35'3, trim_to=0'0, roll_forward_to=34'2))", 2024-04-26T03:38:54.849 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.509662+0000", 2024-04-26T03:38:54.849 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.2977713940000002, 2024-04-26T03:38:54.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0047776379999999999, 2024-04-26T03:38:54.849 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:54.849 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:54.849 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:54.849 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:54.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:54.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.509662+0000", 2024-04-26T03:38:54.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:54.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.509662+0000", 2024-04-26T03:38:54.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:54.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.509663+0000", 2024-04-26T03:38:54.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0079999999999999e-06 2024-04-26T03:38:54.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:54.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.509849+0000", 2024-04-26T03:38:54.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018641099999999999 2024-04-26T03:38:54.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:54.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.509867+0000", 2024-04-26T03:38:54.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7465999999999998e-05 2024-04-26T03:38:54.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:54.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.509876+0000", 2024-04-26T03:38:54.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7790000000000007e-06 2024-04-26T03:38:54.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:54.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.509968+0000", 2024-04-26T03:38:54.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1699000000000004e-05 2024-04-26T03:38:54.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:54.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.509978+0000", 2024-04-26T03:38:54.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.007e-05 2024-04-26T03:38:54.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:54.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.514407+0000", 2024-04-26T03:38:54.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0044286059999999999 2024-04-26T03:38:54.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:54.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.514439+0000", 2024-04-26T03:38:54.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2598999999999997e-05 2024-04-26T03:38:54.853 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.853 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:54.853 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.853 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.8s6 35/32 ECSubWrite(tid=163, reqid=client.4556.0:56, at_version=0'0, trim_to=0'0, roll_forward_to=35'4))", 2024-04-26T03:38:54.853 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.516084+0000", 2024-04-26T03:38:54.853 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.29134948, 2024-04-26T03:38:54.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002118726, 2024-04-26T03:38:54.854 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:54.854 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:54.854 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:54.854 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:54.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:54.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.516084+0000", 2024-04-26T03:38:54.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:54.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.516084+0000", 2024-04-26T03:38:54.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:54.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.516085+0000", 2024-04-26T03:38:54.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0589999999999999e-06 2024-04-26T03:38:54.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:54.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.516100+0000", 2024-04-26T03:38:54.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5634e-05 2024-04-26T03:38:54.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:54.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.516120+0000", 2024-04-26T03:38:54.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9194999999999998e-05 2024-04-26T03:38:54.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:54.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.516129+0000", 2024-04-26T03:38:54.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9809999999999992e-06 2024-04-26T03:38:54.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:54.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.516556+0000", 2024-04-26T03:38:54.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000427611 2024-04-26T03:38:54.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:54.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.516565+0000", 2024-04-26T03:38:54.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9239999999999996e-06 2024-04-26T03:38:54.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:54.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.518162+0000", 2024-04-26T03:38:54.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015966839999999999 2024-04-26T03:38:54.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:54.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.518202+0000", 2024-04-26T03:38:54.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0638e-05 2024-04-26T03:38:54.858 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.858 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:54.858 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.858 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.5s6 35/32 ECSubWrite(tid=4, reqid=client.4556.0:67, at_version=35'3, trim_to=0'0, roll_forward_to=34'2))", 2024-04-26T03:38:54.858 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.519658+0000", 2024-04-26T03:38:54.859 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.2877755510000002, 2024-04-26T03:38:54.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0039326609999999996, 2024-04-26T03:38:54.859 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:54.859 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:54.859 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:54.859 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:54.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:54.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.519658+0000", 2024-04-26T03:38:54.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:54.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.519658+0000", 2024-04-26T03:38:54.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:54.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.519659+0000", 2024-04-26T03:38:54.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.034e-06 2024-04-26T03:38:54.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:54.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.519918+0000", 2024-04-26T03:38:54.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025939799999999999 2024-04-26T03:38:54.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:54.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.519931+0000", 2024-04-26T03:38:54.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3281e-05 2024-04-26T03:38:54.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:54.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.519942+0000", 2024-04-26T03:38:54.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0149999999999999e-05 2024-04-26T03:38:54.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:54.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.520012+0000", 2024-04-26T03:38:54.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0099999999999996e-05 2024-04-26T03:38:54.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:54.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.520043+0000", 2024-04-26T03:38:54.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1559000000000001e-05 2024-04-26T03:38:54.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:54.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.523550+0000", 2024-04-26T03:38:54.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035071600000000001 2024-04-26T03:38:54.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:54.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.523590+0000", 2024-04-26T03:38:54.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9978999999999998e-05 2024-04-26T03:38:54.862 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.863 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:54.863 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.863 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds4 35/32 ECSubWrite(tid=7, reqid=client.4556.0:64, at_version=35'5, trim_to=0'0, roll_forward_to=35'4))", 2024-04-26T03:38:54.863 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.528665+0000", 2024-04-26T03:38:54.863 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.27876805, 2024-04-26T03:38:54.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0061053569999999996, 2024-04-26T03:38:54.863 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:54.863 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:54.863 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:54.863 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:54.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:54.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528665+0000", 2024-04-26T03:38:54.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:54.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528665+0000", 2024-04-26T03:38:54.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:54.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528666+0000", 2024-04-26T03:38:54.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.071e-06 2024-04-26T03:38:54.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:54.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528902+0000", 2024-04-26T03:38:54.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00023567299999999999 2024-04-26T03:38:54.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:54.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528920+0000", 2024-04-26T03:38:54.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8252000000000001e-05 2024-04-26T03:38:54.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:54.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528930+0000", 2024-04-26T03:38:54.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7329999999999998e-06 2024-04-26T03:38:54.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:54.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.529014+0000", 2024-04-26T03:38:54.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3945999999999999e-05 2024-04-26T03:38:54.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:54.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.529031+0000", 2024-04-26T03:38:54.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6960999999999999e-05 2024-04-26T03:38:54.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:54.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.534732+0000", 2024-04-26T03:38:54.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0057011630000000004 2024-04-26T03:38:54.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:54.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.534771+0000", 2024-04-26T03:38:54.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8557999999999999e-05 2024-04-26T03:38:54.867 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.867 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:54.867 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.867 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.2s1 35/32 ECSubWrite(tid=4, reqid=client.4556.0:79, at_version=35'3, trim_to=0'0, roll_forward_to=34'2))", 2024-04-26T03:38:54.867 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.533126+0000", 2024-04-26T03:38:54.867 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.2743068019999999, 2024-04-26T03:38:54.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0049197210000000002, 2024-04-26T03:38:54.867 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:54.867 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:54.868 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:54.868 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:54.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:54.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.533126+0000", 2024-04-26T03:38:54.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:54.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.533126+0000", 2024-04-26T03:38:54.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:54.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.533128+0000", 2024-04-26T03:38:54.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3629999999999999e-06 2024-04-26T03:38:54.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:54.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.533343+0000", 2024-04-26T03:38:54.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021517899999999999 2024-04-26T03:38:54.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:54.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.533362+0000", 2024-04-26T03:38:54.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9483999999999998e-05 2024-04-26T03:38:54.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:54.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.533374+0000", 2024-04-26T03:38:54.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1762e-05 2024-04-26T03:38:54.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:54.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.533463+0000", 2024-04-26T03:38:54.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9028000000000002e-05 2024-04-26T03:38:54.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:54.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.533476+0000", 2024-04-26T03:38:54.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2975e-05 2024-04-26T03:38:54.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:54.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.537997+0000", 2024-04-26T03:38:54.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.004520713 2024-04-26T03:38:54.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:54.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.538046+0000", 2024-04-26T03:38:54.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9217e-05 2024-04-26T03:38:54.871 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.871 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:54.871 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.872 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.4s6 35/32 ECSubWrite(tid=7, reqid=client.4556.0:82, at_version=35'5, trim_to=0'0, roll_forward_to=35'4))", 2024-04-26T03:38:54.872 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.539116+0000", 2024-04-26T03:38:54.872 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.2683172740000002, 2024-04-26T03:38:54.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0044880090000000003, 2024-04-26T03:38:54.872 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:54.872 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:54.872 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:54.872 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:54.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:54.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.539116+0000", 2024-04-26T03:38:54.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:54.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.539116+0000", 2024-04-26T03:38:54.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:54.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.539117+0000", 2024-04-26T03:38:54.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2500000000000004e-07 2024-04-26T03:38:54.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:54.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.539482+0000", 2024-04-26T03:38:54.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036553600000000002 2024-04-26T03:38:54.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:54.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.539545+0000", 2024-04-26T03:38:54.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2570999999999994e-05 2024-04-26T03:38:54.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:54.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.539555+0000", 2024-04-26T03:38:54.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6290000000000005e-06 2024-04-26T03:38:54.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:54.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.539641+0000", 2024-04-26T03:38:54.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5921000000000006e-05 2024-04-26T03:38:54.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:54.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.539650+0000", 2024-04-26T03:38:54.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3570000000000004e-06 2024-04-26T03:38:54.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:54.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.543573+0000", 2024-04-26T03:38:54.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0039227209999999997 2024-04-26T03:38:54.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:54.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.543604+0000", 2024-04-26T03:38:54.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1349000000000001e-05 2024-04-26T03:38:54.876 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.876 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:54.876 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.876 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.8s6 35/32 ECSubWrite(tid=165, reqid=client.4556.0:91, at_version=35'5, trim_to=0'0, roll_forward_to=35'4))", 2024-04-26T03:38:54.876 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.549094+0000", 2024-04-26T03:38:54.876 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.258338841, 2024-04-26T03:38:54.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0044747659999999998, 2024-04-26T03:38:54.877 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:54.877 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:54.877 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:54.877 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:54.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:54.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.549094+0000", 2024-04-26T03:38:54.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:54.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.549094+0000", 2024-04-26T03:38:54.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:54.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.549095+0000", 2024-04-26T03:38:54.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4600000000000003e-07 2024-04-26T03:38:54.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:54.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.549356+0000", 2024-04-26T03:38:54.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026122899999999998 2024-04-26T03:38:54.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:54.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.549369+0000", 2024-04-26T03:38:54.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2624e-05 2024-04-26T03:38:54.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:54.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.549376+0000", 2024-04-26T03:38:54.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1409999999999999e-06 2024-04-26T03:38:54.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:54.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.549434+0000", 2024-04-26T03:38:54.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8034e-05 2024-04-26T03:38:54.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:54.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.549441+0000", 2024-04-26T03:38:54.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4060000000000002e-06 2024-04-26T03:38:54.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:54.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.553538+0000", 2024-04-26T03:38:54.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0040975400000000002 2024-04-26T03:38:54.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:54.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.553569+0000", 2024-04-26T03:38:54.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0945999999999998e-05 2024-04-26T03:38:54.880 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.880 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:54.880 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.881 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.4s6 35/32 ECSubWrite(tid=11, reqid=client.4556.0:100, at_version=35'7, trim_to=0'0, roll_forward_to=35'6))", 2024-04-26T03:38:54.881 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.560536+0000", 2024-04-26T03:38:54.881 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.2468975339999999, 2024-04-26T03:38:54.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0052869859999999996, 2024-04-26T03:38:54.881 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:54.881 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:54.881 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:54.881 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:54.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:54.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.560536+0000", 2024-04-26T03:38:54.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:54.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.560536+0000", 2024-04-26T03:38:54.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:54.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.560536+0000", 2024-04-26T03:38:54.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.23e-07 2024-04-26T03:38:54.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:54.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.560987+0000", 2024-04-26T03:38:54.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00045054099999999999 2024-04-26T03:38:54.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:54.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.561004+0000", 2024-04-26T03:38:54.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6691999999999999e-05 2024-04-26T03:38:54.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:54.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.561012+0000", 2024-04-26T03:38:54.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.225e-06 2024-04-26T03:38:54.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:54.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.561073+0000", 2024-04-26T03:38:54.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1017000000000002e-05 2024-04-26T03:38:54.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:54.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.561082+0000", 2024-04-26T03:38:54.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2830000000000007e-06 2024-04-26T03:38:54.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:54.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.565784+0000", 2024-04-26T03:38:54.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0047014530000000004 2024-04-26T03:38:54.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:54.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.565823+0000", 2024-04-26T03:38:54.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8952000000000002e-05 2024-04-26T03:38:54.885 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.885 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:54.885 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.885 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.4s6 35/32 ECSubWrite(tid=13, reqid=client.4556.0:106, at_version=35'9, trim_to=0'0, roll_forward_to=35'6))", 2024-04-26T03:38:54.885 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.567140+0000", 2024-04-26T03:38:54.885 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.240292991, 2024-04-26T03:38:54.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0045232120000000004, 2024-04-26T03:38:54.886 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:54.886 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:54.886 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:54.886 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:54.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:54.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.567140+0000", 2024-04-26T03:38:54.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:54.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.567140+0000", 2024-04-26T03:38:54.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:54.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.567141+0000", 2024-04-26T03:38:54.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6400000000000001e-07 2024-04-26T03:38:54.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:54.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.567610+0000", 2024-04-26T03:38:54.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00046927699999999999 2024-04-26T03:38:54.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:54.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.567629+0000", 2024-04-26T03:38:54.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8822999999999999e-05 2024-04-26T03:38:54.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:54.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.567644+0000", 2024-04-26T03:38:54.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4507e-05 2024-04-26T03:38:54.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:54.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.567696+0000", 2024-04-26T03:38:54.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2811e-05 2024-04-26T03:38:54.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:54.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.567702+0000", 2024-04-26T03:38:54.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5759999999999996e-06 2024-04-26T03:38:54.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:54.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.571631+0000", 2024-04-26T03:38:54.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0039289659999999999 2024-04-26T03:38:54.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:54.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.571663+0000", 2024-04-26T03:38:54.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2487999999999998e-05 2024-04-26T03:38:54.889 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.889 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:54.889 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.890 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.fs2 35/32 ECSubWrite(tid=11, reqid=client.4556.0:112, at_version=35'3, trim_to=0'0, roll_forward_to=34'2))", 2024-04-26T03:38:54.890 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.576703+0000", 2024-04-26T03:38:54.890 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.230729851, 2024-04-26T03:38:54.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0050673259999999996, 2024-04-26T03:38:54.890 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:54.890 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:54.890 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:54.890 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:54.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:54.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.576703+0000", 2024-04-26T03:38:54.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:54.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.576703+0000", 2024-04-26T03:38:54.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:54.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.576704+0000", 2024-04-26T03:38:54.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6700000000000002e-07 2024-04-26T03:38:54.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:54.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.577225+0000", 2024-04-26T03:38:54.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00052048100000000005 2024-04-26T03:38:54.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:54.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.577240+0000", 2024-04-26T03:38:54.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4883e-05 2024-04-26T03:38:54.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:54.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.577248+0000", 2024-04-26T03:38:54.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8629999999999997e-06 2024-04-26T03:38:54.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:54.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.577324+0000", 2024-04-26T03:38:54.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5649999999999996e-05 2024-04-26T03:38:54.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:54.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.577334+0000", 2024-04-26T03:38:54.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5759999999999995e-06 2024-04-26T03:38:54.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:54.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.581742+0000", 2024-04-26T03:38:54.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0044087459999999998 2024-04-26T03:38:54.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:54.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.581771+0000", 2024-04-26T03:38:54.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.826e-05 2024-04-26T03:38:54.894 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.894 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:54.894 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.894 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.fs2 35/32 ECSubWrite(tid=15, reqid=client.4556.0:136, at_version=35'5, trim_to=0'0, roll_forward_to=35'4))", 2024-04-26T03:38:54.894 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.615680+0000", 2024-04-26T03:38:54.894 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.1917528709999998, 2024-04-26T03:38:54.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0055997080000000001, 2024-04-26T03:38:54.894 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:54.894 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:54.894 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:54.895 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:54.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:54.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.615680+0000", 2024-04-26T03:38:54.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:54.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.615680+0000", 2024-04-26T03:38:54.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:54.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.615682+0000", 2024-04-26T03:38:54.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.221e-06 2024-04-26T03:38:54.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:54.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.616158+0000", 2024-04-26T03:38:54.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00047638800000000001 2024-04-26T03:38:54.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:54.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.616172+0000", 2024-04-26T03:38:54.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3984e-05 2024-04-26T03:38:54.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:54.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.616179+0000", 2024-04-26T03:38:54.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5490000000000001e-06 2024-04-26T03:38:54.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:54.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.616232+0000", 2024-04-26T03:38:54.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2528000000000003e-05 2024-04-26T03:38:54.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:54.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.616240+0000", 2024-04-26T03:38:54.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7889999999999995e-06 2024-04-26T03:38:54.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:54.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.621245+0000", 2024-04-26T03:38:54.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0050054909999999999 2024-04-26T03:38:54.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:54.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.621280+0000", 2024-04-26T03:38:54.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4758000000000002e-05 2024-04-26T03:38:54.898 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.898 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:54.898 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.898 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.2s1 35/32 ECSubWrite(tid=8, reqid=client.4556.0:139, at_version=35'5, trim_to=0'0, roll_forward_to=35'4))", 2024-04-26T03:38:54.898 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.620822+0000", 2024-04-26T03:38:54.898 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.1866110829999998, 2024-04-26T03:38:54.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.005803173, 2024-04-26T03:38:54.899 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:54.899 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:54.899 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:54.899 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:54.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:54.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.620822+0000", 2024-04-26T03:38:54.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:54.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.620822+0000", 2024-04-26T03:38:54.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:54.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:54.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.620823+0000", 2024-04-26T03:38:54.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1399999999999996e-07 2024-04-26T03:38:54.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:54.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.621346+0000", 2024-04-26T03:38:54.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00052321400000000004 2024-04-26T03:38:54.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:54.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.621364+0000", 2024-04-26T03:38:54.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8360999999999999e-05 2024-04-26T03:38:54.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:54.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.621373+0000", 2024-04-26T03:38:54.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4649999999999994e-06 2024-04-26T03:38:54.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:54.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.621428+0000", 2024-04-26T03:38:54.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5062000000000002e-05 2024-04-26T03:38:54.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:54.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.621437+0000", 2024-04-26T03:38:54.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3600000000000002e-06 2024-04-26T03:38:54.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:54.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.626599+0000", 2024-04-26T03:38:54.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.005162191 2024-04-26T03:38:54.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:54.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:54.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:54.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.626625+0000", 2024-04-26T03:38:54.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5806e-05 2024-04-26T03:38:54.902 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.902 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:54.902 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.902 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:54.903 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:54.903 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-26T03:38:54.904 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-04-26T03:38:54.905 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-04-26T03:38:54.902+0000 7f745118d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 31821) UID: 0 2024-04-26T03:38:55.005 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:38:55.003+0000 7fba0d2ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31961) UID: 0 2024-04-26T03:38:55.011 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-26T03:38:55.012 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-04-26T03:38:55.012 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-04-26T03:38:55.012 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-26T03:38:55.020 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-04-26T03:38:55.106 INFO:tasks.ceph.osd.15.smithi092.stderr:2024-04-26T03:38:55.104+0000 7f6d12dec640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 31824) UID: 0 2024-04-26T03:38:55.127 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-26T03:38:55.127 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-04-26T03:38:55.127 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-04-26T03:38:55.127 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-04-26T03:38:55.127 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-26T03:38:55.136 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-04-26T03:38:55.206 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-04-26T03:38:55.205+0000 7fe9223c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31954) UID: 0 2024-04-26T03:38:55.244 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-26T03:38:55.244 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-04-26T03:38:55.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-04-26T03:38:55.244 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-04-26T03:38:55.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.244 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs4 35/32 ECSubWrite(tid=4, reqid=client.4556.0:31, at_version=35'3, trim_to=0'0, roll_forward_to=34'1))", 2024-04-26T03:38:55.244 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.496058+0000", 2024-04-26T03:38:55.244 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.7456325110000002, 2024-04-26T03:38:55.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038517849999999999, 2024-04-26T03:38:55.245 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:55.245 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:55.245 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:55.245 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:55.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.245 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:55.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496058+0000", 2024-04-26T03:38:55.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.245 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:55.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496058+0000", 2024-04-26T03:38:55.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:55.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496059+0000", 2024-04-26T03:38:55.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3859999999999999e-06 2024-04-26T03:38:55.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:55.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496266+0000", 2024-04-26T03:38:55.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020700499999999999 2024-04-26T03:38:55.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:55.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496301+0000", 2024-04-26T03:38:55.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5063e-05 2024-04-26T03:38:55.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:55.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496315+0000", 2024-04-26T03:38:55.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3545e-05 2024-04-26T03:38:55.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496397+0000", 2024-04-26T03:38:55.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2531e-05 2024-04-26T03:38:55.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:55.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496408+0000", 2024-04-26T03:38:55.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0959e-05 2024-04-26T03:38:55.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.248 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:55.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.499863+0000", 2024-04-26T03:38:55.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003454496 2024-04-26T03:38:55.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.248 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:55.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.499910+0000", 2024-04-26T03:38:55.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6799999999999999e-05 2024-04-26T03:38:55.248 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.248 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:55.248 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.249 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.0s1 35/32 ECSubWrite(tid=2, reqid=client.4556.0:40, at_version=35'3, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:38:55.249 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.497691+0000", 2024-04-26T03:38:55.249 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.7439992220000002, 2024-04-26T03:38:55.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0039494580000000003, 2024-04-26T03:38:55.249 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:55.249 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:55.249 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:55.249 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:55.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:55.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.497691+0000", 2024-04-26T03:38:55.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:55.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.497691+0000", 2024-04-26T03:38:55.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:55.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.497692+0000", 2024-04-26T03:38:55.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0359999999999999e-06 2024-04-26T03:38:55.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:55.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.497826+0000", 2024-04-26T03:38:55.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013366499999999999 2024-04-26T03:38:55.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:55.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.497850+0000", 2024-04-26T03:38:55.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4453000000000001e-05 2024-04-26T03:38:55.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:55.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.497861+0000", 2024-04-26T03:38:55.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0424e-05 2024-04-26T03:38:55.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.497942+0000", 2024-04-26T03:38:55.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1271000000000002e-05 2024-04-26T03:38:55.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:55.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.497951+0000", 2024-04-26T03:38:55.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2110000000000004e-06 2024-04-26T03:38:55.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:55.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.501595+0000", 2024-04-26T03:38:55.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0036442969999999999 2024-04-26T03:38:55.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.253 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:55.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.501641+0000", 2024-04-26T03:38:55.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5101000000000001e-05 2024-04-26T03:38:55.253 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.253 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:55.253 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.253 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds2 35/32 ECSubWrite(tid=0, reqid=client.4556.0:43, at_version=35'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:38:55.253 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.501816+0000", 2024-04-26T03:38:55.254 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.7398743749999999, 2024-04-26T03:38:55.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0091953699999999996, 2024-04-26T03:38:55.254 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:55.254 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:55.254 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:55.254 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:55.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.254 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:55.254 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.501816+0000", 2024-04-26T03:38:55.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.254 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:55.254 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.501816+0000", 2024-04-26T03:38:55.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:55.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.501816+0000", 2024-04-26T03:38:55.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8099999999999998e-07 2024-04-26T03:38:55.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:55.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.502179+0000", 2024-04-26T03:38:55.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036287299999999999 2024-04-26T03:38:55.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:55.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.502197+0000", 2024-04-26T03:38:55.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7765e-05 2024-04-26T03:38:55.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:55.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.502207+0000", 2024-04-26T03:38:55.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6190000000000006e-06 2024-04-26T03:38:55.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.502306+0000", 2024-04-26T03:38:55.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9655999999999996e-05 2024-04-26T03:38:55.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:55.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.502315+0000", 2024-04-26T03:38:55.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5629999999999992e-06 2024-04-26T03:38:55.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:55.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.510976+0000", 2024-04-26T03:38:55.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0086612379999999999 2024-04-26T03:38:55.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:55.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.511011+0000", 2024-04-26T03:38:55.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5274999999999997e-05 2024-04-26T03:38:55.258 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.258 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:55.258 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.258 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds2 35/32 ECSubWrite(tid=1, reqid=client.4556.0:44, at_version=35'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:38:55.258 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.504658+0000", 2024-04-26T03:38:55.258 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.7370320590000001, 2024-04-26T03:38:55.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0063889319999999999, 2024-04-26T03:38:55.258 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:55.259 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:55.259 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:55.259 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:55.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:55.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504658+0000", 2024-04-26T03:38:55.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:55.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504658+0000", 2024-04-26T03:38:55.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:55.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504659+0000", 2024-04-26T03:38:55.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3399999999999997e-07 2024-04-26T03:38:55.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:55.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504681+0000", 2024-04-26T03:38:55.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1452999999999999e-05 2024-04-26T03:38:55.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:55.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504700+0000", 2024-04-26T03:38:55.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9772999999999999e-05 2024-04-26T03:38:55.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:55.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504710+0000", 2024-04-26T03:38:55.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2660000000000007e-06 2024-04-26T03:38:55.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.505327+0000", 2024-04-26T03:38:55.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00061779000000000001 2024-04-26T03:38:55.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:55.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.505339+0000", 2024-04-26T03:38:55.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.147e-05 2024-04-26T03:38:55.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:55.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.511016+0000", 2024-04-26T03:38:55.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0056775439999999996 2024-04-26T03:38:55.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:55.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.511047+0000", 2024-04-26T03:38:55.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0701999999999998e-05 2024-04-26T03:38:55.263 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.263 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:55.263 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.263 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.9s3 35/32 ECSubWrite(tid=0, reqid=client.4556.0:46, at_version=35'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:38:55.263 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.505322+0000", 2024-04-26T03:38:55.263 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.7363686700000001, 2024-04-26T03:38:55.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038881240000000002, 2024-04-26T03:38:55.263 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:55.263 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:55.263 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:55.263 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:55.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:55.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.505322+0000", 2024-04-26T03:38:55.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:55.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.505322+0000", 2024-04-26T03:38:55.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:55.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.505322+0000", 2024-04-26T03:38:55.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5899999999999996e-07 2024-04-26T03:38:55.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:55.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.505405+0000", 2024-04-26T03:38:55.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3090000000000006e-05 2024-04-26T03:38:55.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:55.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.505419+0000", 2024-04-26T03:38:55.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4192000000000001e-05 2024-04-26T03:38:55.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:55.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.505429+0000", 2024-04-26T03:38:55.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6169999999999996e-06 2024-04-26T03:38:55.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.505511+0000", 2024-04-26T03:38:55.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2281999999999996e-05 2024-04-26T03:38:55.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:55.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.505522+0000", 2024-04-26T03:38:55.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0488e-05 2024-04-26T03:38:55.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:55.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.509169+0000", 2024-04-26T03:38:55.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0036469800000000002 2024-04-26T03:38:55.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:55.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.509210+0000", 2024-04-26T03:38:55.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0915999999999999e-05 2024-04-26T03:38:55.267 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.267 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:55.267 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.267 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds2 35/32 ECSubWrite(tid=2, reqid=client.4556.0:52, at_version=35'3, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:38:55.267 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.507269+0000", 2024-04-26T03:38:55.268 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.7344216239999999, 2024-04-26T03:38:55.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0045244609999999996, 2024-04-26T03:38:55.268 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:55.268 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:55.268 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:55.268 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:55.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:55.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.507269+0000", 2024-04-26T03:38:55.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:55.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.507269+0000", 2024-04-26T03:38:55.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:55.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.507269+0000", 2024-04-26T03:38:55.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8999999999999995e-07 2024-04-26T03:38:55.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:55.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.507364+0000", 2024-04-26T03:38:55.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4493000000000005e-05 2024-04-26T03:38:55.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:55.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.507379+0000", 2024-04-26T03:38:55.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.499e-05 2024-04-26T03:38:55.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:55.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.507388+0000", 2024-04-26T03:38:55.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4010000000000003e-06 2024-04-26T03:38:55.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.507479+0000", 2024-04-26T03:38:55.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0204000000000006e-05 2024-04-26T03:38:55.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:55.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.507488+0000", 2024-04-26T03:38:55.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5829999999999996e-06 2024-04-26T03:38:55.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:55.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.511761+0000", 2024-04-26T03:38:55.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0042725790000000003 2024-04-26T03:38:55.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:55.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.511793+0000", 2024-04-26T03:38:55.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2421e-05 2024-04-26T03:38:55.271 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.271 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:55.272 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.272 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.es5 35/32 ECSubWrite(tid=2, reqid=client.4556.0:58, at_version=35'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:38:55.272 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.513076+0000", 2024-04-26T03:38:55.272 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.7286145619999997, 2024-04-26T03:38:55.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0049094890000000004, 2024-04-26T03:38:55.272 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:55.272 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:55.272 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:55.272 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:55.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:55.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.513076+0000", 2024-04-26T03:38:55.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:55.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.513076+0000", 2024-04-26T03:38:55.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:55.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.513077+0000", 2024-04-26T03:38:55.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5000000000000001e-07 2024-04-26T03:38:55.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:55.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.513176+0000", 2024-04-26T03:38:55.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9588999999999998e-05 2024-04-26T03:38:55.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:55.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.513210+0000", 2024-04-26T03:38:55.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3800000000000002e-05 2024-04-26T03:38:55.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:55.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.513219+0000", 2024-04-26T03:38:55.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4050000000000006e-06 2024-04-26T03:38:55.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.513378+0000", 2024-04-26T03:38:55.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015877600000000001 2024-04-26T03:38:55.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:55.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.513388+0000", 2024-04-26T03:38:55.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4919999999999999e-06 2024-04-26T03:38:55.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:55.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.517940+0000", 2024-04-26T03:38:55.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0045520250000000003 2024-04-26T03:38:55.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:55.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.517985+0000", 2024-04-26T03:38:55.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5552e-05 2024-04-26T03:38:55.276 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.276 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:55.276 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.276 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.es5 35/32 ECSubWrite(tid=8, reqid=client.4556.0:70, at_version=35'3, trim_to=0'0, roll_forward_to=35'2))", 2024-04-26T03:38:55.276 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.523220+0000", 2024-04-26T03:38:55.276 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.7184703949999998, 2024-04-26T03:38:55.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035939510000000002, 2024-04-26T03:38:55.276 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:55.277 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:55.277 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:55.277 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:55.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:55.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.523220+0000", 2024-04-26T03:38:55.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:55.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.523220+0000", 2024-04-26T03:38:55.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:55.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.523221+0000", 2024-04-26T03:38:55.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1850000000000001e-06 2024-04-26T03:38:55.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:55.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.523321+0000", 2024-04-26T03:38:55.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9692999999999996e-05 2024-04-26T03:38:55.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:55.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.523340+0000", 2024-04-26T03:38:55.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9145000000000001e-05 2024-04-26T03:38:55.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:55.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.523349+0000", 2024-04-26T03:38:55.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3729999999999999e-06 2024-04-26T03:38:55.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.523411+0000", 2024-04-26T03:38:55.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1328999999999995e-05 2024-04-26T03:38:55.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:55.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.523420+0000", 2024-04-26T03:38:55.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2529999999999993e-06 2024-04-26T03:38:55.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:55.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.526775+0000", 2024-04-26T03:38:55.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033546769999999999 2024-04-26T03:38:55.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:55.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.526814+0000", 2024-04-26T03:38:55.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9295999999999998e-05 2024-04-26T03:38:55.280 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.280 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:55.280 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.280 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds2 35/32 ECSubWrite(tid=7, reqid=client.4556.0:64, at_version=35'5, trim_to=0'0, roll_forward_to=35'4))", 2024-04-26T03:38:55.281 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.528723+0000", 2024-04-26T03:38:55.281 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.7129669569999999, 2024-04-26T03:38:55.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.004601511, 2024-04-26T03:38:55.281 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:55.281 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:55.281 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:55.281 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:55.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:55.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528723+0000", 2024-04-26T03:38:55.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:55.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528723+0000", 2024-04-26T03:38:55.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:55.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528726+0000", 2024-04-26T03:38:55.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6560000000000001e-06 2024-04-26T03:38:55.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:55.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528846+0000", 2024-04-26T03:38:55.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120235 2024-04-26T03:38:55.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:55.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528861+0000", 2024-04-26T03:38:55.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4697e-05 2024-04-26T03:38:55.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:55.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528869+0000", 2024-04-26T03:38:55.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3399999999999998e-06 2024-04-26T03:38:55.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528937+0000", 2024-04-26T03:38:55.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8016999999999996e-05 2024-04-26T03:38:55.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:55.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528946+0000", 2024-04-26T03:38:55.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5290000000000008e-06 2024-04-26T03:38:55.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:55.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.533280+0000", 2024-04-26T03:38:55.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.004333801 2024-04-26T03:38:55.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:55.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.533325+0000", 2024-04-26T03:38:55.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5235999999999999e-05 2024-04-26T03:38:55.285 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.285 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:55.285 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.285 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.9s3 35/32 ECSubWrite(tid=10, reqid=client.4556.0:76, at_version=35'3, trim_to=0'0, roll_forward_to=35'2))", 2024-04-26T03:38:55.285 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.530631+0000", 2024-04-26T03:38:55.285 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.7110592369999997, 2024-04-26T03:38:55.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0058993190000000001, 2024-04-26T03:38:55.285 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:55.285 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:55.285 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:55.285 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:55.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:55.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.530631+0000", 2024-04-26T03:38:55.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:55.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.530631+0000", 2024-04-26T03:38:55.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:55.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.530632+0000", 2024-04-26T03:38:55.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3900000000000004e-07 2024-04-26T03:38:55.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:55.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.531212+0000", 2024-04-26T03:38:55.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000579979 2024-04-26T03:38:55.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:55.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.531238+0000", 2024-04-26T03:38:55.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5927000000000001e-05 2024-04-26T03:38:55.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:55.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.531247+0000", 2024-04-26T03:38:55.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8519999999999993e-06 2024-04-26T03:38:55.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.531314+0000", 2024-04-26T03:38:55.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7070000000000007e-05 2024-04-26T03:38:55.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:55.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.531322+0000", 2024-04-26T03:38:55.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6859999999999995e-06 2024-04-26T03:38:55.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:55.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.536494+0000", 2024-04-26T03:38:55.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0051716319999999998 2024-04-26T03:38:55.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:55.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.536530+0000", 2024-04-26T03:38:55.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6334000000000001e-05 2024-04-26T03:38:55.289 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.289 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:55.289 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.289 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.2s3 35/32 ECSubWrite(tid=4, reqid=client.4556.0:79, at_version=35'3, trim_to=0'0, roll_forward_to=34'2))", 2024-04-26T03:38:55.289 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.533108+0000", 2024-04-26T03:38:55.289 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.708582126, 2024-04-26T03:38:55.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0051759689999999999, 2024-04-26T03:38:55.290 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:55.290 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:55.290 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:55.290 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:55.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:55.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.533108+0000", 2024-04-26T03:38:55.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:55.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.533108+0000", 2024-04-26T03:38:55.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:55.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.533109+0000", 2024-04-26T03:38:55.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6200000000000006e-07 2024-04-26T03:38:55.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:55.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.533421+0000", 2024-04-26T03:38:55.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00031138699999999999 2024-04-26T03:38:55.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:55.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.533433+0000", 2024-04-26T03:38:55.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2619000000000001e-05 2024-04-26T03:38:55.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:55.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.533441+0000", 2024-04-26T03:38:55.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2349999999999999e-06 2024-04-26T03:38:55.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.533511+0000", 2024-04-26T03:38:55.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9374000000000004e-05 2024-04-26T03:38:55.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:55.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.533520+0000", 2024-04-26T03:38:55.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3249999999999997e-06 2024-04-26T03:38:55.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:55.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.538235+0000", 2024-04-26T03:38:55.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0047147480000000004 2024-04-26T03:38:55.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:55.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.538284+0000", 2024-04-26T03:38:55.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9319e-05 2024-04-26T03:38:55.293 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.293 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:55.293 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.294 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.es5 35/32 ECSubWrite(tid=16, reqid=client.4556.0:85, at_version=35'5, trim_to=0'0, roll_forward_to=35'4))", 2024-04-26T03:38:55.294 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.542163+0000", 2024-04-26T03:38:55.294 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.6995275200000002, 2024-04-26T03:38:55.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0039697810000000004, 2024-04-26T03:38:55.294 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:55.294 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:55.294 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:55.294 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:55.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:55.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.542163+0000", 2024-04-26T03:38:55.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:55.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.542163+0000", 2024-04-26T03:38:55.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:55.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.542164+0000", 2024-04-26T03:38:55.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4799999999999997e-07 2024-04-26T03:38:55.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:55.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.542257+0000", 2024-04-26T03:38:55.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3525999999999999e-05 2024-04-26T03:38:55.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:55.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.542269+0000", 2024-04-26T03:38:55.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2386e-05 2024-04-26T03:38:55.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:55.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.542276+0000", 2024-04-26T03:38:55.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0859999999999998e-06 2024-04-26T03:38:55.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.542337+0000", 2024-04-26T03:38:55.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1180999999999996e-05 2024-04-26T03:38:55.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:55.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.542344+0000", 2024-04-26T03:38:55.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0960000000000003e-06 2024-04-26T03:38:55.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:55.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.546093+0000", 2024-04-26T03:38:55.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037496650000000001 2024-04-26T03:38:55.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.298 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:55.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.546133+0000", 2024-04-26T03:38:55.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9093e-05 2024-04-26T03:38:55.298 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.298 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:55.298 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.298 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.9s3 35/32 ECSubWrite(tid=20, reqid=client.4556.0:97, at_version=35'5, trim_to=0'0, roll_forward_to=35'4))", 2024-04-26T03:38:55.298 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.556034+0000", 2024-04-26T03:38:55.298 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.6856558330000002, 2024-04-26T03:38:55.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.004447622, 2024-04-26T03:38:55.298 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:55.299 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:55.299 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:55.299 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:55.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:55.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.556034+0000", 2024-04-26T03:38:55.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:55.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.556034+0000", 2024-04-26T03:38:55.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.300 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:55.300 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.556036+0000", 2024-04-26T03:38:55.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0249999999999999e-06 2024-04-26T03:38:55.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.300 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:55.300 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.556161+0000", 2024-04-26T03:38:55.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000125485 2024-04-26T03:38:55.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.300 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:55.300 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.556175+0000", 2024-04-26T03:38:55.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3787e-05 2024-04-26T03:38:55.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:55.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.556181+0000", 2024-04-26T03:38:55.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2489999999999998e-06 2024-04-26T03:38:55.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.556242+0000", 2024-04-26T03:38:55.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0613999999999999e-05 2024-04-26T03:38:55.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:55.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.556247+0000", 2024-04-26T03:38:55.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.75e-06 2024-04-26T03:38:55.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:55.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.560442+0000", 2024-04-26T03:38:55.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0041950859999999998 2024-04-26T03:38:55.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:55.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.560482+0000", 2024-04-26T03:38:55.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9626000000000003e-05 2024-04-26T03:38:55.302 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.302 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:55.302 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.303 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.1s4 35/32 ECSubWrite(tid=16, reqid=client.4556.0:118, at_version=35'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:38:55.303 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.585779+0000", 2024-04-26T03:38:55.303 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.65591176, 2024-04-26T03:38:55.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0052872639999999999, 2024-04-26T03:38:55.303 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:55.303 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:55.303 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:55.303 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:55.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.303 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:55.303 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.585779+0000", 2024-04-26T03:38:55.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.304 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:55.304 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.585779+0000", 2024-04-26T03:38:55.304 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.304 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:55.304 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.585779+0000", 2024-04-26T03:38:55.304 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2100000000000001e-07 2024-04-26T03:38:55.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.304 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:55.304 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.586398+0000", 2024-04-26T03:38:55.304 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00061936399999999996 2024-04-26T03:38:55.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.305 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:55.305 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.586412+0000", 2024-04-26T03:38:55.305 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4246e-05 2024-04-26T03:38:55.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.305 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:55.305 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.586422+0000", 2024-04-26T03:38:55.305 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3919999999999992e-06 2024-04-26T03:38:55.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.305 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.305 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.586496+0000", 2024-04-26T03:38:55.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4559000000000002e-05 2024-04-26T03:38:55.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:55.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.586506+0000", 2024-04-26T03:38:55.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0122999999999999e-05 2024-04-26T03:38:55.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:55.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.591034+0000", 2024-04-26T03:38:55.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0045271699999999996 2024-04-26T03:38:55.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:55.307 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.591066+0000", 2024-04-26T03:38:55.307 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2289000000000002e-05 2024-04-26T03:38:55.307 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.307 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:55.307 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.307 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.es5 35/32 ECSubWrite(tid=24, reqid=client.4556.0:124, at_version=35'7, trim_to=0'0, roll_forward_to=35'6))", 2024-04-26T03:38:55.307 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.596016+0000", 2024-04-26T03:38:55.307 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.6456738789999998, 2024-04-26T03:38:55.307 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0052964839999999997, 2024-04-26T03:38:55.307 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:55.307 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:55.307 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:55.308 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:55.308 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.308 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:55.308 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.596016+0000", 2024-04-26T03:38:55.308 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.308 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.308 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:55.308 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.596016+0000", 2024-04-26T03:38:55.308 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.308 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.308 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:55.308 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.596022+0000", 2024-04-26T03:38:55.308 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3530000000000002e-06 2024-04-26T03:38:55.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.309 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:55.309 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.596141+0000", 2024-04-26T03:38:55.309 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011927 2024-04-26T03:38:55.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.309 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:55.309 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.596154+0000", 2024-04-26T03:38:55.309 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2903e-05 2024-04-26T03:38:55.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.309 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:55.310 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.596161+0000", 2024-04-26T03:38:55.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1119999999999998e-06 2024-04-26T03:38:55.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.310 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.310 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.596246+0000", 2024-04-26T03:38:55.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5056999999999996e-05 2024-04-26T03:38:55.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.310 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:55.310 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.596264+0000", 2024-04-26T03:38:55.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7844000000000001e-05 2024-04-26T03:38:55.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:55.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.601278+0000", 2024-04-26T03:38:55.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0050136 2024-04-26T03:38:55.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:55.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.601313+0000", 2024-04-26T03:38:55.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5345000000000001e-05 2024-04-26T03:38:55.311 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.311 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:55.311 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.311 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.2s3 35/32 ECSubWrite(tid=8, reqid=client.4556.0:139, at_version=35'5, trim_to=0'0, roll_forward_to=35'4))", 2024-04-26T03:38:55.312 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.621036+0000", 2024-04-26T03:38:55.312 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.6206545229999998, 2024-04-26T03:38:55.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0056652040000000001, 2024-04-26T03:38:55.312 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:55.312 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:55.312 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:55.312 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:55.312 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.312 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:55.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.621036+0000", 2024-04-26T03:38:55.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.312 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.312 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.312 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:55.313 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.621036+0000", 2024-04-26T03:38:55.313 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.313 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.313 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:55.313 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.621037+0000", 2024-04-26T03:38:55.313 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.78e-07 2024-04-26T03:38:55.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.313 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.313 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:55.313 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.621467+0000", 2024-04-26T03:38:55.313 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00043015799999999998 2024-04-26T03:38:55.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.314 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:55.314 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.621481+0000", 2024-04-26T03:38:55.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3978999999999999e-05 2024-04-26T03:38:55.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.314 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:55.314 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.621491+0000", 2024-04-26T03:38:55.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7010000000000008e-06 2024-04-26T03:38:55.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.314 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.314 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.621571+0000", 2024-04-26T03:38:55.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0408000000000007e-05 2024-04-26T03:38:55.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:55.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.621581+0000", 2024-04-26T03:38:55.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0079e-05 2024-04-26T03:38:55.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:55.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.626670+0000", 2024-04-26T03:38:55.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0050891920000000002 2024-04-26T03:38:55.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:55.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.626701+0000", 2024-04-26T03:38:55.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0809000000000003e-05 2024-04-26T03:38:55.316 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.316 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:55.316 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.316 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.0s1 35/32 ECSubWrite(tid=9, reqid=client.4556.0:173, at_version=35'6, trim_to=0'0, roll_forward_to=35'5))", 2024-04-26T03:38:55.316 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.722917+0000", 2024-04-26T03:38:55.316 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.5187728639999998, 2024-04-26T03:38:55.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0034330810000000002, 2024-04-26T03:38:55.316 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:55.316 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:55.316 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:55.316 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:55.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.317 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:55.317 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.722917+0000", 2024-04-26T03:38:55.317 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.317 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:55.317 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.722917+0000", 2024-04-26T03:38:55.317 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.317 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:55.317 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.722918+0000", 2024-04-26T03:38:55.317 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9999999999999997e-07 2024-04-26T03:38:55.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.318 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:55.318 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.722971+0000", 2024-04-26T03:38:55.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2465e-05 2024-04-26T03:38:55.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.318 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:55.318 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.722985+0000", 2024-04-26T03:38:55.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4592e-05 2024-04-26T03:38:55.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.318 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:55.319 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.722991+0000", 2024-04-26T03:38:55.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1569999999999997e-06 2024-04-26T03:38:55.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.319 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.319 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.723049+0000", 2024-04-26T03:38:55.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7661999999999998e-05 2024-04-26T03:38:55.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.319 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:55.319 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.723059+0000", 2024-04-26T03:38:55.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5400000000000001e-06 2024-04-26T03:38:55.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.320 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:55.320 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.726319+0000", 2024-04-26T03:38:55.320 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00326049 2024-04-26T03:38:55.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.320 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:55.320 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.726351+0000", 2024-04-26T03:38:55.320 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1474999999999999e-05 2024-04-26T03:38:55.320 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.320 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:55.320 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.320 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.0s1 35/32 ECSubWrite(tid=10, reqid=client.4556.0:174, at_version=35'7, trim_to=0'0, roll_forward_to=35'5))", 2024-04-26T03:38:55.321 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.726352+0000", 2024-04-26T03:38:55.321 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.51533856, 2024-04-26T03:38:55.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0026513320000000002, 2024-04-26T03:38:55.321 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:55.321 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:55.321 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:55.321 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:55.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.321 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:55.321 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.726352+0000", 2024-04-26T03:38:55.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.322 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:55.322 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.726352+0000", 2024-04-26T03:38:55.322 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.322 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:55.322 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.726353+0000", 2024-04-26T03:38:55.322 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7300000000000005e-07 2024-04-26T03:38:55.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.322 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:55.322 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.726422+0000", 2024-04-26T03:38:55.322 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8974999999999996e-05 2024-04-26T03:38:55.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.323 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:55.323 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.726432+0000", 2024-04-26T03:38:55.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0288999999999999e-05 2024-04-26T03:38:55.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.323 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:55.323 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.726437+0000", 2024-04-26T03:38:55.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4500000000000003e-06 2024-04-26T03:38:55.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.323 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.323 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.726475+0000", 2024-04-26T03:38:55.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7834999999999999e-05 2024-04-26T03:38:55.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:55.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.726482+0000", 2024-04-26T03:38:55.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8789999999999997e-06 2024-04-26T03:38:55.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:55.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.728959+0000", 2024-04-26T03:38:55.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024773730000000002 2024-04-26T03:38:55.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:55.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.729003+0000", 2024-04-26T03:38:55.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3757999999999997e-05 2024-04-26T03:38:55.324 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.325 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:55.325 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.325 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.es5 35/32 ECSubWrite(tid=34, reqid=client.4556.0:177, at_version=35'9, trim_to=0'0, roll_forward_to=35'8))", 2024-04-26T03:38:55.325 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.731557+0000", 2024-04-26T03:38:55.325 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.510133696, 2024-04-26T03:38:55.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.004083878, 2024-04-26T03:38:55.325 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:55.325 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:55.325 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:55.325 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:55.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:55.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.731557+0000", 2024-04-26T03:38:55.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.326 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:55.326 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.731557+0000", 2024-04-26T03:38:55.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.326 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:55.326 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.731557+0000", 2024-04-26T03:38:55.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3399999999999999e-07 2024-04-26T03:38:55.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.326 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:55.326 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.731689+0000", 2024-04-26T03:38:55.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013168000000000001 2024-04-26T03:38:55.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.327 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:55.327 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.731709+0000", 2024-04-26T03:38:55.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.986e-05 2024-04-26T03:38:55.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.327 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:55.327 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.731722+0000", 2024-04-26T03:38:55.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2982999999999999e-05 2024-04-26T03:38:55.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.327 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.328 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.731821+0000", 2024-04-26T03:38:55.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8851000000000006e-05 2024-04-26T03:38:55.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.328 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:55.328 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.731836+0000", 2024-04-26T03:38:55.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5319999999999999e-05 2024-04-26T03:38:55.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.328 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:55.328 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.735616+0000", 2024-04-26T03:38:55.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003779784 2024-04-26T03:38:55.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.328 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:55.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.735640+0000", 2024-04-26T03:38:55.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4766e-05 2024-04-26T03:38:55.329 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.329 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:55.329 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.329 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.es5 35/32 ECSubWrite(tid=35, reqid=client.4556.0:178, at_version=35'10, trim_to=0'0, roll_forward_to=35'8))", 2024-04-26T03:38:55.329 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.735899+0000", 2024-04-26T03:38:55.329 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.5057909010000001, 2024-04-26T03:38:55.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023893500000000002, 2024-04-26T03:38:55.329 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:55.329 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:55.329 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:38:55.329 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:55.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:55.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.735899+0000", 2024-04-26T03:38:55.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:55.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.735899+0000", 2024-04-26T03:38:55.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:55.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.735900+0000", 2024-04-26T03:38:55.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.75e-07 2024-04-26T03:38:55.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:55.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.735935+0000", 2024-04-26T03:38:55.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4542000000000002e-05 2024-04-26T03:38:55.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:55.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.735944+0000", 2024-04-26T03:38:55.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.713e-06 2024-04-26T03:38:55.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:55.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.735950+0000", 2024-04-26T03:38:55.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3410000000000002e-06 2024-04-26T03:38:55.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.332 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.735995+0000", 2024-04-26T03:38:55.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5893000000000003e-05 2024-04-26T03:38:55.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:55.332 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.736001+0000", 2024-04-26T03:38:55.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.186e-06 2024-04-26T03:38:55.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:55.332 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.738257+0000", 2024-04-26T03:38:55.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022561410000000001 2024-04-26T03:38:55.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.333 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:55.333 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.738289+0000", 2024-04-26T03:38:55.333 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1958999999999997e-05 2024-04-26T03:38:55.333 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.333 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:55.333 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.333 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.333 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:55.333 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-26T03:38:55.335 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-04-26T03:38:55.335 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-04-26T03:38:55.333+0000 7fdc2d21c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31819) UID: 0 2024-04-26T03:38:55.435 INFO:tasks.ceph.osd.9.smithi042.stderr:2024-04-26T03:38:55.434+0000 7fdac63b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 31960) UID: 0 2024-04-26T03:38:55.483 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-26T03:38:55.483 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-04-26T03:38:55.483 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-04-26T03:38:55.484 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-26T03:38:55.492 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-04-26T03:38:55.505 INFO:tasks.rados.rados.0.smithi092.stdout:96: done (1 left) 2024-04-26T03:38:55.505 INFO:tasks.rados.rados.0.smithi092.stdout:97: delete oid 35 current snap is 3 2024-04-26T03:38:55.505 INFO:tasks.rados.rados.0.smithi092.stdout:95: expect (ObjNum 37 snap 0 seq_num 37) 2024-04-26T03:38:55.506 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-26T03:38:55.524 INFO:tasks.rados.rados.0.smithi092.stdout:95: done (1 left) 2024-04-26T03:38:55.524 INFO:tasks.rados.rados.0.smithi092.stdout:97: done (0 left) 2024-04-26T03:38:55.524 INFO:tasks.rados.rados.0.smithi092.stdout:98: snap_remove snap 1 2024-04-26T03:38:55.576 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:38:55.574+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:38:55.637 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:38:55.636+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:38:55.638 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-26T03:38:55.638 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-04-26T03:38:55.638 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-04-26T03:38:55.638 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-04-26T03:38:55.638 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-26T03:38:55.647 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-04-26T03:38:55.739 INFO:tasks.ceph.osd.10.smithi066.stderr:2024-04-26T03:38:55.737+0000 7f130fcc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 31763) UID: 0 2024-04-26T03:38:55.795 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-26T03:38:55.795 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-04-26T03:38:55.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-04-26T03:38:55.796 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-04-26T03:38:55.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.796 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:46 3.9s0 3:9d1726ae:::smithi09235267-16:head [append 0~262144,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:38:55.796 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.502792+0000", 2024-04-26T03:38:55.796 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.2899339779999996, 2024-04-26T03:38:55.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0098310189999999999, 2024-04-26T03:38:55.796 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:55.796 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:55.796 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:38:55.796 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:38:55.796 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:38:55.796 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:38:55.796 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 46 2024-04-26T03:38:55.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.797 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:55.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:55.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.502792+0000", 2024-04-26T03:38:55.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:55.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.502792+0000", 2024-04-26T03:38:55.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:55.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.502794+0000", 2024-04-26T03:38:55.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0880000000000002e-06 2024-04-26T03:38:55.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:55.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.503889+0000", 2024-04-26T03:38:55.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010954630000000001 2024-04-26T03:38:55.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:55.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.503904+0000", 2024-04-26T03:38:55.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4773000000000001e-05 2024-04-26T03:38:55.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:55.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.503912+0000", 2024-04-26T03:38:55.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2349999999999999e-06 2024-04-26T03:38:55.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504048+0000", 2024-04-26T03:38:55.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000135723 2024-04-26T03:38:55.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:38:55.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504198+0000", 2024-04-26T03:38:55.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015004100000000001 2024-04-26T03:38:55.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:55.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.505280+0000", 2024-04-26T03:38:55.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010818920000000001 2024-04-26T03:38:55.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:55.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.509534+0000", 2024-04-26T03:38:55.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0042545109999999999 2024-04-26T03:38:55.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:38:55.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.512541+0000", 2024-04-26T03:38:55.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0030064409999999999 2024-04-26T03:38:55.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:55.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.512623+0000", 2024-04-26T03:38:55.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1852000000000002e-05 2024-04-26T03:38:55.801 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.801 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:55.801 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.801 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:47 3.9s0 3:9d1726ae:::smithi09235267-16:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:38:55.801 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.504120+0000", 2024-04-26T03:38:55.801 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.2886056740000003, 2024-04-26T03:38:55.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011964519999999999, 2024-04-26T03:38:55.801 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:55.801 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:55.802 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:38:55.802 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:38:55.802 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:38:55.802 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:38:55.802 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47 2024-04-26T03:38:55.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.802 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:55.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:55.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504120+0000", 2024-04-26T03:38:55.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:55.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504120+0000", 2024-04-26T03:38:55.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:55.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504121+0000", 2024-04-26T03:38:55.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0359999999999999e-06 2024-04-26T03:38:55.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:55.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504127+0000", 2024-04-26T03:38:55.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1750000000000002e-06 2024-04-26T03:38:55.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:55.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504138+0000", 2024-04-26T03:38:55.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1045e-05 2024-04-26T03:38:55.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:55.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504145+0000", 2024-04-26T03:38:55.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5919999999999997e-06 2024-04-26T03:38:55.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.508573+0000", 2024-04-26T03:38:55.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.004427762 2024-04-26T03:38:55.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:38:55.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.508680+0000", 2024-04-26T03:38:55.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000107106 2024-04-26T03:38:55.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:55.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.509206+0000", 2024-04-26T03:38:55.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00052654999999999998 2024-04-26T03:38:55.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:55.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.510170+0000", 2024-04-26T03:38:55.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00096335099999999999 2024-04-26T03:38:55.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:38:55.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.515986+0000", 2024-04-26T03:38:55.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0058160449999999997 2024-04-26T03:38:55.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:55.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.516084+0000", 2024-04-26T03:38:55.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8857999999999994e-05 2024-04-26T03:38:55.806 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.806 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:55.806 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.807 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:48 3.9s0 3:9d1726ae:::smithi09235267-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:38:55.807 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.504164+0000", 2024-04-26T03:38:55.807 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.2885617529999998, 2024-04-26T03:38:55.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.014750030000000001, 2024-04-26T03:38:55.807 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:55.807 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:55.807 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-04-26T03:38:55.807 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:38:55.807 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:38:55.807 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:38:55.807 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 48 2024-04-26T03:38:55.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.807 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:55.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:55.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504164+0000", 2024-04-26T03:38:55.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:55.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504164+0000", 2024-04-26T03:38:55.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:55.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504165+0000", 2024-04-26T03:38:55.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8999999999999996e-07 2024-04-26T03:38:55.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:55.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504169+0000", 2024-04-26T03:38:55.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0239999999999999e-06 2024-04-26T03:38:55.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:55.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504178+0000", 2024-04-26T03:38:55.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7610000000000002e-06 2024-04-26T03:38:55.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:55.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504187+0000", 2024-04-26T03:38:55.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1799999999999996e-06 2024-04-26T03:38:55.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.509439+0000", 2024-04-26T03:38:55.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0052526150000000004 2024-04-26T03:38:55.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-04-26T03:38:55.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.509517+0000", 2024-04-26T03:38:55.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7417999999999996e-05 2024-04-26T03:38:55.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.516457+0000", 2024-04-26T03:38:55.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0069402259999999999 2024-04-26T03:38:55.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:38:55.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.516564+0000", 2024-04-26T03:38:55.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010746799999999999 2024-04-26T03:38:55.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:55.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.518914+0000", 2024-04-26T03:38:55.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023496480000000002 2024-04-26T03:38:55.811 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.811 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:55.811 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.812 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:58 3.es0 3:7db045cf:::smithi09235267-20:head [append 0~327680,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:38:55.812 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.511421+0000", 2024-04-26T03:38:55.812 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.2813049110000003, 2024-04-26T03:38:55.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0096719909999999996, 2024-04-26T03:38:55.812 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:55.812 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:55.812 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:38:55.812 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:38:55.812 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:38:55.812 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:38:55.812 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 58 2024-04-26T03:38:55.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.812 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:55.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:55.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.511421+0000", 2024-04-26T03:38:55.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:55.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.511421+0000", 2024-04-26T03:38:55.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:55.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.511422+0000", 2024-04-26T03:38:55.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6670000000000001e-06 2024-04-26T03:38:55.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:55.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.511671+0000", 2024-04-26T03:38:55.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024866999999999999 2024-04-26T03:38:55.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:55.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.511682+0000", 2024-04-26T03:38:55.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0864e-05 2024-04-26T03:38:55.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:55.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.511689+0000", 2024-04-26T03:38:55.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8449999999999997e-06 2024-04-26T03:38:55.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.511752+0000", 2024-04-26T03:38:55.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2955999999999998e-05 2024-04-26T03:38:55.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:38:55.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.511883+0000", 2024-04-26T03:38:55.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013119 2024-04-26T03:38:55.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:55.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.512675+0000", 2024-04-26T03:38:55.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00079197800000000004 2024-04-26T03:38:55.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:55.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.517472+0000", 2024-04-26T03:38:55.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0047972329999999997 2024-04-26T03:38:55.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:38:55.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.521023+0000", 2024-04-26T03:38:55.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035506299999999999 2024-04-26T03:38:55.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:55.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.521093+0000", 2024-04-26T03:38:55.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9957999999999997e-05 2024-04-26T03:38:55.817 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.817 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:55.817 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.817 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:60 3.es0 3:7db045cf:::smithi09235267-20:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:38:55.817 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.512353+0000", 2024-04-26T03:38:55.817 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.2803723040000001, 2024-04-26T03:38:55.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015276305, 2024-04-26T03:38:55.817 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:55.817 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:55.817 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-04-26T03:38:55.817 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:38:55.818 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:38:55.818 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:38:55.818 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 60 2024-04-26T03:38:55.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.818 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:55.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:55.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.512353+0000", 2024-04-26T03:38:55.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:55.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.512353+0000", 2024-04-26T03:38:55.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:55.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.512354+0000", 2024-04-26T03:38:55.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3600000000000003e-07 2024-04-26T03:38:55.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:55.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.512357+0000", 2024-04-26T03:38:55.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3050000000000001e-06 2024-04-26T03:38:55.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:55.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.512365+0000", 2024-04-26T03:38:55.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1950000000000003e-06 2024-04-26T03:38:55.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:55.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.512372+0000", 2024-04-26T03:38:55.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1589999999999998e-06 2024-04-26T03:38:55.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.517366+0000", 2024-04-26T03:38:55.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0049946269999999997 2024-04-26T03:38:55.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-04-26T03:38:55.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.517461+0000", 2024-04-26T03:38:55.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4529000000000002e-05 2024-04-26T03:38:55.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.521520+0000", 2024-04-26T03:38:55.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0040588519999999999 2024-04-26T03:38:55.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:38:55.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.521609+0000", 2024-04-26T03:38:55.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8967000000000004e-05 2024-04-26T03:38:55.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:55.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.527630+0000", 2024-04-26T03:38:55.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0060210350000000001 2024-04-26T03:38:55.822 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.822 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:55.822 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.822 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:70 3.es0 3:75fafc97:::smithi09235267-24:head [append 0~393216,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:38:55.822 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.521212+0000", 2024-04-26T03:38:55.822 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.2715141069999998, 2024-04-26T03:38:55.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0094723229999999995, 2024-04-26T03:38:55.822 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:55.822 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:55.822 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:38:55.823 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:38:55.823 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:38:55.823 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:38:55.823 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 70 2024-04-26T03:38:55.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.823 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:55.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:55.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.521212+0000", 2024-04-26T03:38:55.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:55.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.521212+0000", 2024-04-26T03:38:55.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:55.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.521213+0000", 2024-04-26T03:38:55.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.187e-06 2024-04-26T03:38:55.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:55.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.521616+0000", 2024-04-26T03:38:55.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00040302399999999998 2024-04-26T03:38:55.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:55.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.521627+0000", 2024-04-26T03:38:55.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0944999999999999e-05 2024-04-26T03:38:55.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:55.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.521635+0000", 2024-04-26T03:38:55.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9500000000000001e-06 2024-04-26T03:38:55.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.521897+0000", 2024-04-26T03:38:55.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026204599999999999 2024-04-26T03:38:55.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:38:55.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.522046+0000", 2024-04-26T03:38:55.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000149717 2024-04-26T03:38:55.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:55.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.522934+0000", 2024-04-26T03:38:55.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00088756900000000005 2024-04-26T03:38:55.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:55.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.526998+0000", 2024-04-26T03:38:55.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0040638710000000001 2024-04-26T03:38:55.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:38:55.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.530608+0000", 2024-04-26T03:38:55.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0036101359999999999 2024-04-26T03:38:55.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:55.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.530684+0000", 2024-04-26T03:38:55.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5877999999999994e-05 2024-04-26T03:38:55.828 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.828 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:55.828 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.828 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:71 3.es0 3:75fafc97:::smithi09235267-24:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:38:55.828 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.521650+0000", 2024-04-26T03:38:55.828 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.2710761100000001, 2024-04-26T03:38:55.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0092613739999999993, 2024-04-26T03:38:55.829 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:55.829 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:55.829 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:38:55.829 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:38:55.829 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:38:55.829 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:38:55.829 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 71 2024-04-26T03:38:55.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.829 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:55.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:55.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.521650+0000", 2024-04-26T03:38:55.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:55.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.521650+0000", 2024-04-26T03:38:55.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:55.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.521650+0000", 2024-04-26T03:38:55.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0599999999999999e-07 2024-04-26T03:38:55.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:55.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.521656+0000", 2024-04-26T03:38:55.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2619999999999998e-06 2024-04-26T03:38:55.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:55.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.521664+0000", 2024-04-26T03:38:55.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1310000000000006e-06 2024-04-26T03:38:55.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:55.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.521669+0000", 2024-04-26T03:38:55.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7570000000000001e-06 2024-04-26T03:38:55.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.527686+0000", 2024-04-26T03:38:55.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0060168849999999996 2024-04-26T03:38:55.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:38:55.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.527785+0000", 2024-04-26T03:38:55.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8313000000000006e-05 2024-04-26T03:38:55.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:55.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528279+0000", 2024-04-26T03:38:55.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00049389699999999998 2024-04-26T03:38:55.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:55.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.529192+0000", 2024-04-26T03:38:55.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00091292700000000001 2024-04-26T03:38:55.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:38:55.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.530832+0000", 2024-04-26T03:38:55.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001640915 2024-04-26T03:38:55.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:55.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.530911+0000", 2024-04-26T03:38:55.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8480999999999996e-05 2024-04-26T03:38:55.837 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.837 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:55.837 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.837 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:72 3.es0 3:75fafc97:::smithi09235267-24:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:38:55.837 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.521677+0000", 2024-04-26T03:38:55.837 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.271048661, 2024-04-26T03:38:55.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012551157, 2024-04-26T03:38:55.838 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:55.838 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:55.838 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-04-26T03:38:55.838 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:38:55.838 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:38:55.838 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:38:55.838 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 72 2024-04-26T03:38:55.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.838 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:55.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:55.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.521677+0000", 2024-04-26T03:38:55.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:55.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.521677+0000", 2024-04-26T03:38:55.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:55.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.521677+0000", 2024-04-26T03:38:55.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.15e-07 2024-04-26T03:38:55.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:55.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.521679+0000", 2024-04-26T03:38:55.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9429999999999999e-06 2024-04-26T03:38:55.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:55.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.521684+0000", 2024-04-26T03:38:55.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3610000000000003e-06 2024-04-26T03:38:55.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:55.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.521688+0000", 2024-04-26T03:38:55.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.138e-06 2024-04-26T03:38:55.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528505+0000", 2024-04-26T03:38:55.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0068170679999999999 2024-04-26T03:38:55.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-04-26T03:38:55.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528576+0000", 2024-04-26T03:38:55.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1507000000000005e-05 2024-04-26T03:38:55.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.531827+0000", 2024-04-26T03:38:55.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003250883 2024-04-26T03:38:55.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:38:55.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.531903+0000", 2024-04-26T03:38:55.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5957999999999993e-05 2024-04-26T03:38:55.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:55.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.534228+0000", 2024-04-26T03:38:55.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023248840000000002 2024-04-26T03:38:55.842 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.842 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:55.843 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.843 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:76 3.9s0 3:9da60d2b:::smithi09235267-26:head [append 0~425984,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:38:55.843 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.527914+0000", 2024-04-26T03:38:55.843 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.2648119470000001, 2024-04-26T03:38:55.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0097968980000000001, 2024-04-26T03:38:55.843 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:55.843 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:55.843 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:38:55.843 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:38:55.843 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:38:55.843 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:38:55.843 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 76 2024-04-26T03:38:55.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.844 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:55.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:55.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.527914+0000", 2024-04-26T03:38:55.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:55.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.527914+0000", 2024-04-26T03:38:55.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:55.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.527915+0000", 2024-04-26T03:38:55.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.542e-06 2024-04-26T03:38:55.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:55.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528327+0000", 2024-04-26T03:38:55.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00041166999999999999 2024-04-26T03:38:55.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:55.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528341+0000", 2024-04-26T03:38:55.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3597e-05 2024-04-26T03:38:55.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:55.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528350+0000", 2024-04-26T03:38:55.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1200000000000008e-06 2024-04-26T03:38:55.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528487+0000", 2024-04-26T03:38:55.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013783099999999999 2024-04-26T03:38:55.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:38:55.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528647+0000", 2024-04-26T03:38:55.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015980700000000001 2024-04-26T03:38:55.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:55.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.529618+0000", 2024-04-26T03:38:55.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00097114800000000002 2024-04-26T03:38:55.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:55.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.536639+0000", 2024-04-26T03:38:55.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0070201819999999998 2024-04-26T03:38:55.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:38:55.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.537637+0000", 2024-04-26T03:38:55.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00099828999999999998 2024-04-26T03:38:55.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:55.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.537711+0000", 2024-04-26T03:38:55.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3711e-05 2024-04-26T03:38:55.848 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.848 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:55.848 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.848 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:77 3.9s0 3:9da60d2b:::smithi09235267-26:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:38:55.848 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.528398+0000", 2024-04-26T03:38:55.848 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.2643272789999997, 2024-04-26T03:38:55.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0097093170000000003, 2024-04-26T03:38:55.848 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:55.848 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:55.848 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:38:55.849 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:38:55.849 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:38:55.849 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:38:55.849 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 77 2024-04-26T03:38:55.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.849 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:55.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:55.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528398+0000", 2024-04-26T03:38:55.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:55.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528398+0000", 2024-04-26T03:38:55.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:55.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528399+0000", 2024-04-26T03:38:55.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5499999999999996e-07 2024-04-26T03:38:55.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:55.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528404+0000", 2024-04-26T03:38:55.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9559999999999996e-06 2024-04-26T03:38:55.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:55.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528416+0000", 2024-04-26T03:38:55.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1882000000000001e-05 2024-04-26T03:38:55.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:55.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528424+0000", 2024-04-26T03:38:55.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8099999999999998e-06 2024-04-26T03:38:55.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.535507+0000", 2024-04-26T03:38:55.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0070833789999999999 2024-04-26T03:38:55.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:38:55.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.535597+0000", 2024-04-26T03:38:55.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9215000000000006e-05 2024-04-26T03:38:55.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:55.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.536064+0000", 2024-04-26T03:38:55.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00046694199999999998 2024-04-26T03:38:55.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:55.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.536868+0000", 2024-04-26T03:38:55.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000804804 2024-04-26T03:38:55.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:38:55.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.538026+0000", 2024-04-26T03:38:55.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011578370000000001 2024-04-26T03:38:55.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:55.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.538108+0000", 2024-04-26T03:38:55.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1537000000000002e-05 2024-04-26T03:38:55.853 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.853 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:55.853 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.853 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:78 3.9s0 3:9da60d2b:::smithi09235267-26:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:38:55.853 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.528454+0000", 2024-04-26T03:38:55.853 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.2642715180000001, 2024-04-26T03:38:55.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011500992, 2024-04-26T03:38:55.854 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:55.854 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:55.854 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-04-26T03:38:55.854 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:38:55.854 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:38:55.854 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:38:55.854 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 78 2024-04-26T03:38:55.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.854 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:55.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:55.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528454+0000", 2024-04-26T03:38:55.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:55.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528454+0000", 2024-04-26T03:38:55.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:55.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528455+0000", 2024-04-26T03:38:55.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9699999999999995e-07 2024-04-26T03:38:55.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:55.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528459+0000", 2024-04-26T03:38:55.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1620000000000001e-06 2024-04-26T03:38:55.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:55.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528468+0000", 2024-04-26T03:38:55.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7010000000000002e-06 2024-04-26T03:38:55.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:55.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528474+0000", 2024-04-26T03:38:55.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7039999999999997e-06 2024-04-26T03:38:55.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.536776+0000", 2024-04-26T03:38:55.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0083020709999999994 2024-04-26T03:38:55.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-04-26T03:38:55.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.536858+0000", 2024-04-26T03:38:55.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1130000000000004e-05 2024-04-26T03:38:55.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.538545+0000", 2024-04-26T03:38:55.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016869610000000001 2024-04-26T03:38:55.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:38:55.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.538629+0000", 2024-04-26T03:38:55.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4257000000000003e-05 2024-04-26T03:38:55.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:55.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.539955+0000", 2024-04-26T03:38:55.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013263089999999999 2024-04-26T03:38:55.858 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.858 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:55.858 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.858 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:87 3.es0 3:799cf17d:::smithi09235267-29:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:38:55.859 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.540988+0000", 2024-04-26T03:38:55.859 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.2517371910000001, 2024-04-26T03:38:55.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010406014, 2024-04-26T03:38:55.859 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:55.859 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:55.859 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-04-26T03:38:55.859 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:38:55.859 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:38:55.859 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:38:55.859 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 87 2024-04-26T03:38:55.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.859 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:55.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:55.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.540988+0000", 2024-04-26T03:38:55.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:55.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.540988+0000", 2024-04-26T03:38:55.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:55.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.540989+0000", 2024-04-26T03:38:55.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4400000000000001e-07 2024-04-26T03:38:55.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:55.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.540991+0000", 2024-04-26T03:38:55.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5299999999999999e-06 2024-04-26T03:38:55.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:55.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.540998+0000", 2024-04-26T03:38:55.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0190000000000001e-06 2024-04-26T03:38:55.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:55.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.541005+0000", 2024-04-26T03:38:55.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2149999999999998e-06 2024-04-26T03:38:55.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.546779+0000", 2024-04-26T03:38:55.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0057747659999999998 2024-04-26T03:38:55.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-04-26T03:38:55.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.546880+0000", 2024-04-26T03:38:55.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100657 2024-04-26T03:38:55.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.550192+0000", 2024-04-26T03:38:55.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033114310000000001 2024-04-26T03:38:55.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:38:55.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.550270+0000", 2024-04-26T03:38:55.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.805e-05 2024-04-26T03:38:55.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:55.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.551394+0000", 2024-04-26T03:38:55.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011249020000000001 2024-04-26T03:38:55.863 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.863 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:55.863 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.863 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:99 3.9s0 3:91c00773:::smithi09235267-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:38:55.864 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.555207+0000", 2024-04-26T03:38:55.864 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.2375189759999996, 2024-04-26T03:38:55.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010397744, 2024-04-26T03:38:55.864 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:55.864 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:55.864 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-04-26T03:38:55.864 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:38:55.864 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:38:55.864 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:38:55.864 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 99 2024-04-26T03:38:55.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.864 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:55.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:55.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.555207+0000", 2024-04-26T03:38:55.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:55.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.555207+0000", 2024-04-26T03:38:55.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:55.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.555207+0000", 2024-04-26T03:38:55.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0800000000000005e-07 2024-04-26T03:38:55.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:55.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.555209+0000", 2024-04-26T03:38:55.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3020000000000002e-06 2024-04-26T03:38:55.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:55.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.555216+0000", 2024-04-26T03:38:55.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2349999999999996e-06 2024-04-26T03:38:55.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:55.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.555221+0000", 2024-04-26T03:38:55.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1399999999999999e-06 2024-04-26T03:38:55.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.560263+0000", 2024-04-26T03:38:55.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0050417839999999997 2024-04-26T03:38:55.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-04-26T03:38:55.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.560356+0000", 2024-04-26T03:38:55.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2987999999999998e-05 2024-04-26T03:38:55.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.563913+0000", 2024-04-26T03:38:55.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035570010000000002 2024-04-26T03:38:55.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:38:55.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.563988+0000", 2024-04-26T03:38:55.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5463000000000006e-05 2024-04-26T03:38:55.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:55.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.565604+0000", 2024-04-26T03:38:55.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001616323 2024-04-26T03:38:55.868 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.868 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:55.868 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.868 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:126 3.es0 3:718e77da:::smithi09235267-42:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:38:55.869 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.594835+0000", 2024-04-26T03:38:55.869 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.1978908930000003, 2024-04-26T03:38:55.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0091609139999999992, 2024-04-26T03:38:55.869 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:55.869 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:55.869 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-04-26T03:38:55.869 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:38:55.869 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:38:55.869 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:38:55.869 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126 2024-04-26T03:38:55.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.869 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:55.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:55.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.594835+0000", 2024-04-26T03:38:55.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:55.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.594835+0000", 2024-04-26T03:38:55.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:55.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.594835+0000", 2024-04-26T03:38:55.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9500000000000002e-07 2024-04-26T03:38:55.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:55.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.594838+0000", 2024-04-26T03:38:55.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5150000000000001e-06 2024-04-26T03:38:55.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:55.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.594846+0000", 2024-04-26T03:38:55.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5739999999999998e-06 2024-04-26T03:38:55.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:55.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.594850+0000", 2024-04-26T03:38:55.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4240000000000003e-06 2024-04-26T03:38:55.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.600839+0000", 2024-04-26T03:38:55.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.005989271 2024-04-26T03:38:55.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-04-26T03:38:55.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.600932+0000", 2024-04-26T03:38:55.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2688000000000004e-05 2024-04-26T03:38:55.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.602985+0000", 2024-04-26T03:38:55.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020527750000000002 2024-04-26T03:38:55.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:38:55.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.603072+0000", 2024-04-26T03:38:55.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7082000000000004e-05 2024-04-26T03:38:55.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:55.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.603996+0000", 2024-04-26T03:38:55.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00092389000000000002 2024-04-26T03:38:55.873 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.873 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:55.873 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.874 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:163 3.es0 3:799cf17d:::smithi09235267-29:head [read 0~475136,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:38:55.874 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.637337+0000", 2024-04-26T03:38:55.874 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.1553887070000002, 2024-04-26T03:38:55.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0090862219999999997, 2024-04-26T03:38:55.874 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:55.874 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:55.874 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-04-26T03:38:55.874 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:38:55.874 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:38:55.874 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:38:55.874 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 163 2024-04-26T03:38:55.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.874 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:55.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:55.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.637337+0000", 2024-04-26T03:38:55.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:55.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.637337+0000", 2024-04-26T03:38:55.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:55.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.637337+0000", 2024-04-26T03:38:55.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8799999999999998e-07 2024-04-26T03:38:55.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:55.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.637340+0000", 2024-04-26T03:38:55.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0400000000000001e-06 2024-04-26T03:38:55.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:55.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.637347+0000", 2024-04-26T03:38:55.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3929999999999996e-06 2024-04-26T03:38:55.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:55.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.637351+0000", 2024-04-26T03:38:55.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3619999999999999e-06 2024-04-26T03:38:55.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.639466+0000", 2024-04-26T03:38:55.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002114697 2024-04-26T03:38:55.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:38:55.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.639534+0000", 2024-04-26T03:38:55.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8042999999999999e-05 2024-04-26T03:38:55.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:55.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.646423+0000", 2024-04-26T03:38:55.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0068892989999999998 2024-04-26T03:38:55.878 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.878 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:55.878 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.878 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:164 3.es0 3:799cf17d:::smithi09235267-29:head [read 0~475136,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:38:55.878 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.637357+0000", 2024-04-26T03:38:55.878 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.1553685539999998, 2024-04-26T03:38:55.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0094075860000000008, 2024-04-26T03:38:55.878 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:55.878 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:55.878 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-04-26T03:38:55.878 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:38:55.878 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:38:55.879 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:38:55.879 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 164 2024-04-26T03:38:55.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.879 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:55.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:55.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.637357+0000", 2024-04-26T03:38:55.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:55.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.637357+0000", 2024-04-26T03:38:55.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:55.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.637357+0000", 2024-04-26T03:38:55.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.15e-07 2024-04-26T03:38:55.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:55.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.637360+0000", 2024-04-26T03:38:55.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2989999999999999e-06 2024-04-26T03:38:55.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:55.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.637365+0000", 2024-04-26T03:38:55.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6049999999999997e-06 2024-04-26T03:38:55.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:55.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.637370+0000", 2024-04-26T03:38:55.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4700000000000004e-06 2024-04-26T03:38:55.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.640173+0000", 2024-04-26T03:38:55.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0028027989999999999 2024-04-26T03:38:55.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:38:55.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.640217+0000", 2024-04-26T03:38:55.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.439e-05 2024-04-26T03:38:55.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:55.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.646765+0000", 2024-04-26T03:38:55.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0065477080000000002 2024-04-26T03:38:55.882 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.882 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:55.882 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.882 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:177 3.es0 3:718e77da:::smithi09235267-42:head [append 0~524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:38:55.882 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.730329+0000", 2024-04-26T03:38:55.882 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0623963999999999, 2024-04-26T03:38:55.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0094148259999999994, 2024-04-26T03:38:55.883 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:55.883 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:55.883 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:38:55.883 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:38:55.883 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:38:55.883 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:38:55.883 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 177 2024-04-26T03:38:55.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.883 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:55.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:55.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.730329+0000", 2024-04-26T03:38:55.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:55.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.730329+0000", 2024-04-26T03:38:55.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:55.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.730331+0000", 2024-04-26T03:38:55.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6169999999999999e-06 2024-04-26T03:38:55.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:55.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.730806+0000", 2024-04-26T03:38:55.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00047506000000000002 2024-04-26T03:38:55.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:55.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.730824+0000", 2024-04-26T03:38:55.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8099000000000001e-05 2024-04-26T03:38:55.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:55.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.730835+0000", 2024-04-26T03:38:55.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0933e-05 2024-04-26T03:38:55.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.730890+0000", 2024-04-26T03:38:55.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4642999999999998e-05 2024-04-26T03:38:55.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:38:55.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.730950+0000", 2024-04-26T03:38:55.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0253000000000001e-05 2024-04-26T03:38:55.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:55.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.731522+0000", 2024-04-26T03:38:55.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00057178899999999996 2024-04-26T03:38:55.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:55.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.739208+0000", 2024-04-26T03:38:55.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.007686015 2024-04-26T03:38:55.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:38:55.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.739682+0000", 2024-04-26T03:38:55.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00047473600000000002 2024-04-26T03:38:55.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:55.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.739744+0000", 2024-04-26T03:38:55.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1680999999999994e-05 2024-04-26T03:38:55.887 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.887 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:55.887 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.888 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:180 3.es0 3:718e77da:::smithi09235267-42:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:38:55.888 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.733080+0000", 2024-04-26T03:38:55.888 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0596460690000002, 2024-04-26T03:38:55.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0096743769999999996, 2024-04-26T03:38:55.888 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:55.888 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:55.888 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-04-26T03:38:55.888 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:38:55.888 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:38:55.888 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:38:55.888 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 180 2024-04-26T03:38:55.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.888 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:55.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:55.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.733080+0000", 2024-04-26T03:38:55.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:55.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.733080+0000", 2024-04-26T03:38:55.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:55.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.733080+0000", 2024-04-26T03:38:55.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5200000000000002e-07 2024-04-26T03:38:55.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:55.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.733082+0000", 2024-04-26T03:38:55.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1100000000000001e-06 2024-04-26T03:38:55.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:55.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.733089+0000", 2024-04-26T03:38:55.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5869999999999998e-06 2024-04-26T03:38:55.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:55.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.733094+0000", 2024-04-26T03:38:55.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8169999999999998e-06 2024-04-26T03:38:55.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.739978+0000", 2024-04-26T03:38:55.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.006884945 2024-04-26T03:38:55.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-04-26T03:38:55.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.740068+0000", 2024-04-26T03:38:55.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9840999999999996e-05 2024-04-26T03:38:55.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.741054+0000", 2024-04-26T03:38:55.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00098595499999999995 2024-04-26T03:38:55.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:38:55.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.741130+0000", 2024-04-26T03:38:55.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5283000000000004e-05 2024-04-26T03:38:55.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:55.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.742754+0000", 2024-04-26T03:38:55.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016243869999999999 2024-04-26T03:38:55.892 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.892 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:55.892 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.893 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:207 3.es0 3:718e77da:::smithi09235267-42:head [append 0~638976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:38:55.893 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.806150+0000", 2024-04-26T03:38:55.893 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.9865751930000002, 2024-04-26T03:38:55.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010521529, 2024-04-26T03:38:55.893 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:55.893 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:55.893 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:38:55.893 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:38:55.893 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:38:55.893 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:38:55.893 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 207 2024-04-26T03:38:55.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.893 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:55.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:55.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.806150+0000", 2024-04-26T03:38:55.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:55.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.806150+0000", 2024-04-26T03:38:55.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:55.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.806152+0000", 2024-04-26T03:38:55.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.274e-06 2024-04-26T03:38:55.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:55.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.806688+0000", 2024-04-26T03:38:55.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00053636700000000005 2024-04-26T03:38:55.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:55.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.806700+0000", 2024-04-26T03:38:55.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1969e-05 2024-04-26T03:38:55.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:55.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.806705+0000", 2024-04-26T03:38:55.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2780000000000001e-06 2024-04-26T03:38:55.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.806771+0000", 2024-04-26T03:38:55.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5748999999999997e-05 2024-04-26T03:38:55.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:38:55.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.806837+0000", 2024-04-26T03:38:55.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5445999999999996e-05 2024-04-26T03:38:55.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:38:55.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.807429+0000", 2024-04-26T03:38:55.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00059259600000000003 2024-04-26T03:38:55.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:38:55.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.814358+0000", 2024-04-26T03:38:55.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0069287949999999997 2024-04-26T03:38:55.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:38:55.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.816619+0000", 2024-04-26T03:38:55.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022611710000000002 2024-04-26T03:38:55.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:55.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.816672+0000", 2024-04-26T03:38:55.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2883999999999997e-05 2024-04-26T03:38:55.898 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.898 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:55.898 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.898 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:210 3.es0 3:718e77da:::smithi09235267-42:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:38:55.898 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.808586+0000", 2024-04-26T03:38:55.898 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.9841398469999998, 2024-04-26T03:38:55.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010859396, 2024-04-26T03:38:55.898 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:38:55.898 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:38:55.898 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-04-26T03:38:55.898 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:38:55.899 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:38:55.899 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:38:55.899 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 210 2024-04-26T03:38:55.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.899 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:38:55.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:38:55.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.808586+0000", 2024-04-26T03:38:55.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:38:55.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.808586+0000", 2024-04-26T03:38:55.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:38:55.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:38:55.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.808586+0000", 2024-04-26T03:38:55.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7000000000000001e-07 2024-04-26T03:38:55.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:38:55.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.808587+0000", 2024-04-26T03:38:55.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.251e-06 2024-04-26T03:38:55.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:38:55.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.808591+0000", 2024-04-26T03:38:55.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6849999999999999e-06 2024-04-26T03:38:55.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:38:55.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.808595+0000", 2024-04-26T03:38:55.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6430000000000001e-06 2024-04-26T03:38:55.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.814250+0000", 2024-04-26T03:38:55.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.005655116 2024-04-26T03:38:55.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-04-26T03:38:55.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.814313+0000", 2024-04-26T03:38:55.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3075000000000002e-05 2024-04-26T03:38:55.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:38:55.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.817583+0000", 2024-04-26T03:38:55.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0032703300000000001 2024-04-26T03:38:55.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:38:55.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.817668+0000", 2024-04-26T03:38:55.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5222000000000005e-05 2024-04-26T03:38:55.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:38:55.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:38:55.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:38:55.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.819445+0000", 2024-04-26T03:38:55.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0017768040000000001 2024-04-26T03:38:55.903 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.903 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:55.903 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.903 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:38:55.903 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:38:55.903 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-26T03:38:55.904 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-26T03:38:55.905 INFO:tasks.ceph.osd.14.smithi066.stderr:2024-04-26T03:38:55.903+0000 7f5395f08640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 31765) UID: 0 2024-04-26T03:38:56.006 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-04-26T03:38:56.004+0000 7fe9223c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31954) UID: 0 2024-04-26T03:38:56.024 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:38:56.024 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-04-26T03:38:56.024 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 0 2024-04-26T03:38:56.024 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:38:56.034 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-26T03:38:56.106 INFO:tasks.ceph.osd.10.smithi066.stderr:2024-04-26T03:38:56.104+0000 7f130fcc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 31763) UID: 0 2024-04-26T03:38:56.142 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:38:56.142 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-04-26T03:38:56.142 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-04-26T03:38:56.142 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-04-26T03:38:56.142 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:38:56.150 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-26T03:38:56.207 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-04-26T03:38:56.205+0000 7f745118d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 31821) UID: 0 2024-04-26T03:38:56.300 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:38:56.300 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-04-26T03:38:56.300 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-04-26T03:38:56.300 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-04-26T03:38:56.300 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.300 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.3s5 34/32 ECSubWrite(tid=0, reqid=client.4556.0:16, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:38:56.300 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.487802+0000", 2024-04-26T03:38:56.300 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 4.8096571179999996, 2024-04-26T03:38:56.300 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0063253500000000004, 2024-04-26T03:38:56.300 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:38:56.300 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:38:56.300 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:38:56.301 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:38:56.301 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.301 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:38:56.301 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.487802+0000", 2024-04-26T03:38:56.301 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:56.301 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.301 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.301 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:38:56.301 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.487802+0000", 2024-04-26T03:38:56.301 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:56.301 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.301 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.302 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:38:56.302 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.487802+0000", 2024-04-26T03:38:56.302 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.2e-07 2024-04-26T03:38:56.302 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.302 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.302 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:38:56.302 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.487838+0000", 2024-04-26T03:38:56.302 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.5306999999999999e-05 2024-04-26T03:38:56.302 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.302 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.302 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:38:56.302 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.487853+0000", 2024-04-26T03:38:56.302 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5770999999999999e-05 2024-04-26T03:38:56.302 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.303 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.303 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:38:56.303 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.487863+0000", 2024-04-26T03:38:56.303 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.5580000000000006e-06 2024-04-26T03:38:56.303 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.303 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.303 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:38:56.303 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.487965+0000", 2024-04-26T03:38:56.303 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000102054 2024-04-26T03:38:56.303 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.303 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.303 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:38:56.303 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.487975+0000", 2024-04-26T03:38:56.304 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.9520000000000006e-06 2024-04-26T03:38:56.304 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.304 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.304 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:38:56.304 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.494090+0000", 2024-04-26T03:38:56.304 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0061146120000000002 2024-04-26T03:38:56.304 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.304 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.304 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:38:56.304 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.494127+0000", 2024-04-26T03:38:56.304 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7676000000000003e-05 2024-04-26T03:38:56.304 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:56.304 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:38:56.304 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:56.305 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.305 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.305 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.fs4 34/32 ECSubWrite(tid=2, reqid=client.4556.0:25, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:38:56.305 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.490403+0000", 2024-04-26T03:38:56.305 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 4.8070555639999997, 2024-04-26T03:38:56.305 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0046655009999999998, 2024-04-26T03:38:56.305 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:38:56.305 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:38:56.305 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:38:56.305 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:38:56.305 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.305 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:38:56.305 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.490403+0000", 2024-04-26T03:38:56.305 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:56.306 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.306 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.306 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:38:56.306 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.490403+0000", 2024-04-26T03:38:56.306 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:56.306 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.306 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.306 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:38:56.306 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.490404+0000", 2024-04-26T03:38:56.306 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.0699999999999997e-07 2024-04-26T03:38:56.306 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.307 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.307 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:38:56.307 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.490462+0000", 2024-04-26T03:38:56.307 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.8399000000000001e-05 2024-04-26T03:38:56.307 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.307 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.307 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:38:56.307 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.490476+0000", 2024-04-26T03:38:56.307 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3283e-05 2024-04-26T03:38:56.307 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.307 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.307 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:38:56.307 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.490485+0000", 2024-04-26T03:38:56.307 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.2869999999999993e-06 2024-04-26T03:38:56.308 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.308 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.308 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:38:56.308 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.490556+0000", 2024-04-26T03:38:56.308 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.1235000000000005e-05 2024-04-26T03:38:56.308 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.308 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.308 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:38:56.308 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.490565+0000", 2024-04-26T03:38:56.308 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.9099999999999994e-06 2024-04-26T03:38:56.308 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.308 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.308 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:38:56.308 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.494986+0000", 2024-04-26T03:38:56.308 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0044212720000000004 2024-04-26T03:38:56.309 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.309 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.309 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:38:56.309 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.495069+0000", 2024-04-26T03:38:56.309 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2508000000000004e-05 2024-04-26T03:38:56.309 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:56.309 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:38:56.309 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:56.309 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.309 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.309 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.0s3 35/32 ECSubWrite(tid=2, reqid=client.4556.0:40, at_version=35'3, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:38:56.309 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.497613+0000", 2024-04-26T03:38:56.309 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 4.7998457200000004, 2024-04-26T03:38:56.309 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0057777269999999999, 2024-04-26T03:38:56.309 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:38:56.310 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:38:56.310 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:38:56.310 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:38:56.310 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.310 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:38:56.310 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.497613+0000", 2024-04-26T03:38:56.310 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:56.310 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.310 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.310 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:38:56.310 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.497613+0000", 2024-04-26T03:38:56.310 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:56.310 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.310 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.311 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:38:56.311 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.497614+0000", 2024-04-26T03:38:56.311 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.1900000000000002e-07 2024-04-26T03:38:56.311 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.311 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.311 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:38:56.311 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.497715+0000", 2024-04-26T03:38:56.311 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000101369 2024-04-26T03:38:56.311 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.311 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.311 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:38:56.311 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.497728+0000", 2024-04-26T03:38:56.311 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2741999999999999e-05 2024-04-26T03:38:56.311 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.312 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.312 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:38:56.312 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.497734+0000", 2024-04-26T03:38:56.312 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7010000000000001e-06 2024-04-26T03:38:56.312 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.312 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.312 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:38:56.312 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.497790+0000", 2024-04-26T03:38:56.312 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.6215999999999998e-05 2024-04-26T03:38:56.312 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.312 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.312 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:38:56.312 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.497796+0000", 2024-04-26T03:38:56.312 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.6319999999999996e-06 2024-04-26T03:38:56.313 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.313 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.313 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:38:56.313 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.503349+0000", 2024-04-26T03:38:56.313 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0055537470000000004 2024-04-26T03:38:56.313 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.313 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.313 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:38:56.313 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.503391+0000", 2024-04-26T03:38:56.313 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.1600999999999997e-05 2024-04-26T03:38:56.313 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:56.313 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:38:56.313 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:56.313 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.314 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.314 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.9s1 35/32 ECSubWrite(tid=0, reqid=client.4556.0:46, at_version=35'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:38:56.314 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.505318+0000", 2024-04-26T03:38:56.314 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 4.792141429, 2024-04-26T03:38:56.314 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0045086299999999996, 2024-04-26T03:38:56.314 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:38:56.314 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:38:56.314 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:38:56.314 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:38:56.314 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.314 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:38:56.314 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.505318+0000", 2024-04-26T03:38:56.314 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:56.315 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.315 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.315 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:38:56.315 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.505318+0000", 2024-04-26T03:38:56.315 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:56.315 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.315 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.315 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:38:56.315 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.505318+0000", 2024-04-26T03:38:56.315 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.3799999999999998e-07 2024-04-26T03:38:56.315 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.315 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.315 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:38:56.315 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.505396+0000", 2024-04-26T03:38:56.316 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8432999999999999e-05 2024-04-26T03:38:56.316 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.316 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.316 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:38:56.316 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.505421+0000", 2024-04-26T03:38:56.316 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4539e-05 2024-04-26T03:38:56.316 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.316 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.316 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:38:56.316 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.505431+0000", 2024-04-26T03:38:56.316 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0168e-05 2024-04-26T03:38:56.316 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.316 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.316 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:38:56.317 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.505499+0000", 2024-04-26T03:38:56.317 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.8206000000000004e-05 2024-04-26T03:38:56.317 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.317 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.317 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:38:56.317 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.505508+0000", 2024-04-26T03:38:56.317 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.3659999999999992e-06 2024-04-26T03:38:56.317 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.317 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.317 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:38:56.317 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.509795+0000", 2024-04-26T03:38:56.317 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0042870069999999998 2024-04-26T03:38:56.317 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.317 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.318 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:38:56.318 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.509826+0000", 2024-04-26T03:38:56.318 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.1572999999999997e-05 2024-04-26T03:38:56.318 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:56.318 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:38:56.318 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:56.318 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.318 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.318 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.3s5 35/32 ECSubWrite(tid=4, reqid=client.4556.0:88, at_version=35'3, trim_to=0'0, roll_forward_to=34'2))", 2024-04-26T03:38:56.318 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.545352+0000", 2024-04-26T03:38:56.318 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 4.7521070559999998, 2024-04-26T03:38:56.318 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0049536509999999999, 2024-04-26T03:38:56.319 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:38:56.319 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:38:56.320 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:38:56.320 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:38:56.320 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.320 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:38:56.320 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.545352+0000", 2024-04-26T03:38:56.320 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:56.320 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.320 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.320 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:38:56.320 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.545352+0000", 2024-04-26T03:38:56.320 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:56.320 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.320 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.320 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:38:56.321 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.545353+0000", 2024-04-26T03:38:56.321 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.8599999999999997e-07 2024-04-26T03:38:56.321 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.321 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.321 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:38:56.321 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.545460+0000", 2024-04-26T03:38:56.321 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00010758400000000001 2024-04-26T03:38:56.321 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.321 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.321 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:38:56.321 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.545472+0000", 2024-04-26T03:38:56.321 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1168999999999999e-05 2024-04-26T03:38:56.321 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.321 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.322 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:38:56.322 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.545479+0000", 2024-04-26T03:38:56.322 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5279999999999998e-06 2024-04-26T03:38:56.322 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.322 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.322 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:38:56.322 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.545541+0000", 2024-04-26T03:38:56.322 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.1487000000000003e-05 2024-04-26T03:38:56.322 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.322 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.322 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:38:56.322 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.545549+0000", 2024-04-26T03:38:56.322 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7220000000000005e-06 2024-04-26T03:38:56.323 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.323 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.323 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:38:56.323 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.550269+0000", 2024-04-26T03:38:56.323 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0047195650000000002 2024-04-26T03:38:56.323 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.323 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.323 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:38:56.323 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.550306+0000", 2024-04-26T03:38:56.323 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.6709999999999999e-05 2024-04-26T03:38:56.323 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:56.323 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:38:56.323 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:56.323 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.323 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.324 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.3s5 35/32 ECSubWrite(tid=5, reqid=client.4556.0:89, at_version=35'4, trim_to=0'0, roll_forward_to=34'2))", 2024-04-26T03:38:56.324 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.550876+0000", 2024-04-26T03:38:56.324 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 4.7465829140000002, 2024-04-26T03:38:56.324 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0058882370000000002, 2024-04-26T03:38:56.324 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:38:56.324 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:38:56.324 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:38:56.324 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:38:56.324 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.324 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:38:56.324 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.550876+0000", 2024-04-26T03:38:56.324 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:56.324 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.324 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.325 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:38:56.325 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.550876+0000", 2024-04-26T03:38:56.325 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:56.325 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.325 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.325 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:38:56.325 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.550877+0000", 2024-04-26T03:38:56.325 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.09e-07 2024-04-26T03:38:56.325 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.325 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.325 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:38:56.325 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.550895+0000", 2024-04-26T03:38:56.325 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7581999999999999e-05 2024-04-26T03:38:56.325 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.325 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.326 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:38:56.326 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.550917+0000", 2024-04-26T03:38:56.326 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2612999999999998e-05 2024-04-26T03:38:56.326 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.326 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.326 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:38:56.326 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.550925+0000", 2024-04-26T03:38:56.326 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.0669999999999992e-06 2024-04-26T03:38:56.326 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.326 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.326 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:38:56.326 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.550994+0000", 2024-04-26T03:38:56.326 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.8757999999999995e-05 2024-04-26T03:38:56.326 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.327 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.327 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:38:56.327 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.551002+0000", 2024-04-26T03:38:56.327 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9310000000000008e-06 2024-04-26T03:38:56.327 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.327 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.327 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:38:56.327 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.556730+0000", 2024-04-26T03:38:56.327 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0057283070000000002 2024-04-26T03:38:56.327 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.327 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.327 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:38:56.327 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.556764+0000", 2024-04-26T03:38:56.327 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.4069999999999997e-05 2024-04-26T03:38:56.328 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:56.328 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:38:56.328 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:56.328 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.328 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.328 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.3s5 35/32 ECSubWrite(tid=6, reqid=client.4556.0:94, at_version=35'5, trim_to=0'0, roll_forward_to=34'2))", 2024-04-26T03:38:56.328 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.552428+0000", 2024-04-26T03:38:56.328 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 4.7450310949999999, 2024-04-26T03:38:56.328 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00476867, 2024-04-26T03:38:56.328 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:38:56.328 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:38:56.328 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:38:56.328 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:38:56.328 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.328 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:38:56.329 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.552428+0000", 2024-04-26T03:38:56.329 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:56.329 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.329 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.329 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:38:56.329 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.552428+0000", 2024-04-26T03:38:56.329 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:56.329 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.329 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.329 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:38:56.329 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.552429+0000", 2024-04-26T03:38:56.329 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.062e-06 2024-04-26T03:38:56.329 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.330 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.330 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:38:56.330 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.552531+0000", 2024-04-26T03:38:56.330 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000102051 2024-04-26T03:38:56.330 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.330 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.330 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:38:56.330 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.552550+0000", 2024-04-26T03:38:56.330 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.906e-05 2024-04-26T03:38:56.330 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.330 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.330 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:38:56.330 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.552559+0000", 2024-04-26T03:38:56.330 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.0080000000000008e-06 2024-04-26T03:38:56.331 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.331 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.331 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:38:56.331 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.552620+0000", 2024-04-26T03:38:56.331 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.1309999999999994e-05 2024-04-26T03:38:56.331 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.331 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.331 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:38:56.331 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.552631+0000", 2024-04-26T03:38:56.331 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0621999999999999e-05 2024-04-26T03:38:56.331 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.331 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.331 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:38:56.331 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.557166+0000", 2024-04-26T03:38:56.332 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0045348130000000004 2024-04-26T03:38:56.332 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.332 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.332 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:38:56.332 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.557197+0000", 2024-04-26T03:38:56.332 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.0744000000000002e-05 2024-04-26T03:38:56.332 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:56.332 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:38:56.332 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:56.332 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.332 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.332 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.9s1 35/32 ECSubWrite(tid=20, reqid=client.4556.0:97, at_version=35'5, trim_to=0'0, roll_forward_to=35'4))", 2024-04-26T03:38:56.332 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.555957+0000", 2024-04-26T03:38:56.332 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 4.7415024399999997, 2024-04-26T03:38:56.333 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.006868023, 2024-04-26T03:38:56.333 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:38:56.333 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:38:56.333 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:38:56.333 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:38:56.333 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.333 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:38:56.333 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.555957+0000", 2024-04-26T03:38:56.333 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:56.333 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.333 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.333 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:38:56.333 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.555957+0000", 2024-04-26T03:38:56.333 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:56.334 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.334 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.334 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:38:56.334 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.555964+0000", 2024-04-26T03:38:56.334 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.7360000000000001e-06 2024-04-26T03:38:56.334 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.334 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.334 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:38:56.334 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.556167+0000", 2024-04-26T03:38:56.334 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000202986 2024-04-26T03:38:56.334 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.334 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.334 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:38:56.334 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.556183+0000", 2024-04-26T03:38:56.335 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6175999999999999e-05 2024-04-26T03:38:56.335 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.335 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.335 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:38:56.335 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.556191+0000", 2024-04-26T03:38:56.335 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9899999999999997e-06 2024-04-26T03:38:56.335 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.335 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.335 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:38:56.335 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.556259+0000", 2024-04-26T03:38:56.335 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7986999999999998e-05 2024-04-26T03:38:56.335 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.335 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.335 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:38:56.336 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.556280+0000", 2024-04-26T03:38:56.336 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0392999999999999e-05 2024-04-26T03:38:56.336 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.336 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.336 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:38:56.336 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.562792+0000", 2024-04-26T03:38:56.336 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0065119560000000002 2024-04-26T03:38:56.336 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.336 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.336 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:38:56.336 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.562825+0000", 2024-04-26T03:38:56.336 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.2799000000000002e-05 2024-04-26T03:38:56.336 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:56.336 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:38:56.337 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:56.337 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.337 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.337 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.4s3 35/32 ECSubWrite(tid=11, reqid=client.4556.0:100, at_version=35'7, trim_to=0'0, roll_forward_to=35'6))", 2024-04-26T03:38:56.337 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.560517+0000", 2024-04-26T03:38:56.337 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 4.7369423590000004, 2024-04-26T03:38:56.337 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0051878360000000004, 2024-04-26T03:38:56.337 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:38:56.337 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:38:56.337 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:38:56.337 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:38:56.337 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.337 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:38:56.337 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.560517+0000", 2024-04-26T03:38:56.337 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:56.338 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.338 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.338 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:38:56.338 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.560517+0000", 2024-04-26T03:38:56.338 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:56.338 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.338 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.338 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:38:56.338 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.560517+0000", 2024-04-26T03:38:56.338 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7899999999999998e-07 2024-04-26T03:38:56.338 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.338 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.338 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:38:56.339 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.560631+0000", 2024-04-26T03:38:56.339 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00011341 2024-04-26T03:38:56.339 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.339 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.339 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:38:56.339 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.560647+0000", 2024-04-26T03:38:56.339 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6480000000000001e-05 2024-04-26T03:38:56.339 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.339 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.339 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:38:56.339 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.560655+0000", 2024-04-26T03:38:56.339 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.1540000000000002e-06 2024-04-26T03:38:56.339 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.339 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.340 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:38:56.340 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.560733+0000", 2024-04-26T03:38:56.340 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.7423999999999996e-05 2024-04-26T03:38:56.340 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.340 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.340 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:38:56.340 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.560742+0000", 2024-04-26T03:38:56.340 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.8449999999999992e-06 2024-04-26T03:38:56.340 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.340 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.340 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:38:56.340 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.565670+0000", 2024-04-26T03:38:56.340 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0049284940000000003 2024-04-26T03:38:56.340 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.341 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.341 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:38:56.341 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.565704+0000", 2024-04-26T03:38:56.341 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.4350000000000001e-05 2024-04-26T03:38:56.341 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:56.341 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:38:56.341 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:56.341 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.341 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.341 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.4s3 35/32 ECSubWrite(tid=12, reqid=client.4556.0:101, at_version=35'8, trim_to=0'0, roll_forward_to=35'6))", 2024-04-26T03:38:56.341 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.564690+0000", 2024-04-26T03:38:56.341 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 4.7327685510000004, 2024-04-26T03:38:56.341 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0069190800000000002, 2024-04-26T03:38:56.341 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:38:56.341 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:38:56.342 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:38:56.342 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:38:56.342 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.342 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:38:56.342 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.564690+0000", 2024-04-26T03:38:56.342 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:56.342 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.342 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.342 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:38:56.342 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.564690+0000", 2024-04-26T03:38:56.342 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:56.342 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.342 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.342 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:38:56.343 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.564694+0000", 2024-04-26T03:38:56.343 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.6449999999999998e-06 2024-04-26T03:38:56.343 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.343 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.343 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:38:56.343 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.564710+0000", 2024-04-26T03:38:56.343 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5979999999999999e-05 2024-04-26T03:38:56.343 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.343 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.343 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:38:56.343 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.564725+0000", 2024-04-26T03:38:56.343 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4876e-05 2024-04-26T03:38:56.343 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.343 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.344 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:38:56.344 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.564734+0000", 2024-04-26T03:38:56.344 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7220000000000005e-06 2024-04-26T03:38:56.344 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.344 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.344 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:38:56.344 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.565327+0000", 2024-04-26T03:38:56.344 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000592849 2024-04-26T03:38:56.344 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.344 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.344 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:38:56.344 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.565334+0000", 2024-04-26T03:38:56.344 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9270000000000005e-06 2024-04-26T03:38:56.344 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.345 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.345 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:38:56.345 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.571566+0000", 2024-04-26T03:38:56.345 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0062319070000000001 2024-04-26T03:38:56.345 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.345 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.345 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:38:56.345 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.571610+0000", 2024-04-26T03:38:56.345 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.3173999999999997e-05 2024-04-26T03:38:56.345 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:56.345 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:38:56.345 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:56.345 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.345 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.345 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.4s3 35/32 ECSubWrite(tid=13, reqid=client.4556.0:106, at_version=35'9, trim_to=0'0, roll_forward_to=35'6))", 2024-04-26T03:38:56.346 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.567072+0000", 2024-04-26T03:38:56.346 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 4.7303873950000002, 2024-04-26T03:38:56.346 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0074599230000000002, 2024-04-26T03:38:56.346 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:38:56.346 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:38:56.346 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:38:56.346 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:38:56.346 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.346 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:38:56.346 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.567072+0000", 2024-04-26T03:38:56.346 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:56.346 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.346 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.346 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:38:56.347 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.567072+0000", 2024-04-26T03:38:56.347 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:56.347 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.347 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.347 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:38:56.347 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.567072+0000", 2024-04-26T03:38:56.347 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.8100000000000003e-07 2024-04-26T03:38:56.347 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.347 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.347 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:38:56.347 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.567302+0000", 2024-04-26T03:38:56.347 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000229457 2024-04-26T03:38:56.348 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.348 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.348 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:38:56.348 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.567315+0000", 2024-04-26T03:38:56.348 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2845e-05 2024-04-26T03:38:56.348 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.348 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.348 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:38:56.348 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.567322+0000", 2024-04-26T03:38:56.348 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8979999999999996e-06 2024-04-26T03:38:56.348 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.348 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.348 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:38:56.348 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.567396+0000", 2024-04-26T03:38:56.348 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3398000000000004e-05 2024-04-26T03:38:56.349 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.349 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.349 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:38:56.349 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.567413+0000", 2024-04-26T03:38:56.349 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7e-05 2024-04-26T03:38:56.349 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.349 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.349 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:38:56.349 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.574497+0000", 2024-04-26T03:38:56.349 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0070843640000000001 2024-04-26T03:38:56.349 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.349 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.349 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:38:56.349 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.574532+0000", 2024-04-26T03:38:56.350 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.4379999999999999e-05 2024-04-26T03:38:56.350 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:56.350 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:38:56.350 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:56.350 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.350 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.350 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.3s5 35/32 ECSubWrite(tid=11, reqid=client.4556.0:109, at_version=35'7, trim_to=0'0, roll_forward_to=35'6))", 2024-04-26T03:38:56.350 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.570835+0000", 2024-04-26T03:38:56.350 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 4.726623945, 2024-04-26T03:38:56.350 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0064321869999999998, 2024-04-26T03:38:56.350 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:38:56.350 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:38:56.350 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:38:56.350 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:38:56.351 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.351 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:38:56.351 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.570835+0000", 2024-04-26T03:38:56.351 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:56.351 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.351 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.351 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:38:56.351 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.570835+0000", 2024-04-26T03:38:56.351 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:56.351 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.351 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.351 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:38:56.351 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.570836+0000", 2024-04-26T03:38:56.351 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4160000000000001e-06 2024-04-26T03:38:56.351 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.352 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.352 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:38:56.352 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.570979+0000", 2024-04-26T03:38:56.352 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014225500000000001 2024-04-26T03:38:56.352 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.352 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.352 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:38:56.352 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.570994+0000", 2024-04-26T03:38:56.352 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4885000000000001e-05 2024-04-26T03:38:56.352 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.352 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.352 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:38:56.352 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.571002+0000", 2024-04-26T03:38:56.352 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.6750000000000008e-06 2024-04-26T03:38:56.353 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.353 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.353 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:38:56.353 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.571066+0000", 2024-04-26T03:38:56.353 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.3583000000000004e-05 2024-04-26T03:38:56.353 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.353 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.353 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:38:56.353 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.571074+0000", 2024-04-26T03:38:56.353 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.0239999999999997e-06 2024-04-26T03:38:56.353 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.353 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.353 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:38:56.353 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.577224+0000", 2024-04-26T03:38:56.353 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0061504400000000001 2024-04-26T03:38:56.354 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.354 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.354 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:38:56.354 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.577267+0000", 2024-04-26T03:38:56.354 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.2908999999999999e-05 2024-04-26T03:38:56.354 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:56.354 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:38:56.354 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:56.354 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.354 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.354 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.3s5 35/32 ECSubWrite(tid=12, reqid=client.4556.0:110, at_version=35'8, trim_to=0'0, roll_forward_to=35'6))", 2024-04-26T03:38:56.354 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.575357+0000", 2024-04-26T03:38:56.354 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 4.7221016929999999, 2024-04-26T03:38:56.354 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0045976469999999998, 2024-04-26T03:38:56.355 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:38:56.355 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:38:56.355 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:38:56.355 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:38:56.355 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.355 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:38:56.355 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.575357+0000", 2024-04-26T03:38:56.355 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:56.355 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.355 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.355 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:38:56.355 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.575357+0000", 2024-04-26T03:38:56.355 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:56.355 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.356 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.356 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:38:56.356 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.575358+0000", 2024-04-26T03:38:56.356 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1900000000000001e-07 2024-04-26T03:38:56.356 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.356 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.356 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:38:56.356 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.575374+0000", 2024-04-26T03:38:56.356 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5835999999999999e-05 2024-04-26T03:38:56.356 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.356 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.356 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:38:56.356 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.575388+0000", 2024-04-26T03:38:56.356 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3845e-05 2024-04-26T03:38:56.357 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.357 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.357 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:38:56.357 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.575396+0000", 2024-04-26T03:38:56.357 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5469999999999997e-06 2024-04-26T03:38:56.357 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.357 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.357 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:38:56.357 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.575458+0000", 2024-04-26T03:38:56.357 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.1171000000000001e-05 2024-04-26T03:38:56.357 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.357 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.357 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:38:56.357 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.575466+0000", 2024-04-26T03:38:56.358 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.4670000000000004e-06 2024-04-26T03:38:56.358 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.358 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.358 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:38:56.358 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.579921+0000", 2024-04-26T03:38:56.358 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0044544770000000001 2024-04-26T03:38:56.358 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.358 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.358 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:38:56.358 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.579955+0000", 2024-04-26T03:38:56.358 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.4384999999999997e-05 2024-04-26T03:38:56.358 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:56.358 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:38:56.358 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:56.359 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.359 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.359 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.fs4 35/32 ECSubWrite(tid=11, reqid=client.4556.0:112, at_version=35'3, trim_to=0'0, roll_forward_to=34'2))", 2024-04-26T03:38:56.359 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.576574+0000", 2024-04-26T03:38:56.359 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 4.7208853519999998, 2024-04-26T03:38:56.359 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0063654619999999997, 2024-04-26T03:38:56.359 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:38:56.359 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:38:56.359 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:38:56.359 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:38:56.359 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.359 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:38:56.359 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.576574+0000", 2024-04-26T03:38:56.359 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:56.360 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.360 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.360 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:38:56.360 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.576574+0000", 2024-04-26T03:38:56.360 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:56.360 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.360 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.360 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:38:56.360 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.576574+0000", 2024-04-26T03:38:56.360 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.3600000000000003e-07 2024-04-26T03:38:56.360 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.360 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.360 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:38:56.360 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.577534+0000", 2024-04-26T03:38:56.360 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000959489 2024-04-26T03:38:56.361 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.361 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.361 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:38:56.361 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.577545+0000", 2024-04-26T03:38:56.361 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1158000000000001e-05 2024-04-26T03:38:56.361 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.361 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.361 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:38:56.361 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.577550+0000", 2024-04-26T03:38:56.361 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.2800000000000003e-06 2024-04-26T03:38:56.361 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.361 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.361 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:38:56.361 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.577605+0000", 2024-04-26T03:38:56.362 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.4882000000000001e-05 2024-04-26T03:38:56.362 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.362 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.362 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:38:56.362 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.577613+0000", 2024-04-26T03:38:56.362 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9160000000000001e-06 2024-04-26T03:38:56.362 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.362 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.362 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:38:56.362 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.582907+0000", 2024-04-26T03:38:56.362 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0052939060000000001 2024-04-26T03:38:56.362 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.362 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.362 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:38:56.363 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.582939+0000", 2024-04-26T03:38:56.363 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.2194999999999999e-05 2024-04-26T03:38:56.363 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:56.363 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:38:56.363 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:56.363 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.363 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.363 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.1s6 35/32 ECSubWrite(tid=16, reqid=client.4556.0:118, at_version=35'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:38:56.363 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.585549+0000", 2024-04-26T03:38:56.363 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 4.7119096599999999, 2024-04-26T03:38:56.363 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0065009400000000002, 2024-04-26T03:38:56.363 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:38:56.363 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:38:56.364 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:38:56.364 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:38:56.364 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.364 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:38:56.364 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.585549+0000", 2024-04-26T03:38:56.364 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:56.364 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.364 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.364 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:38:56.364 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.585549+0000", 2024-04-26T03:38:56.364 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:56.364 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.364 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.364 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:38:56.365 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.585550+0000", 2024-04-26T03:38:56.365 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.3799999999999998e-07 2024-04-26T03:38:56.365 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.365 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.365 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:38:56.365 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.586338+0000", 2024-04-26T03:38:56.365 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00078816900000000002 2024-04-26T03:38:56.365 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.365 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.365 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:38:56.365 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.586352+0000", 2024-04-26T03:38:56.365 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4386000000000001e-05 2024-04-26T03:38:56.365 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.365 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.365 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:38:56.366 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.586362+0000", 2024-04-26T03:38:56.366 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.9899999999999992e-06 2024-04-26T03:38:56.366 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.366 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.366 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:38:56.366 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.586432+0000", 2024-04-26T03:38:56.366 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.9306000000000004e-05 2024-04-26T03:38:56.366 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.366 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.366 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:38:56.366 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.586437+0000", 2024-04-26T03:38:56.366 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.9799999999999998e-06 2024-04-26T03:38:56.366 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.367 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.367 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:38:56.367 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.592022+0000", 2024-04-26T03:38:56.367 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0055854090000000004 2024-04-26T03:38:56.367 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.367 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.367 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:38:56.367 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.592050+0000", 2024-04-26T03:38:56.367 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8362000000000001e-05 2024-04-26T03:38:56.367 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:56.367 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:38:56.367 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:56.367 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.367 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.368 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.fs4 35/32 ECSubWrite(tid=15, reqid=client.4556.0:136, at_version=35'5, trim_to=0'0, roll_forward_to=35'4))", 2024-04-26T03:38:56.368 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.615534+0000", 2024-04-26T03:38:56.368 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 4.6819249489999999, 2024-04-26T03:38:56.368 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0058271440000000002, 2024-04-26T03:38:56.368 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:38:56.368 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:38:56.368 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:38:56.368 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:38:56.368 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.368 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:38:56.368 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.615534+0000", 2024-04-26T03:38:56.368 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:56.368 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.368 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.369 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:38:56.369 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.615534+0000", 2024-04-26T03:38:56.369 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:56.369 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.369 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.369 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:38:56.369 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.615535+0000", 2024-04-26T03:38:56.369 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.0400000000000005e-07 2024-04-26T03:38:56.369 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.369 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.369 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:38:56.369 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.616126+0000", 2024-04-26T03:38:56.369 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00059120199999999998 2024-04-26T03:38:56.369 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.369 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.370 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:38:56.370 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.616142+0000", 2024-04-26T03:38:56.370 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6223999999999999e-05 2024-04-26T03:38:56.370 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.370 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.370 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:38:56.370 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.616153+0000", 2024-04-26T03:38:56.370 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0781999999999999e-05 2024-04-26T03:38:56.370 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.370 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.370 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:38:56.370 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.616230+0000", 2024-04-26T03:38:56.370 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.6668999999999994e-05 2024-04-26T03:38:56.371 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.371 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.371 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:38:56.371 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.616241+0000", 2024-04-26T03:38:56.371 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1177e-05 2024-04-26T03:38:56.371 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.371 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.371 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:38:56.371 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.621328+0000", 2024-04-26T03:38:56.371 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0050865320000000004 2024-04-26T03:38:56.371 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.371 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.371 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:38:56.371 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.621361+0000", 2024-04-26T03:38:56.372 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.3654e-05 2024-04-26T03:38:56.372 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:56.372 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:38:56.372 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:56.372 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.372 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.372 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.3s5 35/32 ECSubWrite(tid=15, reqid=client.4556.0:142, at_version=35'9, trim_to=0'0, roll_forward_to=35'8))", 2024-04-26T03:38:56.372 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.622899+0000", 2024-04-26T03:38:56.372 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 4.6745602909999997, 2024-04-26T03:38:56.372 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0045371300000000003, 2024-04-26T03:38:56.372 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:38:56.372 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:38:56.372 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:38:56.372 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:38:56.373 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.373 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:38:56.373 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.622899+0000", 2024-04-26T03:38:56.373 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:56.373 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.373 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.373 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:38:56.373 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.622899+0000", 2024-04-26T03:38:56.373 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:56.373 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.373 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.373 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:38:56.373 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.622900+0000", 2024-04-26T03:38:56.373 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.8899999999999998e-07 2024-04-26T03:38:56.374 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.374 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.374 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:38:56.374 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.622946+0000", 2024-04-26T03:38:56.374 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6326999999999999e-05 2024-04-26T03:38:56.374 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.374 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.374 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:38:56.374 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.622961+0000", 2024-04-26T03:38:56.374 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5444999999999999e-05 2024-04-26T03:38:56.374 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.374 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.374 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:38:56.375 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.622971+0000", 2024-04-26T03:38:56.375 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1710000000000008e-06 2024-04-26T03:38:56.375 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.375 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.375 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:38:56.375 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.623055+0000", 2024-04-26T03:38:56.375 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.4815999999999996e-05 2024-04-26T03:38:56.375 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.375 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.375 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:38:56.375 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.623064+0000", 2024-04-26T03:38:56.375 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.9809999999999992e-06 2024-04-26T03:38:56.375 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.375 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.376 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:38:56.376 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.627396+0000", 2024-04-26T03:38:56.376 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0043312860000000002 2024-04-26T03:38:56.376 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.376 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.376 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:38:56.376 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.627436+0000", 2024-04-26T03:38:56.376 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.0114999999999998e-05 2024-04-26T03:38:56.376 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:56.376 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:38:56.376 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:56.376 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.376 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.376 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.6s3 35/32 ECSubWrite(tid=0, reqid=client.4556.0:146, at_version=35'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:38:56.377 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.630308+0000", 2024-04-26T03:38:56.377 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 4.667150715, 2024-04-26T03:38:56.377 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0078713510000000004, 2024-04-26T03:38:56.377 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:38:56.377 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:38:56.377 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:38:56.377 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:38:56.377 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.377 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:38:56.377 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.630308+0000", 2024-04-26T03:38:56.377 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:56.377 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.377 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.377 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:38:56.378 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.630308+0000", 2024-04-26T03:38:56.378 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:56.378 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.378 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.378 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:38:56.378 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.630309+0000", 2024-04-26T03:38:56.378 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.9500000000000002e-07 2024-04-26T03:38:56.378 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.378 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.378 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:38:56.378 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.630752+0000", 2024-04-26T03:38:56.378 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00044313600000000001 2024-04-26T03:38:56.378 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.378 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.379 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:38:56.379 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.630769+0000", 2024-04-26T03:38:56.379 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6541999999999998e-05 2024-04-26T03:38:56.379 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.379 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.379 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:38:56.379 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.630777+0000", 2024-04-26T03:38:56.379 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.3410000000000003e-06 2024-04-26T03:38:56.379 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.379 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.379 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:38:56.379 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.630840+0000", 2024-04-26T03:38:56.379 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.3303e-05 2024-04-26T03:38:56.379 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.380 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.380 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:38:56.380 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.630848+0000", 2024-04-26T03:38:56.380 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2630000000000003e-06 2024-04-26T03:38:56.380 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.380 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.380 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:38:56.380 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.638146+0000", 2024-04-26T03:38:56.380 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0072979819999999997 2024-04-26T03:38:56.380 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.380 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.380 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:38:56.380 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.638180+0000", 2024-04-26T03:38:56.380 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.3188999999999997e-05 2024-04-26T03:38:56.380 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:56.381 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:38:56.381 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:56.381 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.381 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.381 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.6s3 35/32 ECSubWrite(tid=1, reqid=client.4556.0:147, at_version=35'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:38:56.381 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.634041+0000", 2024-04-26T03:38:56.381 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 4.6634182710000003, 2024-04-26T03:38:56.381 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0061232500000000002, 2024-04-26T03:38:56.381 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:38:56.381 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:38:56.381 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:38:56.381 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:38:56.381 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.381 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:38:56.382 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.634041+0000", 2024-04-26T03:38:56.382 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:56.382 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.382 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.382 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:38:56.382 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.634041+0000", 2024-04-26T03:38:56.382 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:56.382 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.382 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.382 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:38:56.382 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.634041+0000", 2024-04-26T03:38:56.382 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6800000000000001e-07 2024-04-26T03:38:56.382 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.382 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.383 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:38:56.383 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.634221+0000", 2024-04-26T03:38:56.383 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00017996200000000001 2024-04-26T03:38:56.383 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.383 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.383 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:38:56.383 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.634235+0000", 2024-04-26T03:38:56.383 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3478e-05 2024-04-26T03:38:56.383 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.383 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.383 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:38:56.383 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.634273+0000", 2024-04-26T03:38:56.383 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8469e-05 2024-04-26T03:38:56.383 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.383 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.384 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:38:56.384 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.634756+0000", 2024-04-26T03:38:56.384 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00048309699999999999 2024-04-26T03:38:56.384 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.384 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.384 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:38:56.384 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.634764+0000", 2024-04-26T03:38:56.384 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2749999999999995e-06 2024-04-26T03:38:56.384 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.384 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.384 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:38:56.384 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.640129+0000", 2024-04-26T03:38:56.384 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.005364654 2024-04-26T03:38:56.384 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.385 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.385 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:38:56.385 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.640164+0000", 2024-04-26T03:38:56.385 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.4847e-05 2024-04-26T03:38:56.385 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:56.385 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:38:56.385 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:56.385 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.385 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.385 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.6s3 35/32 ECSubWrite(tid=2, reqid=client.4556.0:148, at_version=35'3, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:38:56.385 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.636626+0000", 2024-04-26T03:38:56.385 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 4.6608330269999998, 2024-04-26T03:38:56.385 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0066727840000000002, 2024-04-26T03:38:56.385 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:38:56.386 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:38:56.386 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:38:56.386 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:38:56.386 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.386 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:38:56.386 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.636626+0000", 2024-04-26T03:38:56.386 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:56.386 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.386 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.386 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:38:56.386 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.636626+0000", 2024-04-26T03:38:56.386 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:38:56.386 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.386 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.387 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:38:56.387 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.636627+0000", 2024-04-26T03:38:56.387 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7299999999999995e-07 2024-04-26T03:38:56.387 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.387 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.387 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:38:56.387 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.636643+0000", 2024-04-26T03:38:56.387 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6765999999999998e-05 2024-04-26T03:38:56.387 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.387 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.387 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:38:56.387 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.636659+0000", 2024-04-26T03:38:56.387 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5831000000000001e-05 2024-04-26T03:38:56.387 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.387 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.388 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:38:56.388 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.636668+0000", 2024-04-26T03:38:56.388 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5280000000000003e-06 2024-04-26T03:38:56.388 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.388 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.388 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:38:56.388 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.637982+0000", 2024-04-26T03:38:56.388 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001314684 2024-04-26T03:38:56.388 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.388 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.388 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:38:56.388 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.637990+0000", 2024-04-26T03:38:56.388 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5279999999999998e-06 2024-04-26T03:38:56.388 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.389 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.389 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:38:56.389 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.643262+0000", 2024-04-26T03:38:56.389 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0052716009999999999 2024-04-26T03:38:56.389 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:38:56.389 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:38:56.389 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:38:56.389 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.643299+0000", 2024-04-26T03:38:56.389 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7172999999999998e-05 2024-04-26T03:38:56.389 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:56.389 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:38:56.389 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:56.389 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:38:56.389 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:38:56.390 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:38:56.391 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-26T03:38:56.392 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:38:56.389+0000 7f60d3c4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31757) UID: 0 2024-04-26T03:38:56.425 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-26T03:38:56.491 INFO:tasks.ceph.osd.12.smithi028.stderr:2024-04-26T03:38:56.490+0000 7f114fe14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 31967) UID: 0 2024-04-26T03:38:56.505 INFO:teuthology.orchestra.run.smithi028.stderr:nodeep-scrub is unset 2024-04-26T03:38:56.509 INFO:tasks.rados.rados.0.smithi092.stdout:98: done (0 left) 2024-04-26T03:38:56.509 INFO:tasks.rados.rados.0.smithi092.stdout:99: read oid 39 snap -1 2024-04-26T03:38:56.509 INFO:tasks.rados.rados.0.smithi092.stdout:99: expect (ObjNum 38 snap 0 seq_num 38) 2024-04-26T03:38:56.509 INFO:tasks.rados.rados.0.smithi092.stdout:100: read oid 26 snap -1 2024-04-26T03:38:56.509 INFO:tasks.rados.rados.0.smithi092.stdout:100: expect (ObjNum 25 snap 0 seq_num 25) 2024-04-26T03:38:56.510 INFO:tasks.rados.rados.0.smithi092.stdout:101: rollback oid 10 current snap is 3 2024-04-26T03:38:56.510 INFO:tasks.rados.rados.0.smithi092.stdout:rollback oid 10 to 0 2024-04-26T03:38:56.510 INFO:tasks.rados.rados.0.smithi092.stdout:102: read oid 27 snap 0 2024-04-26T03:38:56.510 INFO:tasks.rados.rados.0.smithi092.stdout:102: expect deleted 2024-04-26T03:38:56.510 INFO:tasks.rados.rados.0.smithi092.stdout:103: rmattr oid 2 current snap is 3 2024-04-26T03:38:56.510 INFO:tasks.rados.rados.0.smithi092.stdout:103: done (4 left) 2024-04-26T03:38:56.510 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 28 current snap is 3 2024-04-26T03:38:56.510 INFO:tasks.rados.rados.0.smithi092.stdout:104: seq_num 65 ranges {458752=720896,1179648=376832} 2024-04-26T03:38:56.517 INFO:tasks.rados.rados.0.smithi092.stdout:104: writing smithi09235267-28 from 458752 to 1179648 tid 1 2024-04-26T03:38:56.521 INFO:tasks.rados.rados.0.smithi092.stdout:104: writing smithi09235267-28 from 1179648 to 1556480 tid 2 2024-04-26T03:38:56.521 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 19 current snap is 3 2024-04-26T03:38:56.521 INFO:tasks.rados.rados.0.smithi092.stdout:105: seq_num 66 ranges {311296=737280,1048576=376832} 2024-04-26T03:38:56.527 INFO:tasks.rados.rados.0.smithi092.stdout:105: writing smithi09235267-19 from 311296 to 1048576 tid 1 2024-04-26T03:38:56.530 INFO:tasks.rados.rados.0.smithi092.stdout:105: writing smithi09235267-19 from 1048576 to 1425408 tid 2 2024-04-26T03:38:56.530 INFO:tasks.rados.rados.0.smithi092.stdout:106: snap_create 2024-04-26T03:38:56.530 INFO:tasks.rados.rados.0.smithi092.stdout:100: expect (ObjNum 25 snap 0 seq_num 25) 2024-04-26T03:38:56.542 INFO:tasks.rados.rados.0.smithi092.stdout:99: expect (ObjNum 38 snap 0 seq_num 38) 2024-04-26T03:38:56.553 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:38:56.553 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-04-26T03:38:56.553 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 0 2024-04-26T03:38:56.553 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:38:56.559 INFO:tasks.rados.rados.0.smithi092.stdout:101: finishing rollback tid 0 to smithi09235267-10 2024-04-26T03:38:56.559 INFO:tasks.rados.rados.0.smithi092.stdout:101: finishing rollback tid 1 to smithi09235267-10 2024-04-26T03:38:56.560 INFO:tasks.rados.rados.0.smithi092.stdout:101: finishing rollback tid 2 to smithi09235267-10 2024-04-26T03:38:56.560 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 10 v 16 (ObjNum 54 snap 0 seq_num 54) dirty exists 2024-04-26T03:38:56.560 INFO:tasks.rados.rados.0.smithi092.stdout:104: finishing write tid 1 to smithi09235267-28 2024-04-26T03:38:56.560 INFO:tasks.rados.rados.0.smithi092.stdout:104: finishing write tid 2 to smithi09235267-28 2024-04-26T03:38:56.560 INFO:tasks.rados.rados.0.smithi092.stdout:104: finishing write tid 3 to smithi09235267-28 2024-04-26T03:38:56.560 INFO:tasks.rados.rados.0.smithi092.stdout:104: finishing write tid 4 to smithi09235267-28 2024-04-26T03:38:56.560 INFO:tasks.rados.rados.0.smithi092.stdout:104: oid 28 updating version 0 to 14 2024-04-26T03:38:56.560 INFO:tasks.rados.rados.0.smithi092.stdout:104: oid 28 updating version 14 to 15 2024-04-26T03:38:56.560 INFO:tasks.rados.rados.0.smithi092.stdout:104: oid 28 updating version 15 to 16 2024-04-26T03:38:56.560 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 28 v 16 (ObjNum 65 snap 3 seq_num 65) dirty exists 2024-04-26T03:38:56.560 INFO:tasks.rados.rados.0.smithi092.stdout:104: left oid 28 (ObjNum 65 snap 3 seq_num 65) 2024-04-26T03:38:56.560 INFO:tasks.rados.rados.0.smithi092.stdout:105: finishing write tid 1 to smithi09235267-19 2024-04-26T03:38:56.560 INFO:tasks.rados.rados.0.smithi092.stdout:105: finishing write tid 2 to smithi09235267-19 2024-04-26T03:38:56.560 INFO:tasks.rados.rados.0.smithi092.stdout:105: finishing write tid 3 to smithi09235267-19 2024-04-26T03:38:56.561 INFO:tasks.rados.rados.0.smithi092.stdout:105: finishing write tid 4 to smithi09235267-19 2024-04-26T03:38:56.561 INFO:tasks.rados.rados.0.smithi092.stdout:105: oid 19 updating version 0 to 9 2024-04-26T03:38:56.561 INFO:tasks.rados.rados.0.smithi092.stdout:105: oid 19 version 9 is already newer than 8 2024-04-26T03:38:56.561 INFO:tasks.rados.rados.0.smithi092.stdout:105: oid 19 updating version 9 to 10 2024-04-26T03:38:56.561 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 19 v 10 (ObjNum 66 snap 3 seq_num 66) dirty exists 2024-04-26T03:38:56.561 INFO:tasks.rados.rados.0.smithi092.stdout:105: left oid 19 (ObjNum 66 snap 3 seq_num 66) 2024-04-26T03:38:56.562 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-26T03:38:56.569 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0: {} 2024-04-26T03:38:56.569 INFO:teuthology.orchestra.run.smithi028.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T03:38:56.588 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1: {} 2024-04-26T03:38:56.588 INFO:teuthology.orchestra.run.smithi028.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T03:38:56.592 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-26T03:38:56.591+0000 7f610b3a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31816) UID: 0 2024-04-26T03:38:56.606 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2: {} 2024-04-26T03:38:56.606 INFO:teuthology.orchestra.run.smithi028.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T03:38:56.624 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3: {} 2024-04-26T03:38:56.625 INFO:teuthology.orchestra.run.smithi028.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T03:38:56.641 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4: {} 2024-04-26T03:38:56.642 INFO:teuthology.orchestra.run.smithi028.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T03:38:56.661 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5: {} 2024-04-26T03:38:56.661 INFO:teuthology.orchestra.run.smithi028.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T03:38:56.670 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:38:56.670 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-04-26T03:38:56.670 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-04-26T03:38:56.670 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-04-26T03:38:56.670 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:38:56.677 INFO:teuthology.orchestra.run.smithi028.stdout:osd.6: {} 2024-04-26T03:38:56.678 INFO:teuthology.orchestra.run.smithi028.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T03:38:56.679 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-26T03:38:56.693 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:38:56.691+0000 7fba0d2ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31961) UID: 0 2024-04-26T03:38:56.696 INFO:teuthology.orchestra.run.smithi028.stdout:osd.7: {} 2024-04-26T03:38:56.696 INFO:teuthology.orchestra.run.smithi028.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T03:38:56.713 INFO:teuthology.orchestra.run.smithi028.stdout:osd.8: {} 2024-04-26T03:38:56.713 INFO:teuthology.orchestra.run.smithi028.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T03:38:56.731 INFO:teuthology.orchestra.run.smithi028.stdout:osd.9: {} 2024-04-26T03:38:56.732 INFO:teuthology.orchestra.run.smithi028.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T03:38:56.749 INFO:teuthology.orchestra.run.smithi028.stdout:osd.10: {} 2024-04-26T03:38:56.749 INFO:teuthology.orchestra.run.smithi028.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T03:38:56.768 INFO:teuthology.orchestra.run.smithi028.stdout:osd.11: {} 2024-04-26T03:38:56.768 INFO:teuthology.orchestra.run.smithi028.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T03:38:56.785 INFO:teuthology.orchestra.run.smithi028.stdout:osd.12: {} 2024-04-26T03:38:56.785 INFO:teuthology.orchestra.run.smithi028.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T03:38:56.785 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:56.785 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:56.795 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-04-26T03:38:56.795 INFO:tasks.ceph.osd.10.smithi066.stderr:2024-04-26T03:38:56.792+0000 7f130fcc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 31763) UID: 0 2024-04-26T03:38:56.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:38:56.802+0000 7f6ec1275640 0 -- 172.21.15.28:0/1784981475 >> [v2:172.21.15.42:6817/301360893,v1:172.21.15.42:6819/301360893] conn(0x7f6ebc21ca60 msgr2=0x7f6ebc21ee50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-26T03:38:56.804 INFO:teuthology.orchestra.run.smithi028.stdout:osd.13: {} 2024-04-26T03:38:56.804 INFO:teuthology.orchestra.run.smithi028.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T03:38:56.821 INFO:teuthology.orchestra.run.smithi028.stdout:osd.14: {} 2024-04-26T03:38:56.822 INFO:teuthology.orchestra.run.smithi028.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T03:38:56.839 INFO:teuthology.orchestra.run.smithi028.stdout:osd.15: {} 2024-04-26T03:38:56.839 INFO:teuthology.orchestra.run.smithi028.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T03:38:56.895 INFO:tasks.ceph.osd.10.smithi066.stderr:2024-04-26T03:38:56.893+0000 7f130fcc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 31763) UID: 0 2024-04-26T03:38:56.942 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:56.942 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:56.951 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-04-26T03:38:57.035 INFO:tasks.ceph.osd.10.smithi066.stderr:2024-04-26T03:38:57.033+0000 7f130fcc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 31763) UID: 0 2024-04-26T03:38:57.097 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:57.098 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:57.098 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-04-26T03:38:57.096+0000 7f745118d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 31821) UID: 0 2024-04-26T03:38:57.106 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-04-26T03:38:57.199 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-26T03:38:57.198+0000 7f610b3a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31816) UID: 0 2024-04-26T03:38:57.253 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:57.253 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:57.262 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-04-26T03:38:57.300 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-04-26T03:38:57.299+0000 7fe9223c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31954) UID: 0 2024-04-26T03:38:57.369 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:57.369 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:57.378 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-04-26T03:38:57.402 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:38:57.400+0000 7fb78c544640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31956) UID: 0 2024-04-26T03:38:57.484 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:57.485 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:57.494 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-04-26T03:38:57.515 INFO:tasks.rados.rados.0.smithi092.stdout:99: done (6 left) 2024-04-26T03:38:57.515 INFO:tasks.rados.rados.0.smithi092.stdout:100: done (5 left) 2024-04-26T03:38:57.515 INFO:tasks.rados.rados.0.smithi092.stdout:101: done (4 left) 2024-04-26T03:38:57.515 INFO:tasks.rados.rados.0.smithi092.stdout:102: done (3 left) 2024-04-26T03:38:57.515 INFO:tasks.rados.rados.0.smithi092.stdout:104: done (2 left) 2024-04-26T03:38:57.515 INFO:tasks.rados.rados.0.smithi092.stdout:105: done (1 left) 2024-04-26T03:38:57.515 INFO:tasks.rados.rados.0.smithi092.stdout:106: done (0 left) 2024-04-26T03:38:57.516 INFO:tasks.rados.rados.0.smithi092.stdout:107: rollback oid 3 current snap is 4 2024-04-26T03:38:57.516 INFO:tasks.rados.rados.0.smithi092.stdout:rollback oid 3 to 3 2024-04-26T03:38:57.516 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 13 current snap is 4 2024-04-26T03:38:57.516 INFO:tasks.rados.rados.0.smithi092.stdout:108: seq_num 67 ranges {0=753664,753664=376832} 2024-04-26T03:38:57.519 INFO:tasks.rados.rados.0.smithi092.stdout:108: writing smithi09235267-13 from 0 to 753664 tid 1 2024-04-26T03:38:57.521 INFO:tasks.rados.rados.0.smithi092.stdout:108: writing smithi09235267-13 from 753664 to 1130496 tid 2 2024-04-26T03:38:57.521 INFO:tasks.rados.rados.0.smithi092.stdout:109: delete oid 11 current snap is 4 2024-04-26T03:38:57.521 INFO:tasks.rados.rados.0.smithi092.stdout:107: finishing rollback tid 0 to smithi09235267-3 2024-04-26T03:38:57.526 INFO:tasks.rados.rados.0.smithi092.stdout:109: done (2 left) 2024-04-26T03:38:57.526 INFO:tasks.rados.rados.0.smithi092.stdout:110: snap_remove snap 0 2024-04-26T03:38:57.538 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-04-26T03:38:57.536+0000 7fee562fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31759) UID: 0 2024-04-26T03:38:57.604 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:38:57.602+0000 7fb78c544640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31956) UID: 0 2024-04-26T03:38:57.604 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:57.605 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:57.613 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-26T03:38:57.705 INFO:tasks.ceph.osd.10.smithi066.stderr:2024-04-26T03:38:57.704+0000 7f130fcc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 31763) UID: 0 2024-04-26T03:38:57.729 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:57.729 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:57.739 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-26T03:38:57.806 INFO:tasks.ceph.osd.9.smithi042.stderr:2024-04-26T03:38:57.804+0000 7fdac63b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 31960) UID: 0 2024-04-26T03:38:57.819 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-04-26T03:38:57.819 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-26T03:38:57.820 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_0 2024-04-26T03:38:57.820 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T03:38:57.842 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:57.842 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:57.851 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-26T03:38:57.907 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:38:57.905+0000 7fb78c544640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31956) UID: 0 2024-04-26T03:38:57.954 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:57.954 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:57.963 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-26T03:38:58.007 INFO:tasks.ceph.osd.14.smithi066.stderr:2024-04-26T03:38:58.006+0000 7f5395f08640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 31765) UID: 0 2024-04-26T03:38:58.068 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:58.068 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:58.076 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-26T03:38:58.108 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:38:58.107+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:38:58.129 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:38:58.131 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-04-26T03:38:58.142 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2024-04-26T03:38:58.053545+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":29311520,"num_objects":52,"num_object_clones":6,"num_object_copies":354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":300,"num_read_kb":17280,"num_write":428,"num_write_kb":34356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":318,"ondisk_log_size":318,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":124,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":504408,"kb_used_data":55420,"kb_used_omap":24,"kb_used_meta":448743,"kb_avail":1509445032,"statfs":{"total":1546188226560,"available":1545671712768,"internally_reserved":0,"allocated":56750080,"data_stored":58742276,"data_compressed":485549,"data_compressed_allocated":1056768,"data_compressed_original":3006464,"omap_allocated":25522,"internal_metadata":459512910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9945088,"num_objects":18,"num_object_clones":6,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":48,"num_read_kb":8517,"num_write":101,"num_write_kb":9712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000912"},"pg_stats":[{"pgid":"3.f","version":"37'11","reported_seq":53,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T03:38:55.511164+0000","last_change":"2024-04-26T03:38:55.511554+0000","last_active":"2024-04-26T03:38:55.511164+0000","last_peered":"2024-04-26T03:38:55.511164+0000","last_clean":"2024-04-26T03:38:55.511164+0000","last_became_active":"2024-04-26T03:38:49.486850+0000","last_became_peered":"2024-04-26T03:38:49.486850+0000","last_unstale":"2024-04-26T03:38:55.511164+0000","last_undegraded":"2024-04-26T03:38:55.511164+0000","last_fullsized":"2024-04-26T03:38:55.511164+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:48.978362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056541000000000002,"stat_sum":{"num_bytes":2490368,"num_objects":4,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":11,"num_read_kb":1829,"num_write":29,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,13,5,14,2,1,0],"acting":[11,13,5,14,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"4","length":"1"}]},{"pgid":"3.e","version":"35'14","reported_seq":70,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T03:38:56.516011+0000","last_change":"2024-04-26T03:38:56.516550+0000","last_active":"2024-04-26T03:38:56.516011+0000","last_peered":"2024-04-26T03:38:56.516011+0000","last_clean":"2024-04-26T03:38:56.516011+0000","last_became_active":"2024-04-26T03:38:49.941260+0000","last_became_peered":"2024-04-26T03:38:49.941260+0000","last_unstale":"2024-04-26T03:38:56.516011+0000","last_undegraded":"2024-04-26T03:38:56.516011+0000","last_fullsized":"2024-04-26T03:38:56.516011+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:51.502432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040673949999999997,"stat_sum":{"num_bytes":3817472,"num_objects":4,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":18,"num_read_kb":3464,"num_write":14,"num_write_kb":3728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,14,6,11,8,9,12],"acting":[13,14,6,11,8,9,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"3","length":"2"}]},{"pgid":"3.d","version":"35'7","reported_seq":53,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T03:38:56.516300+0000","last_change":"2024-04-26T03:38:56.516671+0000","last_active":"2024-04-26T03:38:56.516300+0000","last_peered":"2024-04-26T03:38:56.516300+0000","last_clean":"2024-04-26T03:38:56.516300+0000","last_became_active":"2024-04-26T03:38:49.511783+0000","last_became_peered":"2024-04-26T03:38:49.511783+0000","last_unstale":"2024-04-26T03:38:56.516300+0000","last_undegraded":"2024-04-26T03:38:56.516300+0000","last_fullsized":"2024-04-26T03:38:56.516300+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:00.858212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058021600000000004,"stat_sum":{"num_bytes":606208,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":3,"num_write":7,"num_write_kb":880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,9,1,5,8,4],"acting":[7,3,9,1,5,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3","length":"2"}]},{"pgid":"3.c","version":"37'6","reported_seq":43,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T03:38:55.510976+0000","last_change":"2024-04-26T03:38:55.511320+0000","last_active":"2024-04-26T03:38:55.510976+0000","last_peered":"2024-04-26T03:38:55.510976+0000","last_clean":"2024-04-26T03:38:55.510976+0000","last_became_active":"2024-04-26T03:38:49.941017+0000","last_became_peered":"2024-04-26T03:38:49.941017+0000","last_unstale":"2024-04-26T03:38:55.510976+0000","last_undegraded":"2024-04-26T03:38:55.510976+0000","last_fullsized":"2024-04-26T03:38:55.510976+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:08.671306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060879,"stat_sum":{"num_bytes":245760,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":24,"num_write_kb":240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,0,1,9,12,14],"acting":[11,8,0,1,9,12,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"4","length":"1"}]},{"pgid":"3.b","version":"35'12","reported_seq":41,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-26T03:38:51.833946+0000","last_change":"2024-04-26T03:38:49.941847+0000","last_active":"2024-04-26T03:38:51.833946+0000","last_peered":"2024-04-26T03:38:51.833946+0000","last_clean":"2024-04-26T03:38:51.833946+0000","last_became_active":"2024-04-26T03:38:49.940561+0000","last_became_peered":"2024-04-26T03:38:49.940561+0000","last_unstale":"2024-04-26T03:38:51.833946+0000","last_undegraded":"2024-04-26T03:38:51.833946+0000","last_fullsized":"2024-04-26T03:38:51.833946+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:58.752641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1867776,"num_objects":4,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":2119,"num_write":31,"num_write_kb":2032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,10,12,1,7,14],"acting":[4,0,10,12,1,7,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"35'18","reported_seq":55,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T03:38:53.417658+0000","last_change":"2024-04-26T03:38:49.487414+0000","last_active":"2024-04-26T03:38:53.417658+0000","last_peered":"2024-04-26T03:38:53.417658+0000","last_clean":"2024-04-26T03:38:53.417658+0000","last_became_active":"2024-04-26T03:38:49.486365+0000","last_became_peered":"2024-04-26T03:38:49.486365+0000","last_unstale":"2024-04-26T03:38:53.417658+0000","last_undegraded":"2024-04-26T03:38:53.417658+0000","last_fullsized":"2024-04-26T03:38:53.417658+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:06.952341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4997120,"num_objects":7,"num_object_clones":0,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":23,"num_read_kb":3082,"num_write":18,"num_write_kb":4880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,10,13,1,11],"acting":[3,7,6,10,13,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"35'6","reported_seq":55,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T03:38:56.517864+0000","last_change":"2024-04-26T03:38:56.518618+0000","last_active":"2024-04-26T03:38:56.517864+0000","last_peered":"2024-04-26T03:38:56.517864+0000","last_clean":"2024-04-26T03:38:56.517864+0000","last_became_active":"2024-04-26T03:38:49.512126+0000","last_became_peered":"2024-04-26T03:38:49.512126+0000","last_unstale":"2024-04-26T03:38:56.517864+0000","last_undegraded":"2024-04-26T03:38:56.517864+0000","last_fullsized":"2024-04-26T03:38:56.517864+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:07.298480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061642609999999999,"stat_sum":{"num_bytes":1228800,"num_objects":3,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":10,"num_read_kb":1252,"num_write":6,"num_write_kb":1200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2,10,9,15,8,4],"acting":[13,2,10,9,15,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"3","length":"2"}]},{"pgid":"3.8","version":"40'10","reported_seq":65,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T03:38:56.545982+0000","last_change":"2024-04-26T03:38:56.516809+0000","last_active":"2024-04-26T03:38:56.545982+0000","last_peered":"2024-04-26T03:38:56.545982+0000","last_clean":"2024-04-26T03:38:56.545982+0000","last_became_active":"2024-04-26T03:38:49.487363+0000","last_became_peered":"2024-04-26T03:38:49.487363+0000","last_unstale":"2024-04-26T03:38:56.545982+0000","last_undegraded":"2024-04-26T03:38:56.545982+0000","last_fullsized":"2024-04-26T03:38:56.545982+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:10.849932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065928099999999995,"stat_sum":{"num_bytes":1966080,"num_objects":4,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":19,"num_read_kb":1927,"num_write":9,"num_write_kb":1920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,10,11,15,5],"acting":[6,0,1,10,11,15,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"3","length":"2"}]},{"pgid":"1.0","version":"27'157","reported_seq":237,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T03:38:56.516331+0000","last_change":"2024-04-26T03:38:34.390206+0000","last_active":"2024-04-26T03:38:56.516331+0000","last_peered":"2024-04-26T03:38:56.516331+0000","last_clean":"2024-04-26T03:38:56.516331+0000","last_became_active":"2024-04-26T03:38:34.389099+0000","last_became_peered":"2024-04-26T03:38:34.389099+0000","last_unstale":"2024-04-26T03:38:56.516331+0000","last_undegraded":"2024-04-26T03:38:56.516331+0000","last_fullsized":"2024-04-26T03:38:56.516331+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:32.306810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:32.306810+0000","last_clean_scrub_stamp":"2024-04-26T03:38:32.306810+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:40.216975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":36,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T03:38:56.507965+0000","last_change":"2024-04-26T03:38:36.431667+0000","last_active":"2024-04-26T03:38:56.507965+0000","last_peered":"2024-04-26T03:38:56.507965+0000","last_clean":"2024-04-26T03:38:56.507965+0000","last_became_active":"2024-04-26T03:38:36.431449+0000","last_became_peered":"2024-04-26T03:38:36.431449+0000","last_unstale":"2024-04-26T03:38:56.507965+0000","last_undegraded":"2024-04-26T03:38:56.507965+0000","last_fullsized":"2024-04-26T03:38:56.507965+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:47.562648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.2","version":"35'11","reported_seq":59,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T03:38:56.508341+0000","last_change":"2024-04-26T03:38:56.508650+0000","last_active":"2024-04-26T03:38:56.508341+0000","last_peered":"2024-04-26T03:38:56.508341+0000","last_clean":"2024-04-26T03:38:56.508341+0000","last_became_active":"2024-04-26T03:38:49.511901+0000","last_became_peered":"2024-04-26T03:38:49.511901+0000","last_unstale":"2024-04-26T03:38:56.508341+0000","last_undegraded":"2024-04-26T03:38:56.508341+0000","last_fullsized":"2024-04-26T03:38:56.508341+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:04.430841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050564100000000003,"stat_sum":{"num_bytes":1097728,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":4,"num_write":11,"num_write_kb":2272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,5,3,9,15,10,11],"acting":[14,5,3,9,15,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"3","length":"2"}]},{"pgid":"2.0","version":"0'0","reported_seq":28,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T03:38:53.417539+0000","last_change":"2024-04-26T03:38:36.431325+0000","last_active":"2024-04-26T03:38:53.417539+0000","last_peered":"2024-04-26T03:38:53.417539+0000","last_clean":"2024-04-26T03:38:53.417539+0000","last_became_active":"2024-04-26T03:38:36.431121+0000","last_became_peered":"2024-04-26T03:38:36.431121+0000","last_unstale":"2024-04-26T03:38:53.417539+0000","last_undegraded":"2024-04-26T03:38:53.417539+0000","last_fullsized":"2024-04-26T03:38:53.417539+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:53.294619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"35'2","reported_seq":23,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-26T03:38:51.594325+0000","last_change":"2024-04-26T03:38:49.512649+0000","last_active":"2024-04-26T03:38:51.594325+0000","last_peered":"2024-04-26T03:38:51.594325+0000","last_clean":"2024-04-26T03:38:51.594325+0000","last_became_active":"2024-04-26T03:38:49.511968+0000","last_became_peered":"2024-04-26T03:38:49.511968+0000","last_unstale":"2024-04-26T03:38:51.594325+0000","last_undegraded":"2024-04-26T03:38:51.594325+0000","last_fullsized":"2024-04-26T03:38:51.594325+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:51.699598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":671744,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":2,"num_write_kb":656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,14,10,9,1,2],"acting":[4,11,14,10,9,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":36,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T03:38:56.507917+0000","last_change":"2024-04-26T03:38:37.255450+0000","last_active":"2024-04-26T03:38:56.507917+0000","last_peered":"2024-04-26T03:38:56.507917+0000","last_clean":"2024-04-26T03:38:56.507917+0000","last_became_active":"2024-04-26T03:38:37.255181+0000","last_became_peered":"2024-04-26T03:38:37.255181+0000","last_unstale":"2024-04-26T03:38:56.507917+0000","last_undegraded":"2024-04-26T03:38:56.507917+0000","last_fullsized":"2024-04-26T03:38:56.507917+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:00.449388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0],"acting":[14,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.0","version":"35'9","reported_seq":36,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T03:38:52.488184+0000","last_change":"2024-04-26T03:38:49.513053+0000","last_active":"2024-04-26T03:38:52.488184+0000","last_peered":"2024-04-26T03:38:52.488184+0000","last_clean":"2024-04-26T03:38:52.488184+0000","last_became_active":"2024-04-26T03:38:49.511916+0000","last_became_peered":"2024-04-26T03:38:49.511916+0000","last_unstale":"2024-04-26T03:38:52.488184+0000","last_undegraded":"2024-04-26T03:38:52.488184+0000","last_fullsized":"2024-04-26T03:38:52.488184+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:17.528906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015808,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":9,"num_write_kb":1296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,9,15,2,6,14,4],"acting":[1,9,15,2,6,14,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T03:38:55.510599+0000","last_change":"2024-04-26T03:38:48.475569+0000","last_active":"2024-04-26T03:38:55.510599+0000","last_peered":"2024-04-26T03:38:55.510599+0000","last_clean":"2024-04-26T03:38:55.510599+0000","last_became_active":"2024-04-26T03:38:48.475155+0000","last_became_peered":"2024-04-26T03:38:48.475155+0000","last_unstale":"2024-04-26T03:38:55.510599+0000","last_undegraded":"2024-04-26T03:38:55.510599+0000","last_fullsized":"2024-04-26T03:38:55.510599+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:38.214686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,12],"acting":[11,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3","version":"35'14","reported_seq":66,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T03:38:56.516270+0000","last_change":"2024-04-26T03:38:56.517166+0000","last_active":"2024-04-26T03:38:56.516270+0000","last_peered":"2024-04-26T03:38:56.516270+0000","last_clean":"2024-04-26T03:38:56.516270+0000","last_became_active":"2024-04-26T03:38:49.486695+0000","last_became_peered":"2024-04-26T03:38:49.486695+0000","last_unstale":"2024-04-26T03:38:56.516270+0000","last_undegraded":"2024-04-26T03:38:56.516270+0000","last_fullsized":"2024-04-26T03:38:56.516270+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:10.614495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010931039999999999,"stat_sum":{"num_bytes":3555328,"num_objects":5,"num_object_clones":0,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":2359,"num_write":14,"num_write_kb":3472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,7,4,15,1,2,3],"acting":[10,7,4,15,1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"3","length":"2"}]},{"pgid":"3.5","version":"40'20","reported_seq":67,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T03:38:56.525112+0000","last_change":"2024-04-26T03:38:56.526205+0000","last_active":"2024-04-26T03:38:56.525112+0000","last_peered":"2024-04-26T03:38:56.525112+0000","last_clean":"2024-04-26T03:38:56.525112+0000","last_became_active":"2024-04-26T03:38:49.485980+0000","last_became_peered":"2024-04-26T03:38:49.485980+0000","last_unstale":"2024-04-26T03:38:56.525112+0000","last_undegraded":"2024-04-26T03:38:56.525112+0000","last_fullsized":"2024-04-26T03:38:56.525112+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:17.753522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014456504,"stat_sum":{"num_bytes":1458176,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":1109,"num_write":31,"num_write_kb":1424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,8,4,2,7,0,5],"acting":[15,8,4,2,7,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"3","length":"2"}]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-26T03:38:51.489737+0000","last_change":"2024-04-26T03:38:37.255389+0000","last_active":"2024-04-26T03:38:51.489737+0000","last_peered":"2024-04-26T03:38:51.489737+0000","last_clean":"2024-04-26T03:38:51.489737+0000","last_became_active":"2024-04-26T03:38:37.255066+0000","last_became_peered":"2024-04-26T03:38:37.255066+0000","last_unstale":"2024-04-26T03:38:51.489737+0000","last_undegraded":"2024-04-26T03:38:51.489737+0000","last_fullsized":"2024-04-26T03:38:51.489737+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:43.860480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"40'16","reported_seq":65,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T03:38:56.537103+0000","last_change":"2024-04-26T03:38:56.511211+0000","last_active":"2024-04-26T03:38:56.537103+0000","last_peered":"2024-04-26T03:38:56.537103+0000","last_clean":"2024-04-26T03:38:56.537103+0000","last_became_active":"2024-04-26T03:38:49.487567+0000","last_became_peered":"2024-04-26T03:38:49.487567+0000","last_unstale":"2024-04-26T03:38:56.537103+0000","last_undegraded":"2024-04-26T03:38:56.537103+0000","last_fullsized":"2024-04-26T03:38:56.537103+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:00.356120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075568599999999999,"stat_sum":{"num_bytes":2752512,"num_objects":5,"num_object_clones":1,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":12,"num_read_kb":6,"num_write":15,"num_write_kb":3536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,13,4,2,6,10,5],"acting":[8,13,4,2,6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"3","length":"2"}]},{"pgid":"2.5","version":"0'0","reported_seq":35,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T03:38:56.513386+0000","last_change":"2024-04-26T03:38:38.187714+0000","last_active":"2024-04-26T03:38:56.513386+0000","last_peered":"2024-04-26T03:38:56.513386+0000","last_clean":"2024-04-26T03:38:56.513386+0000","last_became_active":"2024-04-26T03:38:38.187383+0000","last_became_peered":"2024-04-26T03:38:38.187383+0000","last_unstale":"2024-04-26T03:38:56.513386+0000","last_undegraded":"2024-04-26T03:38:56.513386+0000","last_fullsized":"2024-04-26T03:38:56.513386+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:59.062623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4],"acting":[9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"34'2","reported_seq":24,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-26T03:38:51.640816+0000","last_change":"2024-04-26T03:38:49.488210+0000","last_active":"2024-04-26T03:38:51.640816+0000","last_peered":"2024-04-26T03:38:51.640816+0000","last_clean":"2024-04-26T03:38:51.640816+0000","last_became_active":"2024-04-26T03:38:49.487523+0000","last_became_peered":"2024-04-26T03:38:49.487523+0000","last_unstale":"2024-04-26T03:38:51.640816+0000","last_undegraded":"2024-04-26T03:38:51.640816+0000","last_fullsized":"2024-04-26T03:38:51.640816+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:51.051910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131072,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":2,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,13,6,0,14,10,11],"acting":[4,13,6,0,14,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":36,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T03:38:56.510340+0000","last_change":"2024-04-26T03:38:36.484782+0000","last_active":"2024-04-26T03:38:56.510340+0000","last_peered":"2024-04-26T03:38:56.510340+0000","last_clean":"2024-04-26T03:38:56.510340+0000","last_became_active":"2024-04-26T03:38:36.484388+0000","last_became_peered":"2024-04-26T03:38:36.484388+0000","last_unstale":"2024-04-26T03:38:56.510340+0000","last_undegraded":"2024-04-26T03:38:56.510340+0000","last_fullsized":"2024-04-26T03:38:56.510340+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:36.318287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,13],"acting":[8,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.6","version":"35'3","reported_seq":24,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-26T03:38:51.646353+0000","last_change":"2024-04-26T03:38:49.942015+0000","last_active":"2024-04-26T03:38:51.646353+0000","last_peered":"2024-04-26T03:38:51.646353+0000","last_clean":"2024-04-26T03:38:51.646353+0000","last_became_active":"2024-04-26T03:38:49.941208+0000","last_became_peered":"2024-04-26T03:38:49.941208+0000","last_unstale":"2024-04-26T03:38:51.646353+0000","last_undegraded":"2024-04-26T03:38:51.646353+0000","last_fullsized":"2024-04-26T03:38:51.646353+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:10.599503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":819200,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":3,"num_write_kb":800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,14,6,2,12,7,1],"acting":[0,14,6,2,12,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":36,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T03:38:56.511628+0000","last_change":"2024-04-26T03:38:37.359252+0000","last_active":"2024-04-26T03:38:56.511628+0000","last_peered":"2024-04-26T03:38:56.511628+0000","last_clean":"2024-04-26T03:38:56.511628+0000","last_became_active":"2024-04-26T03:38:37.358910+0000","last_became_peered":"2024-04-26T03:38:37.358910+0000","last_unstale":"2024-04-26T03:38:56.511628+0000","last_undegraded":"2024-04-26T03:38:56.511628+0000","last_fullsized":"2024-04-26T03:38:56.511628+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:04.186001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":28721152,"num_objects":50,"num_object_clones":6,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":164,"num_read_kb":17162,"num_write":225,"num_write_kb":30896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":50946048,"data_stored":54898688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":112,"acting":112,"num_store_stats":16},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":13852,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":24,"seq":103079215113,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33596,"kb_used_data":2412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338244,"statfs":{"total":96636764160,"available":96602361856,"internally_reserved":0,"allocated":2469888,"data_stored":3572454,"data_compressed":34152,"data_compressed_allocated":90112,"data_compressed_original":626688,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[4,6,7,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215113,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29880,"kb_used_data":2860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341960,"statfs":{"total":96636764160,"available":96606167040,"internally_reserved":0,"allocated":2928640,"data_stored":2762948,"data_compressed":25218,"data_compressed_allocated":53248,"data_compressed_original":106496,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":23,"seq":98784247818,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34500,"kb_used_data":3320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337340,"statfs":{"total":96636764160,"available":96601436160,"internally_reserved":0,"allocated":3399680,"data_stored":3236034,"data_compressed":23211,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280520,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31260,"kb_used_data":4240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340580,"statfs":{"total":96636764160,"available":96604753920,"internally_reserved":0,"allocated":4341760,"data_stored":4836244,"data_compressed":40167,"data_compressed_allocated":102400,"data_compressed_original":651264,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,5,7,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280520,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30660,"kb_used_data":3640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341180,"statfs":{"total":96636764160,"available":96605368320,"internally_reserved":0,"allocated":3727360,"data_stored":3920158,"data_compressed":31239,"data_compressed_allocated":65536,"data_compressed_original":131072,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,3,4,5,6,8,9,10,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":22,"seq":94489280521,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29896,"kb_used_data":2872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341944,"statfs":{"total":96636764160,"available":96606150656,"internally_reserved":0,"allocated":2940928,"data_stored":3262836,"data_compressed":33241,"data_compressed_allocated":69632,"data_compressed_original":139264,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[2,4,5,6,7,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280519,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30188,"kb_used_data":3164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341652,"statfs":{"total":96636764160,"available":96605851648,"internally_reserved":0,"allocated":3239936,"data_stored":3082612,"data_compressed":33241,"data_compressed_allocated":69632,"data_compressed_original":139264,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,2,4,7,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280520,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31972,"kb_used_data":4952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339868,"statfs":{"total":96636764160,"available":96604024832,"internally_reserved":0,"allocated":5070848,"data_stored":4899102,"data_compressed":31239,"data_compressed_allocated":65536,"data_compressed_original":131072,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[1,2,3,4,7,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313225,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31544,"kb_used_data":4524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340296,"statfs":{"total":96636764160,"available":96604463104,"internally_reserved":0,"allocated":4632576,"data_stored":4907294,"data_compressed":31239,"data_compressed_allocated":65536,"data_compressed_original":131072,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,5,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":12,"up_from":22,"seq":94489280519,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29060,"kb_used_data":2040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342780,"statfs":{"total":96636764160,"available":96607006720,"internally_reserved":0,"allocated":2088960,"data_stored":1933598,"data_compressed":31239,"data_compressed_allocated":65536,"data_compressed_original":131072,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,7,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280519,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31192,"kb_used_data":4168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340648,"statfs":{"total":96636764160,"available":96604823552,"internally_reserved":0,"allocated":4268032,"data_stored":4639092,"data_compressed":33241,"data_compressed_allocated":69632,"data_compressed_original":139264,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[2,4,6,7,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247816,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33444,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338396,"statfs":{"total":96636764160,"available":96602517504,"internally_reserved":0,"allocated":2318336,"data_stored":2154690,"data_compressed":23211,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,6,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280519,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31036,"kb_used_data":4012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340804,"statfs":{"total":96636764160,"available":96604983296,"internally_reserved":0,"allocated":4108288,"data_stored":4381044,"data_compressed":33241,"data_compressed_allocated":69632,"data_compressed_original":139264,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,3,5,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":23,"seq":98784247818,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35720,"kb_used_data":4536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336120,"statfs":{"total":96636764160,"available":96600186880,"internally_reserved":0,"allocated":4644864,"data_stored":4479172,"data_compressed":25218,"data_compressed_allocated":53248,"data_compressed_original":106496,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,4,6,7,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280519,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29948,"kb_used_data":2924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341892,"statfs":{"total":96636764160,"available":96606097408,"internally_reserved":0,"allocated":2994176,"data_stored":3262836,"data_compressed":33241,"data_compressed_allocated":69632,"data_compressed_original":139264,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,4,5,7,8,9,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247816,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30512,"kb_used_data":3492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341328,"statfs":{"total":96636764160,"available":96605519872,"internally_reserved":0,"allocated":3575808,"data_stored":3412162,"data_compressed":23211,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1966080,"data_stored":1966080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4280320,"data_stored":4280320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3223552,"data_stored":3223552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2564096,"data_stored":2564096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":3047424,"data_stored":3047424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2039808,"data_stored":2834432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3874816,"data_stored":4067328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3325952,"data_stored":3690496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":2527232,"data_stored":3022848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":2826240,"data_stored":2842624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":5222400,"data_stored":5414912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4231168,"data_stored":4677632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":1687552,"data_stored":1703936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":3854336,"data_stored":4399104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":3694592,"data_stored":4141056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":2580480,"data_stored":3022848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T03:38:58.144 INFO:tasks.ceph.ceph_manager.ceph:increase pool size by 10 2024-04-26T03:38:58.144 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set unique_pool_0 pg_num 26 2024-04-26T03:38:58.178 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:58.178 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:58.187 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-26T03:38:58.230 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-04-26T03:38:58.229+0000 7f745118d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 31821) UID: 0 2024-04-26T03:38:58.289 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:58.289 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:58.297 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-04-26T03:38:58.339 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-04-26T03:38:58.337+0000 7f745118d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 31821) UID: 0 2024-04-26T03:38:58.411 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-04-26T03:38:58.409+0000 7fdc2d21c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31819) UID: 0 2024-04-26T03:38:58.440 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:58.440 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:58.449 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-04-26T03:38:58.515 INFO:teuthology.orchestra.run.smithi028.stderr:set pool 3 pg_num to 26 2024-04-26T03:38:58.519 INFO:tasks.rados.rados.0.smithi092.stdout:110: done (2 left) 2024-04-26T03:38:58.519 INFO:tasks.rados.rados.0.smithi092.stdout:111: setattr oid 22 current snap is 4 2024-04-26T03:38:58.519 INFO:tasks.rados.rados.0.smithi092.stdout:107: finishing rollback tid 1 to smithi09235267-3 2024-04-26T03:38:58.519 INFO:tasks.rados.rados.0.smithi092.stdout:107: finishing rollback tid 2 to smithi09235267-3 2024-04-26T03:38:58.519 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 3 v 16 (ObjNum 2 snap 0 seq_num 2) dirty exists 2024-04-26T03:38:58.519 INFO:tasks.rados.rados.0.smithi092.stdout:108: finishing write tid 1 to smithi09235267-13 2024-04-26T03:38:58.519 INFO:tasks.rados.rados.0.smithi092.stdout:108: finishing write tid 2 to smithi09235267-13 2024-04-26T03:38:58.519 INFO:tasks.rados.rados.0.smithi092.stdout:108: finishing write tid 3 to smithi09235267-13 2024-04-26T03:38:58.520 INFO:tasks.rados.rados.0.smithi092.stdout:108: finishing write tid 4 to smithi09235267-13 2024-04-26T03:38:58.520 INFO:tasks.rados.rados.0.smithi092.stdout:108: oid 13 updating version 0 to 19 2024-04-26T03:38:58.520 INFO:tasks.rados.rados.0.smithi092.stdout:108: oid 13 version 19 is already newer than 18 2024-04-26T03:38:58.520 INFO:tasks.rados.rados.0.smithi092.stdout:108: oid 13 version 19 is already newer than 17 2024-04-26T03:38:58.520 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 13 v 19 (ObjNum 67 snap 4 seq_num 67) dirty exists 2024-04-26T03:38:58.520 INFO:tasks.rados.rados.0.smithi092.stdout:108: left oid 13 (ObjNum 67 snap 4 seq_num 67) 2024-04-26T03:38:58.520 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-04-26T03:38:58.517+0000 7fdc2d21c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31819) UID: 0 2024-04-26T03:38:58.521 INFO:tasks.rados.rados.0.smithi092.stdout:107: done (2 left) 2024-04-26T03:38:58.521 INFO:tasks.rados.rados.0.smithi092.stdout:108: done (1 left) 2024-04-26T03:38:58.521 INFO:tasks.rados.rados.0.smithi092.stdout:112: rollback oid 13 current snap is 4 2024-04-26T03:38:58.521 INFO:tasks.rados.rados.0.smithi092.stdout:rollback oid 13 to 3 2024-04-26T03:38:58.521 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 43 current snap is 4 2024-04-26T03:38:58.521 INFO:tasks.rados.rados.0.smithi092.stdout:113: seq_num 69 ranges {720896=786432,1507328=376832} 2024-04-26T03:38:58.528 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-26T03:38:58.529 INFO:tasks.rados.rados.0.smithi092.stdout:113: writing smithi09235267-43 from 720896 to 1507328 tid 1 2024-04-26T03:38:58.532 INFO:tasks.rados.rados.0.smithi092.stdout:113: writing smithi09235267-43 from 1507328 to 1884160 tid 2 2024-04-26T03:38:58.532 INFO:tasks.rados.rados.0.smithi092.stdout:114: copy_from oid 2 from oid 27 current snap is 4 2024-04-26T03:38:58.532 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 22 v 9 (ObjNum 21 snap 0 seq_num 21) dirty exists 2024-04-26T03:38:58.532 INFO:tasks.rados.rados.0.smithi092.stdout:112: finishing rollback tid 0 to smithi09235267-13 2024-04-26T03:38:58.532 INFO:tasks.rados.rados.0.smithi092.stdout:112: finishing rollback tid 1 to smithi09235267-13 2024-04-26T03:38:58.533 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 13 v 21 (ObjNum 33 snap 0 seq_num 2187377731) dirty dne 2024-04-26T03:38:58.533 INFO:tasks.rados.rados.0.smithi092.stdout:111: done (3 left) 2024-04-26T03:38:58.533 INFO:tasks.rados.rados.0.smithi092.stdout:112: done (2 left) 2024-04-26T03:38:58.533 INFO:tasks.rados.rados.0.smithi092.stdout:115: setattr oid 3 current snap is 4 2024-04-26T03:38:58.534 INFO:tasks.rados.rados.0.smithi092.stdout:116: copy_from oid 4 from oid 46 current snap is 4 2024-04-26T03:38:58.534 INFO:tasks.rados.rados.0.smithi092.stdout:117: read oid 25 snap -1 2024-04-26T03:38:58.534 INFO:tasks.rados.rados.0.smithi092.stdout:117: expect (ObjNum 24 snap 0 seq_num 24) 2024-04-26T03:38:58.534 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 11 current snap is 4 2024-04-26T03:38:58.534 INFO:tasks.rados.rados.0.smithi092.stdout:118: seq_num 70 ranges {0=802816,802816=376832} 2024-04-26T03:38:58.539 INFO:tasks.rados.rados.0.smithi092.stdout:118: writing smithi09235267-11 from 0 to 802816 tid 1 2024-04-26T03:38:58.542 INFO:tasks.rados.rados.0.smithi092.stdout:118: writing smithi09235267-11 from 802816 to 1179648 tid 2 2024-04-26T03:38:58.542 INFO:tasks.rados.rados.0.smithi092.stdout:119: delete oid 50 current snap is 4 2024-04-26T03:38:58.542 INFO:tasks.rados.rados.0.smithi092.stdout:114: finishing copy_from to smithi09235267-2 2024-04-26T03:38:58.542 INFO:tasks.rados.rados.0.smithi092.stdout:114: got expected ENOENT (src dne) 2024-04-26T03:38:58.542 INFO:tasks.rados.rados.0.smithi092.stdout:114: finishing copy_from racing read to smithi09235267-2 2024-04-26T03:38:58.543 INFO:tasks.rados.rados.0.smithi092.stdout:117: expect (ObjNum 24 snap 0 seq_num 24) 2024-04-26T03:38:58.557 INFO:tasks.rados.rados.0.smithi092.stdout:113: finishing write tid 1 to smithi09235267-43 2024-04-26T03:38:58.557 INFO:tasks.rados.rados.0.smithi092.stdout:113: finishing write tid 2 to smithi09235267-43 2024-04-26T03:38:58.557 INFO:tasks.rados.rados.0.smithi092.stdout:113: finishing write tid 3 to smithi09235267-43 2024-04-26T03:38:58.557 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 3 v 28 (ObjNum 2 snap 0 seq_num 2) dirty exists 2024-04-26T03:38:58.557 INFO:tasks.rados.rados.0.smithi092.stdout:113: finishing write tid 4 to smithi09235267-43 2024-04-26T03:38:58.557 INFO:tasks.rados.rados.0.smithi092.stdout:113: oid 43 updating version 0 to 26 2024-04-26T03:38:58.557 INFO:tasks.rados.rados.0.smithi092.stdout:113: oid 43 updating version 26 to 27 2024-04-26T03:38:58.557 INFO:tasks.rados.rados.0.smithi092.stdout:113: oid 43 version 27 is already newer than 25 2024-04-26T03:38:58.557 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 43 v 27 (ObjNum 69 snap 4 seq_num 69) dirty exists 2024-04-26T03:38:58.557 INFO:tasks.rados.rados.0.smithi092.stdout:113: left oid 43 (ObjNum 69 snap 4 seq_num 69) 2024-04-26T03:38:58.558 INFO:tasks.rados.rados.0.smithi092.stdout:113: done (6 left) 2024-04-26T03:38:58.558 INFO:tasks.rados.rados.0.smithi092.stdout:114: done (5 left) 2024-04-26T03:38:58.558 INFO:tasks.rados.rados.0.smithi092.stdout:115: done (4 left) 2024-04-26T03:38:58.558 INFO:tasks.rados.rados.0.smithi092.stdout:117: done (3 left) 2024-04-26T03:38:58.558 INFO:tasks.rados.rados.0.smithi092.stdout:119: done (2 left) 2024-04-26T03:38:58.558 INFO:tasks.rados.rados.0.smithi092.stdout:120: snap_create 2024-04-26T03:38:58.563 INFO:tasks.rados.rados.0.smithi092.stdout:116: finishing copy_from racing read to smithi09235267-4 2024-04-26T03:38:58.567 INFO:tasks.rados.rados.0.smithi092.stdout:118: finishing write tid 1 to smithi09235267-11 2024-04-26T03:38:58.570 INFO:tasks.rados.rados.0.smithi092.stdout:116: finishing copy_from to smithi09235267-4 2024-04-26T03:38:58.571 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 4 v 11 (ObjNum 56 snap 0 seq_num 56) dirty exists 2024-04-26T03:38:58.571 INFO:tasks.rados.rados.0.smithi092.stdout:118: finishing write tid 2 to smithi09235267-11 2024-04-26T03:38:58.571 INFO:tasks.rados.rados.0.smithi092.stdout:118: finishing write tid 3 to smithi09235267-11 2024-04-26T03:38:58.571 INFO:tasks.rados.rados.0.smithi092.stdout:118: finishing write tid 4 to smithi09235267-11 2024-04-26T03:38:58.571 INFO:tasks.rados.rados.0.smithi092.stdout:118: oid 11 updating version 0 to 18 2024-04-26T03:38:58.571 INFO:tasks.rados.rados.0.smithi092.stdout:118: oid 11 updating version 18 to 19 2024-04-26T03:38:58.572 INFO:tasks.rados.rados.0.smithi092.stdout:118: oid 11 version 19 is already newer than 15 2024-04-26T03:38:58.572 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 11 v 19 (ObjNum 70 snap 4 seq_num 70) dirty exists 2024-04-26T03:38:58.572 INFO:tasks.rados.rados.0.smithi092.stdout:118: left oid 11 (ObjNum 70 snap 4 seq_num 70) 2024-04-26T03:38:58.609 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:58.610 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:58.613 INFO:tasks.ceph.osd.8.smithi028.stderr:2024-04-26T03:38:58.611+0000 7f5bb75d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 31968) UID: 0 2024-04-26T03:38:58.618 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-04-26T03:38:58.714 INFO:tasks.ceph.osd.8.smithi028.stderr:2024-04-26T03:38:58.712+0000 7f5bb75d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 31968) UID: 0 2024-04-26T03:38:58.760 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:58.761 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:58.769 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-04-26T03:38:58.815 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:38:58.813+0000 7fba0d2ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31961) UID: 0 2024-04-26T03:38:58.850 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-26T03:38:58.872 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:58.872 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:58.880 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-04-26T03:38:58.916 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-04-26T03:38:58.915+0000 7fee562fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31759) UID: 0 2024-04-26T03:38:58.975 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0: {} 2024-04-26T03:38:58.976 INFO:teuthology.orchestra.run.smithi028.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T03:38:58.983 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:58.983 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:58.992 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-04-26T03:38:58.994 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1: {} 2024-04-26T03:38:58.995 INFO:teuthology.orchestra.run.smithi028.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T03:38:59.013 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2: {} 2024-04-26T03:38:59.013 INFO:teuthology.orchestra.run.smithi028.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T03:38:59.017 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:38:59.016+0000 7fb78c544640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31956) UID: 0 2024-04-26T03:38:59.031 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3: {} 2024-04-26T03:38:59.031 INFO:teuthology.orchestra.run.smithi028.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T03:38:59.048 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4: {} 2024-04-26T03:38:59.048 INFO:teuthology.orchestra.run.smithi028.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T03:38:59.066 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5: {} 2024-04-26T03:38:59.066 INFO:teuthology.orchestra.run.smithi028.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T03:38:59.084 INFO:teuthology.orchestra.run.smithi028.stdout:osd.6: {} 2024-04-26T03:38:59.084 INFO:teuthology.orchestra.run.smithi028.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T03:38:59.094 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:38:59.094 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:38:59.102 INFO:teuthology.orchestra.run.smithi028.stdout:osd.7: {} 2024-04-26T03:38:59.102 INFO:teuthology.orchestra.run.smithi028.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T03:38:59.103 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-26T03:38:59.119 INFO:teuthology.orchestra.run.smithi028.stdout:osd.8: {} 2024-04-26T03:38:59.119 INFO:teuthology.orchestra.run.smithi028.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T03:38:59.119 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-04-26T03:38:59.117+0000 7fdc2d21c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31819) UID: 0 2024-04-26T03:38:59.137 INFO:teuthology.orchestra.run.smithi028.stdout:osd.9: {} 2024-04-26T03:38:59.137 INFO:teuthology.orchestra.run.smithi028.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T03:38:59.155 INFO:teuthology.orchestra.run.smithi028.stdout:osd.10: {} 2024-04-26T03:38:59.155 INFO:teuthology.orchestra.run.smithi028.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T03:38:59.174 INFO:teuthology.orchestra.run.smithi028.stdout:osd.11: {} 2024-04-26T03:38:59.174 INFO:teuthology.orchestra.run.smithi028.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T03:38:59.190 INFO:teuthology.orchestra.run.smithi028.stdout:osd.12: {} 2024-04-26T03:38:59.190 INFO:teuthology.orchestra.run.smithi028.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T03:38:59.209 INFO:teuthology.orchestra.run.smithi028.stdout:osd.13: {} 2024-04-26T03:38:59.210 INFO:teuthology.orchestra.run.smithi028.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T03:38:59.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:38:59.207+0000 7fe4c37fe640 0 -- 172.21.15.28:0/3409890719 >> [v2:172.21.15.66:6824/1534905602,v1:172.21.15.66:6825/1534905602] conn(0x7fe4c4220080 msgr2=0x7fe4c42224b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-26T03:38:59.211 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:38:59.211 INFO:teuthology.orchestra.run.smithi028.stdout: "ops": [], 2024-04-26T03:38:59.211 INFO:teuthology.orchestra.run.smithi028.stdout: "num_ops": 0 2024-04-26T03:38:59.211 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:38:59.220 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-04-26T03:38:59.219+0000 7fee562fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31759) UID: 0 2024-04-26T03:38:59.221 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-26T03:38:59.321 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-04-26T03:38:59.320+0000 7f745118d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 31821) UID: 0 2024-04-26T03:38:59.369 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:38:59.369 INFO:teuthology.orchestra.run.smithi028.stdout: "ops": [], 2024-04-26T03:38:59.369 INFO:teuthology.orchestra.run.smithi028.stdout: "complaint_time": 30, 2024-04-26T03:38:59.369 INFO:teuthology.orchestra.run.smithi028.stdout: "num_blocked_ops": 0 2024-04-26T03:38:59.370 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:38:59.377 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-26T03:38:59.422 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-04-26T03:38:59.420+0000 7fdc2d21c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31819) UID: 0 2024-04-26T03:38:59.445 INFO:teuthology.orchestra.run.smithi028.stdout:osd.14: {} 2024-04-26T03:38:59.446 INFO:teuthology.orchestra.run.smithi028.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T03:38:59.479 INFO:teuthology.orchestra.run.smithi028.stdout:osd.15: {} 2024-04-26T03:38:59.480 INFO:teuthology.orchestra.run.smithi028.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T03:38:59.515 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:38:59.515 INFO:teuthology.orchestra.run.smithi028.stdout: "size": 20, 2024-04-26T03:38:59.515 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 600, 2024-04-26T03:38:59.516 INFO:teuthology.orchestra.run.smithi028.stdout: "ops": [ 2024-04-26T03:38:59.516 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.516 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.8s1 34/32 ECSubWrite(tid=157, reqid=client.4556.0:4, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:38:59.516 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.481665+0000", 2024-04-26T03:38:59.516 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 8.0311826279999998, 2024-04-26T03:38:59.516 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0036141789999999999, 2024-04-26T03:38:59.516 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:38:59.516 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:38:59.516 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:38:59.516 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:38:59.516 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.516 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:38:59.517 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.481665+0000", 2024-04-26T03:38:59.517 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:38:59.517 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.517 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.517 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:38:59.517 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.481665+0000", 2024-04-26T03:38:59.517 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:38:59.517 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.517 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.517 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:38:59.517 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.481666+0000", 2024-04-26T03:38:59.517 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.6899999999999999e-07 2024-04-26T03:38:59.517 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.518 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.518 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:38:59.518 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.481697+0000", 2024-04-26T03:38:59.518 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.1278000000000001e-05 2024-04-26T03:38:59.518 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.518 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.518 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:38:59.518 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.481709+0000", 2024-04-26T03:38:59.518 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.2148e-05 2024-04-26T03:38:59.518 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.518 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.519 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:38:59.519 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.481717+0000", 2024-04-26T03:38:59.519 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.1210000000000007e-06 2024-04-26T03:38:59.519 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.519 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.519 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:38:59.519 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.481796+0000", 2024-04-26T03:38:59.519 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.8555999999999998e-05 2024-04-26T03:38:59.519 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.519 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.519 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:38:59.519 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.481808+0000", 2024-04-26T03:38:59.520 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.2463000000000001e-05 2024-04-26T03:38:59.520 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.520 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.520 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:38:59.520 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.485221+0000", 2024-04-26T03:38:59.520 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0034129099999999999 2024-04-26T03:38:59.520 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.520 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.520 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:38:59.520 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.485279+0000", 2024-04-26T03:38:59.520 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.8434000000000003e-05 2024-04-26T03:38:59.520 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:59.521 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:59.521 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:59.521 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.521 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.521 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.bs1 34/32 ECSubWrite(tid=0, reqid=client.4556.0:7, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:38:59.521 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.482567+0000", 2024-04-26T03:38:59.521 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 8.0302809719999999, 2024-04-26T03:38:59.521 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0033302169999999999, 2024-04-26T03:38:59.521 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:38:59.521 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:38:59.521 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:38:59.522 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:38:59.522 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.522 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:38:59.522 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.482567+0000", 2024-04-26T03:38:59.522 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:38:59.522 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.522 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.522 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:38:59.522 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.482567+0000", 2024-04-26T03:38:59.522 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:38:59.522 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.523 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.523 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:38:59.523 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.482567+0000", 2024-04-26T03:38:59.523 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.3500000000000002e-07 2024-04-26T03:38:59.523 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.523 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.523 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:38:59.523 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.482648+0000", 2024-04-26T03:38:59.523 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.0710999999999993e-05 2024-04-26T03:38:59.523 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.523 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.523 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:38:59.524 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.482661+0000", 2024-04-26T03:38:59.524 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.3471e-05 2024-04-26T03:38:59.524 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.524 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.524 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:38:59.524 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.482672+0000", 2024-04-26T03:38:59.524 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0149000000000001e-05 2024-04-26T03:38:59.524 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.524 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.524 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:38:59.525 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.482742+0000", 2024-04-26T03:38:59.525 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.0506000000000006e-05 2024-04-26T03:38:59.525 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.525 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.525 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:38:59.525 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.482752+0000", 2024-04-26T03:38:59.525 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.798e-06 2024-04-26T03:38:59.525 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.525 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.525 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:38:59.525 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.485857+0000", 2024-04-26T03:38:59.525 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0031046789999999999 2024-04-26T03:38:59.526 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.526 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.526 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:38:59.526 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.485897+0000", 2024-04-26T03:38:59.526 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.0568000000000002e-05 2024-04-26T03:38:59.526 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:59.526 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:59.526 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:59.526 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.526 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.526 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.cs2 34/32 ECSubWrite(tid=0, reqid=client.4556.0:10, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:38:59.526 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.482653+0000", 2024-04-26T03:38:59.527 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 8.0301946659999999, 2024-04-26T03:38:59.527 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0046127659999999999, 2024-04-26T03:38:59.527 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:38:59.527 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:38:59.527 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:38:59.527 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:38:59.527 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.527 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:38:59.527 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.482653+0000", 2024-04-26T03:38:59.527 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:38:59.527 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.528 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.528 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:38:59.528 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.482653+0000", 2024-04-26T03:38:59.528 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:38:59.528 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.528 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.528 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:38:59.528 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.482653+0000", 2024-04-26T03:38:59.528 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.48e-07 2024-04-26T03:38:59.528 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.528 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.528 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:38:59.529 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.482701+0000", 2024-04-26T03:38:59.529 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.7145999999999999e-05 2024-04-26T03:38:59.529 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.529 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.529 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:38:59.529 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.482715+0000", 2024-04-26T03:38:59.529 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.4184e-05 2024-04-26T03:38:59.529 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.529 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.529 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:38:59.529 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.482725+0000", 2024-04-26T03:38:59.529 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0696999999999999e-05 2024-04-26T03:38:59.529 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.530 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.530 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:38:59.530 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.483034+0000", 2024-04-26T03:38:59.530 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00030876200000000002 2024-04-26T03:38:59.530 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.530 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.530 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:38:59.530 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.483129+0000", 2024-04-26T03:38:59.530 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.4938000000000005e-05 2024-04-26T03:38:59.530 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.530 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.530 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:38:59.531 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.487207+0000", 2024-04-26T03:38:59.531 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0040782609999999997 2024-04-26T03:38:59.531 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.531 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.531 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:38:59.531 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.487266+0000", 2024-04-26T03:38:59.531 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.8529999999999997e-05 2024-04-26T03:38:59.531 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:59.531 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:59.531 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:59.531 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.532 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.532 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.cs2 34/32 ECSubWrite(tid=1, reqid=client.4556.0:11, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:38:59.532 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.484629+0000", 2024-04-26T03:38:59.532 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 8.0282189099999997, 2024-04-26T03:38:59.532 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0030236640000000001, 2024-04-26T03:38:59.532 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:38:59.532 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:38:59.532 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:38:59.532 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:38:59.532 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.532 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:38:59.532 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.484629+0000", 2024-04-26T03:38:59.533 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:38:59.533 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.533 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.533 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:38:59.533 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.484629+0000", 2024-04-26T03:38:59.533 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:38:59.533 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.533 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.533 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:38:59.533 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.484630+0000", 2024-04-26T03:38:59.533 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.4200000000000005e-07 2024-04-26T03:38:59.533 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.534 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.534 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:38:59.534 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.484653+0000", 2024-04-26T03:38:59.534 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.2796e-05 2024-04-26T03:38:59.534 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.534 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.534 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:38:59.534 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.484666+0000", 2024-04-26T03:38:59.534 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.3679000000000001e-05 2024-04-26T03:38:59.534 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.535 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.535 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:38:59.535 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.484675+0000", 2024-04-26T03:38:59.535 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.5890000000000003e-06 2024-04-26T03:38:59.535 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.535 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.535 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:38:59.535 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.486646+0000", 2024-04-26T03:38:59.535 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0019716040000000001 2024-04-26T03:38:59.535 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.535 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.535 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:38:59.536 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.486658+0000", 2024-04-26T03:38:59.536 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1345000000000001e-05 2024-04-26T03:38:59.536 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.536 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.537 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:38:59.537 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.487606+0000", 2024-04-26T03:38:59.537 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00094857700000000004 2024-04-26T03:38:59.537 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.537 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.537 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:38:59.537 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.487653+0000", 2024-04-26T03:38:59.537 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.6232000000000001e-05 2024-04-26T03:38:59.537 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:59.537 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:59.537 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:59.537 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.538 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.538 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.5s5 34/32 ECSubWrite(tid=0, reqid=client.4556.0:28, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:38:59.538 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.490515+0000", 2024-04-26T03:38:59.538 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 8.0223332860000003, 2024-04-26T03:38:59.538 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0027919870000000001, 2024-04-26T03:38:59.538 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:38:59.538 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:38:59.538 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:38:59.538 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:38:59.538 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.538 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:38:59.538 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.490515+0000", 2024-04-26T03:38:59.538 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:38:59.538 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.539 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.539 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:38:59.539 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.490515+0000", 2024-04-26T03:38:59.539 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:38:59.539 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.539 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.539 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:38:59.539 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.490515+0000", 2024-04-26T03:38:59.539 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.34e-07 2024-04-26T03:38:59.539 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.539 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.540 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:38:59.540 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.490801+0000", 2024-04-26T03:38:59.540 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00028655599999999999 2024-04-26T03:38:59.540 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.540 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.540 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:38:59.540 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.490813+0000", 2024-04-26T03:38:59.540 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1052e-05 2024-04-26T03:38:59.540 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.540 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.540 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:38:59.540 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.490832+0000", 2024-04-26T03:38:59.540 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.9208e-05 2024-04-26T03:38:59.541 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.541 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.541 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:38:59.541 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.490920+0000", 2024-04-26T03:38:59.541 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.8121000000000006e-05 2024-04-26T03:38:59.541 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.541 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.541 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:38:59.541 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.490930+0000", 2024-04-26T03:38:59.541 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0059e-05 2024-04-26T03:38:59.541 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.542 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.542 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:38:59.542 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493273+0000", 2024-04-26T03:38:59.542 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0023435460000000002 2024-04-26T03:38:59.542 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.542 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.542 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:38:59.542 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493307+0000", 2024-04-26T03:38:59.542 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.3111000000000002e-05 2024-04-26T03:38:59.542 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:59.542 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:59.542 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:59.542 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.542 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.543 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.fs6 34/32 ECSubWrite(tid=2, reqid=client.4556.0:25, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:38:59.543 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.494334+0000", 2024-04-26T03:38:59.543 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 8.0185136979999996, 2024-04-26T03:38:59.543 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0054451489999999998, 2024-04-26T03:38:59.543 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:38:59.543 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:38:59.543 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:38:59.543 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:38:59.543 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.543 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:38:59.543 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.494334+0000", 2024-04-26T03:38:59.543 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:38:59.543 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.543 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.543 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:38:59.544 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.494334+0000", 2024-04-26T03:38:59.544 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:38:59.544 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.544 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.544 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:38:59.544 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.494335+0000", 2024-04-26T03:38:59.544 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.5999999999999999e-07 2024-04-26T03:38:59.544 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.545 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.545 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:38:59.545 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.494376+0000", 2024-04-26T03:38:59.545 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.1412000000000003e-05 2024-04-26T03:38:59.545 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.545 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.545 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:38:59.545 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.494390+0000", 2024-04-26T03:38:59.545 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.3658999999999999e-05 2024-04-26T03:38:59.546 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.546 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.546 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:38:59.546 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.494400+0000", 2024-04-26T03:38:59.546 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0472000000000001e-05 2024-04-26T03:38:59.546 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.546 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.546 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:38:59.546 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.494489+0000", 2024-04-26T03:38:59.546 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.9083000000000001e-05 2024-04-26T03:38:59.546 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.546 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.546 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:38:59.546 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.494499+0000", 2024-04-26T03:38:59.547 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0227999999999999e-05 2024-04-26T03:38:59.547 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.547 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.547 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:38:59.547 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.499741+0000", 2024-04-26T03:38:59.547 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0052412070000000003 2024-04-26T03:38:59.547 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.547 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.547 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:38:59.547 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.499779+0000", 2024-04-26T03:38:59.547 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.8727999999999999e-05 2024-04-26T03:38:59.547 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:59.547 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:59.548 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:59.548 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.548 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.548 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.fs6 34/32 ECSubWrite(tid=3, reqid=client.4556.0:26, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:38:59.548 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.495620+0000", 2024-04-26T03:38:59.548 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 8.0172281129999998, 2024-04-26T03:38:59.548 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0043293760000000002, 2024-04-26T03:38:59.548 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:38:59.548 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:38:59.548 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:38:59.548 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:38:59.548 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.548 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:38:59.548 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495620+0000", 2024-04-26T03:38:59.548 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:38:59.549 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.549 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.549 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:38:59.549 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495620+0000", 2024-04-26T03:38:59.549 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:38:59.549 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.549 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.549 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:38:59.549 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495620+0000", 2024-04-26T03:38:59.549 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.0299999999999999e-07 2024-04-26T03:38:59.549 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.549 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.549 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:38:59.550 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495638+0000", 2024-04-26T03:38:59.550 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.7638999999999998e-05 2024-04-26T03:38:59.550 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.550 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.550 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:38:59.550 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495649+0000", 2024-04-26T03:38:59.550 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1582e-05 2024-04-26T03:38:59.550 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.550 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.550 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:38:59.551 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495658+0000", 2024-04-26T03:38:59.551 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.8079999999999994e-06 2024-04-26T03:38:59.551 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.551 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.551 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:38:59.551 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.497340+0000", 2024-04-26T03:38:59.551 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.001682123 2024-04-26T03:38:59.551 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.551 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.551 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:38:59.551 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.497348+0000", 2024-04-26T03:38:59.551 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.9610000000000005e-06 2024-04-26T03:38:59.552 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.552 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.552 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:38:59.552 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.499786+0000", 2024-04-26T03:38:59.552 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0024377439999999999 2024-04-26T03:38:59.552 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.552 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.552 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:38:59.552 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.499949+0000", 2024-04-26T03:38:59.552 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00016301600000000001 2024-04-26T03:38:59.552 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:59.552 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:59.552 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:59.553 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.553 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.553 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.cs2 35/32 ECSubWrite(tid=4, reqid=client.4556.0:31, at_version=35'3, trim_to=0'0, roll_forward_to=34'1))", 2024-04-26T03:38:59.553 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.495683+0000", 2024-04-26T03:38:59.553 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 8.0171643919999998, 2024-04-26T03:38:59.553 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0044340489999999998, 2024-04-26T03:38:59.553 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:38:59.553 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:38:59.553 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:38:59.553 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:38:59.554 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.554 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:38:59.554 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495683+0000", 2024-04-26T03:38:59.554 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:38:59.558 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.558 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.558 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:38:59.558 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495683+0000", 2024-04-26T03:38:59.558 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:38:59.559 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.559 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.559 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:38:59.559 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495684+0000", 2024-04-26T03:38:59.559 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.4200000000000001e-07 2024-04-26T03:38:59.559 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.559 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.560 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:38:59.560 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495769+0000", 2024-04-26T03:38:59.560 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.5544000000000006e-05 2024-04-26T03:38:59.560 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.560 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.560 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:38:59.560 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495780+0000", 2024-04-26T03:38:59.560 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.006e-05 2024-04-26T03:38:59.560 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.561 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.561 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:38:59.561 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495788+0000", 2024-04-26T03:38:59.561 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.157e-06 2024-04-26T03:38:59.561 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.561 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.561 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:38:59.561 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495848+0000", 2024-04-26T03:38:59.561 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.0044000000000003e-05 2024-04-26T03:38:59.561 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.561 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.561 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:38:59.561 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495857+0000", 2024-04-26T03:38:59.561 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.6369999999999994e-06 2024-04-26T03:38:59.562 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.562 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.562 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:38:59.562 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.500093+0000", 2024-04-26T03:38:59.562 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0042353470000000004 2024-04-26T03:38:59.562 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.562 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.562 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:38:59.562 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.500118+0000", 2024-04-26T03:38:59.562 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.4817999999999999e-05 2024-04-26T03:38:59.562 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:59.562 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:59.562 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:59.562 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.563 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.563 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.bs1 35/32 ECSubWrite(tid=6, reqid=client.4556.0:37, at_version=35'3, trim_to=0'0, roll_forward_to=34'2))", 2024-04-26T03:38:59.563 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.502006+0000", 2024-04-26T03:38:59.563 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 8.0108423690000006, 2024-04-26T03:38:59.563 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0036371519999999998, 2024-04-26T03:38:59.563 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:38:59.563 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:38:59.563 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:38:59.563 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:38:59.563 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.563 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:38:59.563 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.502006+0000", 2024-04-26T03:38:59.563 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:38:59.563 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.563 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.564 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:38:59.564 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.502006+0000", 2024-04-26T03:38:59.564 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:38:59.564 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.564 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.564 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:38:59.564 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.502006+0000", 2024-04-26T03:38:59.564 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.1800000000000005e-07 2024-04-26T03:38:59.564 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.564 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.564 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:38:59.564 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.502051+0000", 2024-04-26T03:38:59.564 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.5151000000000002e-05 2024-04-26T03:38:59.564 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.564 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.565 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:38:59.565 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.502064+0000", 2024-04-26T03:38:59.565 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.2247e-05 2024-04-26T03:38:59.565 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.565 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.565 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:38:59.565 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.502072+0000", 2024-04-26T03:38:59.565 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.6600000000000001e-06 2024-04-26T03:38:59.565 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.565 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.565 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:38:59.565 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.502369+0000", 2024-04-26T03:38:59.565 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00029663299999999998 2024-04-26T03:38:59.566 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.566 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.566 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:38:59.566 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.502379+0000", 2024-04-26T03:38:59.566 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.6879999999999994e-06 2024-04-26T03:38:59.566 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.566 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.566 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:38:59.566 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.505603+0000", 2024-04-26T03:38:59.566 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0032241100000000001 2024-04-26T03:38:59.566 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.566 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.566 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:38:59.567 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.505643+0000", 2024-04-26T03:38:59.567 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.9845000000000002e-05 2024-04-26T03:38:59.567 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:59.567 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:59.567 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:59.567 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.567 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.567 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.bs1 35/32 ECSubWrite(tid=9, reqid=client.4556.0:38, at_version=35'4, trim_to=0'0, roll_forward_to=34'2))", 2024-04-26T03:38:59.567 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.506640+0000", 2024-04-26T03:38:59.567 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 8.0062080949999999, 2024-04-26T03:38:59.568 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.004235122, 2024-04-26T03:38:59.568 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:38:59.568 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:38:59.568 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:38:59.568 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:38:59.568 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.568 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:38:59.568 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.506640+0000", 2024-04-26T03:38:59.568 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:38:59.568 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.568 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.568 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:38:59.568 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.506640+0000", 2024-04-26T03:38:59.568 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:38:59.569 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.569 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.569 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:38:59.569 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.506641+0000", 2024-04-26T03:38:59.569 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.2720000000000001e-06 2024-04-26T03:38:59.569 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.569 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.569 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:38:59.569 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.506661+0000", 2024-04-26T03:38:59.569 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.0217999999999999e-05 2024-04-26T03:38:59.569 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.569 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.570 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:38:59.570 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.506681+0000", 2024-04-26T03:38:59.570 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.9548e-05 2024-04-26T03:38:59.570 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.570 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.570 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:38:59.570 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.506691+0000", 2024-04-26T03:38:59.571 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.9860000000000006e-06 2024-04-26T03:38:59.571 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.571 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.571 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:38:59.571 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.506753+0000", 2024-04-26T03:38:59.571 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.1752000000000001e-05 2024-04-26T03:38:59.572 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.572 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.572 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:38:59.572 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.506762+0000", 2024-04-26T03:38:59.572 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.6090000000000007e-06 2024-04-26T03:38:59.572 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.572 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.573 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:38:59.573 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.510834+0000", 2024-04-26T03:38:59.573 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0040720979999999997 2024-04-26T03:38:59.573 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.573 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.573 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:38:59.574 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.510875+0000", 2024-04-26T03:38:59.574 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.0639000000000002e-05 2024-04-26T03:38:59.574 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:59.574 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:59.574 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:59.574 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.575 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.575 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.8s1 35/32 ECSubWrite(tid=161, reqid=client.4556.0:55, at_version=35'3, trim_to=0'0, roll_forward_to=34'2))", 2024-04-26T03:38:59.575 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.509448+0000", 2024-04-26T03:38:59.575 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 8.0033998410000002, 2024-04-26T03:38:59.575 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0040386969999999999, 2024-04-26T03:38:59.575 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:38:59.576 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:38:59.576 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:38:59.576 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:38:59.576 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.576 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:38:59.576 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.509448+0000", 2024-04-26T03:38:59.576 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:38:59.577 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.577 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.577 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:38:59.577 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.509448+0000", 2024-04-26T03:38:59.577 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:38:59.577 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.578 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.578 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:38:59.578 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.509449+0000", 2024-04-26T03:38:59.578 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.9299999999999996e-07 2024-04-26T03:38:59.578 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.578 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.579 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:38:59.579 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.509648+0000", 2024-04-26T03:38:59.579 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00019865700000000001 2024-04-26T03:38:59.579 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.579 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.579 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:38:59.580 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.509661+0000", 2024-04-26T03:38:59.580 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.3886999999999999e-05 2024-04-26T03:38:59.580 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.580 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.580 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:38:59.580 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.509670+0000", 2024-04-26T03:38:59.580 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.0010000000000007e-06 2024-04-26T03:38:59.581 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.581 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.581 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:38:59.581 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.509740+0000", 2024-04-26T03:38:59.581 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.9215999999999996e-05 2024-04-26T03:38:59.581 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.582 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.582 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:38:59.582 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.509747+0000", 2024-04-26T03:38:59.582 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.5730000000000002e-06 2024-04-26T03:38:59.582 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.582 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.582 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:38:59.583 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.513452+0000", 2024-04-26T03:38:59.583 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0037046919999999999 2024-04-26T03:38:59.583 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.583 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.583 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:38:59.583 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.513487+0000", 2024-04-26T03:38:59.584 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.4777999999999998e-05 2024-04-26T03:38:59.584 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:59.584 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:59.584 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:59.584 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.584 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.585 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.bs1 35/32 ECSubWrite(tid=10, reqid=client.4556.0:61, at_version=35'5, trim_to=0'0, roll_forward_to=35'3))", 2024-04-26T03:38:59.585 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.514843+0000", 2024-04-26T03:38:59.585 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 7.9980050340000002, 2024-04-26T03:38:59.585 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.004073587, 2024-04-26T03:38:59.585 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:38:59.585 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:38:59.586 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:38:59.586 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:38:59.586 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.586 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:38:59.586 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.514843+0000", 2024-04-26T03:38:59.586 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:38:59.587 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.587 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.587 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:38:59.587 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.514843+0000", 2024-04-26T03:38:59.587 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:38:59.587 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.588 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.588 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:38:59.588 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.514844+0000", 2024-04-26T03:38:59.588 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.046e-06 2024-04-26T03:38:59.588 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.588 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.589 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:38:59.589 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.514981+0000", 2024-04-26T03:38:59.589 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000137403 2024-04-26T03:38:59.589 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.589 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.589 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:38:59.590 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.515000+0000", 2024-04-26T03:38:59.590 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.8238999999999999e-05 2024-04-26T03:38:59.590 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.590 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.590 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:38:59.591 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.515010+0000", 2024-04-26T03:38:59.591 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0349999999999999e-05 2024-04-26T03:38:59.591 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.591 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.591 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:38:59.591 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.515087+0000", 2024-04-26T03:38:59.591 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.7537e-05 2024-04-26T03:38:59.592 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.592 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.592 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:38:59.592 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.515097+0000", 2024-04-26T03:38:59.592 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.5619999999999993e-06 2024-04-26T03:38:59.592 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.593 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.593 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:38:59.593 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.518878+0000", 2024-04-26T03:38:59.593 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.003781422 2024-04-26T03:38:59.593 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.593 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.594 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:38:59.594 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.518916+0000", 2024-04-26T03:38:59.594 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.8028000000000003e-05 2024-04-26T03:38:59.594 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:59.594 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:59.594 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:59.594 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.594 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.595 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.5s5 35/32 ECSubWrite(tid=4, reqid=client.4556.0:67, at_version=35'3, trim_to=0'0, roll_forward_to=34'2))", 2024-04-26T03:38:59.595 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.519606+0000", 2024-04-26T03:38:59.595 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 7.9932419770000003, 2024-04-26T03:38:59.595 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0049051119999999997, 2024-04-26T03:38:59.595 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:38:59.595 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:38:59.595 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:38:59.595 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:38:59.596 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.596 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:38:59.596 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.519606+0000", 2024-04-26T03:38:59.596 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:38:59.596 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.596 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.596 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:38:59.596 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.519606+0000", 2024-04-26T03:38:59.596 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:38:59.597 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.597 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.597 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:38:59.597 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.519607+0000", 2024-04-26T03:38:59.597 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.7700000000000004e-07 2024-04-26T03:38:59.597 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.597 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.597 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:38:59.597 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.519905+0000", 2024-04-26T03:38:59.598 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00029801900000000001 2024-04-26T03:38:59.598 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.598 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.598 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:38:59.598 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.519919+0000", 2024-04-26T03:38:59.598 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.3823e-05 2024-04-26T03:38:59.599 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.599 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.599 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:38:59.599 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.519927+0000", 2024-04-26T03:38:59.599 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.473e-06 2024-04-26T03:38:59.599 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.599 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.599 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:38:59.599 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.520040+0000", 2024-04-26T03:38:59.599 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000112716 2024-04-26T03:38:59.599 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.599 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.599 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:38:59.600 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.520048+0000", 2024-04-26T03:38:59.600 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.3899999999999993e-06 2024-04-26T03:38:59.600 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.600 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.600 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:38:59.600 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.524478+0000", 2024-04-26T03:38:59.600 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0044294069999999998 2024-04-26T03:38:59.600 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.600 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.600 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:38:59.600 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.524511+0000", 2024-04-26T03:38:59.600 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.3507000000000003e-05 2024-04-26T03:38:59.600 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:59.600 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:59.601 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:59.601 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.601 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.601 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.8s1 35/32 ECSubWrite(tid=165, reqid=client.4556.0:91, at_version=35'5, trim_to=0'0, roll_forward_to=35'4))", 2024-04-26T03:38:59.601 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.548957+0000", 2024-04-26T03:38:59.601 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 7.9638909509999998, 2024-04-26T03:38:59.601 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0045428129999999997, 2024-04-26T03:38:59.601 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:38:59.601 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:38:59.601 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:38:59.601 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:38:59.601 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.601 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:38:59.601 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.548957+0000", 2024-04-26T03:38:59.602 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:38:59.602 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.602 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.602 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:38:59.602 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.548957+0000", 2024-04-26T03:38:59.602 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:38:59.602 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.602 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.602 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:38:59.602 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.548958+0000", 2024-04-26T03:38:59.602 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.274e-06 2024-04-26T03:38:59.602 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.602 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.602 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:38:59.603 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.549325+0000", 2024-04-26T03:38:59.603 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00036678600000000002 2024-04-26T03:38:59.603 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.603 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.603 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:38:59.603 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.549346+0000", 2024-04-26T03:38:59.603 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.1129e-05 2024-04-26T03:38:59.603 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.603 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.603 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:38:59.603 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.549357+0000", 2024-04-26T03:38:59.604 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.136e-05 2024-04-26T03:38:59.604 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.604 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.604 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:38:59.604 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.549427+0000", 2024-04-26T03:38:59.604 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.9187e-05 2024-04-26T03:38:59.605 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.605 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.605 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:38:59.605 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.549437+0000", 2024-04-26T03:38:59.605 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0526999999999999e-05 2024-04-26T03:38:59.605 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.605 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.605 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:38:59.605 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.553469+0000", 2024-04-26T03:38:59.606 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0040316950000000001 2024-04-26T03:38:59.606 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.606 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.606 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:38:59.606 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.553500+0000", 2024-04-26T03:38:59.606 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.0855000000000002e-05 2024-04-26T03:38:59.606 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:59.606 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:59.606 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:59.606 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.607 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.607 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.fs6 35/32 ECSubWrite(tid=11, reqid=client.4556.0:112, at_version=35'3, trim_to=0'0, roll_forward_to=34'2))", 2024-04-26T03:38:59.607 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.576625+0000", 2024-04-26T03:38:59.607 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 7.9362229280000003, 2024-04-26T03:38:59.607 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.004650497, 2024-04-26T03:38:59.607 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:38:59.607 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:38:59.607 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:38:59.607 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:38:59.607 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.607 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:38:59.607 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.576625+0000", 2024-04-26T03:38:59.607 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:38:59.608 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.608 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.608 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:38:59.608 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.576625+0000", 2024-04-26T03:38:59.608 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:38:59.608 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.608 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.608 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:38:59.608 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.576626+0000", 2024-04-26T03:38:59.608 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.443e-06 2024-04-26T03:38:59.608 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.608 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.608 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:38:59.608 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.576826+0000", 2024-04-26T03:38:59.608 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00019968399999999999 2024-04-26T03:38:59.609 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.609 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.609 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:38:59.609 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.576844+0000", 2024-04-26T03:38:59.609 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.7779e-05 2024-04-26T03:38:59.609 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.609 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.609 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:38:59.609 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.576855+0000", 2024-04-26T03:38:59.609 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0881e-05 2024-04-26T03:38:59.609 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.609 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.609 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:38:59.609 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.576940+0000", 2024-04-26T03:38:59.609 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.5303999999999995e-05 2024-04-26T03:38:59.610 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.610 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.610 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:38:59.610 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.576947+0000", 2024-04-26T03:38:59.610 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.1890000000000001e-06 2024-04-26T03:38:59.610 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.610 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.610 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:38:59.610 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.581237+0000", 2024-04-26T03:38:59.610 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.004289635 2024-04-26T03:38:59.610 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.610 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.610 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:38:59.611 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.581275+0000", 2024-04-26T03:38:59.611 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.8581999999999998e-05 2024-04-26T03:38:59.611 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:59.611 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:59.611 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:59.611 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.611 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.611 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.bs1 35/32 ECSubWrite(tid=20, reqid=client.4556.0:127, at_version=35'7, trim_to=0'0, roll_forward_to=35'6))", 2024-04-26T03:38:59.611 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.601261+0000", 2024-04-26T03:38:59.611 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 7.9115869530000005, 2024-04-26T03:38:59.611 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0047710069999999999, 2024-04-26T03:38:59.611 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:38:59.611 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:38:59.611 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:38:59.611 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:38:59.612 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.612 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:38:59.612 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601261+0000", 2024-04-26T03:38:59.612 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:38:59.612 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.612 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.612 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:38:59.612 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601261+0000", 2024-04-26T03:38:59.612 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:38:59.612 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.612 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.612 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:38:59.612 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601262+0000", 2024-04-26T03:38:59.612 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1480000000000001e-06 2024-04-26T03:38:59.613 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.613 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.613 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:38:59.613 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601407+0000", 2024-04-26T03:38:59.613 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000144917 2024-04-26T03:38:59.613 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.613 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.613 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:38:59.613 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601430+0000", 2024-04-26T03:38:59.613 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.3011e-05 2024-04-26T03:38:59.613 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.613 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.613 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:38:59.613 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601440+0000", 2024-04-26T03:38:59.613 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0319999999999999e-05 2024-04-26T03:38:59.614 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.614 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.614 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:38:59.614 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601513+0000", 2024-04-26T03:38:59.614 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.2940000000000003e-05 2024-04-26T03:38:59.614 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.614 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.614 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:38:59.614 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601523+0000", 2024-04-26T03:38:59.614 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.5179999999999993e-06 2024-04-26T03:38:59.614 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.615 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.615 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:38:59.615 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.606000+0000", 2024-04-26T03:38:59.615 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00447759 2024-04-26T03:38:59.615 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.615 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.615 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:38:59.615 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.606032+0000", 2024-04-26T03:38:59.615 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.1563000000000003e-05 2024-04-26T03:38:59.615 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:59.615 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:59.615 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:59.615 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.615 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.615 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.fs6 40/32 ECSubWrite(tid=32, reqid=osd.11.0:32, at_version=40'13, trim_to=0'0, roll_forward_to=37'11))", 2024-04-26T03:38:59.616 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:56.508419+0000", 2024-04-26T03:38:59.616 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 3.0044290980000001, 2024-04-26T03:38:59.616 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0028936320000000001, 2024-04-26T03:38:59.616 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:38:59.616 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:38:59.616 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:38:59.616 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:38:59.616 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.616 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:38:59.616 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.508419+0000", 2024-04-26T03:38:59.616 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:38:59.616 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.616 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.616 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:38:59.616 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.508419+0000", 2024-04-26T03:38:59.617 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:38:59.617 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.617 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.617 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:38:59.617 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.508419+0000", 2024-04-26T03:38:59.617 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.1999999999999999e-07 2024-04-26T03:38:59.617 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.617 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.617 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:38:59.617 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.508437+0000", 2024-04-26T03:38:59.617 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.7312e-05 2024-04-26T03:38:59.617 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.617 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.617 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:38:59.618 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.508446+0000", 2024-04-26T03:38:59.618 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.3649999999999993e-06 2024-04-26T03:38:59.618 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.618 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.618 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:38:59.618 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.508453+0000", 2024-04-26T03:38:59.618 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.9990000000000002e-06 2024-04-26T03:38:59.618 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.618 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.618 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:38:59.618 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.508511+0000", 2024-04-26T03:38:59.618 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.8001999999999998e-05 2024-04-26T03:38:59.618 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.619 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.619 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:38:59.619 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.508521+0000", 2024-04-26T03:38:59.619 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.8649999999999996e-06 2024-04-26T03:38:59.619 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.619 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.619 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:38:59.619 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.511271+0000", 2024-04-26T03:38:59.619 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0027496220000000002 2024-04-26T03:38:59.619 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.619 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.619 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:38:59.619 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.511312+0000", 2024-04-26T03:38:59.619 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.1847000000000001e-05 2024-04-26T03:38:59.619 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:59.620 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:59.620 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:59.620 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.620 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.620 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.5s5 40/32 ECSubWrite(tid=22, reqid=client.4556.0:256, at_version=40'14, trim_to=0'0, roll_forward_to=37'13))", 2024-04-26T03:38:59.620 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:56.512677+0000", 2024-04-26T03:38:59.620 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 3.000171001, 2024-04-26T03:38:59.620 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.003562085, 2024-04-26T03:38:59.620 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:38:59.620 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:38:59.620 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:38:59.620 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:38:59.620 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.620 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:38:59.620 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.512677+0000", 2024-04-26T03:38:59.621 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:38:59.621 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.621 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.621 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:38:59.621 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.512677+0000", 2024-04-26T03:38:59.621 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:38:59.621 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.621 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.621 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:38:59.621 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.512677+0000", 2024-04-26T03:38:59.621 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.4200000000000002e-07 2024-04-26T03:38:59.621 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.621 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.621 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:38:59.621 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.512695+0000", 2024-04-26T03:38:59.622 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.823e-05 2024-04-26T03:38:59.622 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.622 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.622 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:38:59.622 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.512707+0000", 2024-04-26T03:38:59.622 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1239e-05 2024-04-26T03:38:59.622 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.622 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.622 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:38:59.622 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.512717+0000", 2024-04-26T03:38:59.622 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0149999999999999e-05 2024-04-26T03:38:59.622 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.622 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.622 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:38:59.623 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.512768+0000", 2024-04-26T03:38:59.623 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.0843000000000001e-05 2024-04-26T03:38:59.623 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.623 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.623 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:38:59.623 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.512779+0000", 2024-04-26T03:38:59.623 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1065e-05 2024-04-26T03:38:59.623 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.623 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.623 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:38:59.623 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.516193+0000", 2024-04-26T03:38:59.623 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0034144370000000002 2024-04-26T03:38:59.623 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.623 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.623 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:38:59.624 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.516239+0000", 2024-04-26T03:38:59.624 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.5878999999999999e-05 2024-04-26T03:38:59.624 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:59.624 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:59.624 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:59.624 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.624 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.624 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.cs2 40/32 ECSubWrite(tid=34, reqid=osd.11.0:31, at_version=0'0, trim_to=0'0, roll_forward_to=40'8))", 2024-04-26T03:38:59.624 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:56.512799+0000", 2024-04-26T03:38:59.624 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 3.0000491519999999, 2024-04-26T03:38:59.624 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0037140379999999998, 2024-04-26T03:38:59.624 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:38:59.624 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:38:59.624 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:38:59.625 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:38:59.625 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.625 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:38:59.625 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.512799+0000", 2024-04-26T03:38:59.625 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:38:59.625 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.625 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.625 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:38:59.625 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.512799+0000", 2024-04-26T03:38:59.625 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:38:59.625 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.625 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.625 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:38:59.626 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.512799+0000", 2024-04-26T03:38:59.626 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.3599999999999999e-07 2024-04-26T03:38:59.626 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.626 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.626 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:38:59.626 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.512807+0000", 2024-04-26T03:38:59.627 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.9910000000000002e-06 2024-04-26T03:38:59.627 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.627 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.627 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:38:59.627 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.512815+0000", 2024-04-26T03:38:59.627 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.1300000000000001e-06 2024-04-26T03:38:59.627 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.627 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.627 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:38:59.627 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.512822+0000", 2024-04-26T03:38:59.627 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.2010000000000002e-06 2024-04-26T03:38:59.627 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.627 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.627 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:38:59.628 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.512964+0000", 2024-04-26T03:38:59.628 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000141094 2024-04-26T03:38:59.628 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.628 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.628 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:38:59.628 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.512974+0000", 2024-04-26T03:38:59.628 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0831e-05 2024-04-26T03:38:59.628 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.628 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.628 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:38:59.628 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.515958+0000", 2024-04-26T03:38:59.628 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.002983762 2024-04-26T03:38:59.628 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.628 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.629 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:38:59.629 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.516513+0000", 2024-04-26T03:38:59.629 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00055459300000000003 2024-04-26T03:38:59.629 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:59.629 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:59.629 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:59.629 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.629 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.629 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.8s1 40/32 ECSubWrite(tid=178, reqid=client.4556.0:266, at_version=40'8, trim_to=0'0, roll_forward_to=35'6))", 2024-04-26T03:38:59.629 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:56.533458+0000", 2024-04-26T03:38:59.629 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 2.9793895629999998, 2024-04-26T03:38:59.629 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0056363389999999998, 2024-04-26T03:38:59.629 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:38:59.629 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:38:59.630 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:38:59.630 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:38:59.630 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.630 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:38:59.630 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.533458+0000", 2024-04-26T03:38:59.630 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:38:59.630 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.630 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.630 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:38:59.630 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.533458+0000", 2024-04-26T03:38:59.630 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:38:59.630 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.630 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.630 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:38:59.631 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.533458+0000", 2024-04-26T03:38:59.631 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.31e-07 2024-04-26T03:38:59.631 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.631 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.631 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:38:59.631 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.533829+0000", 2024-04-26T03:38:59.631 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000370882 2024-04-26T03:38:59.631 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.631 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.631 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:38:59.631 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.533841+0000", 2024-04-26T03:38:59.631 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1347e-05 2024-04-26T03:38:59.631 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.631 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.631 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:38:59.632 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.533848+0000", 2024-04-26T03:38:59.632 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.0010000000000004e-06 2024-04-26T03:38:59.632 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.632 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.632 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:38:59.632 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.533905+0000", 2024-04-26T03:38:59.632 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.7398000000000002e-05 2024-04-26T03:38:59.632 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.632 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.632 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:38:59.632 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.533913+0000", 2024-04-26T03:38:59.632 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.5669999999999997e-06 2024-04-26T03:38:59.632 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.632 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.632 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:38:59.633 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.539068+0000", 2024-04-26T03:38:59.633 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0051553160000000001 2024-04-26T03:38:59.633 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:38:59.633 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:38:59.633 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:38:59.633 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.539095+0000", 2024-04-26T03:38:59.633 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.6696999999999999e-05 2024-04-26T03:38:59.633 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:59.633 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:59.633 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:59.633 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:38:59.633 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:38:59.633 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:38:59.635 INFO:teuthology.orchestra.run.smithi028.stderr:noscrub is set 2024-04-26T03:38:59.636 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-26T03:38:59.638 INFO:tasks.ceph.osd.8.smithi028.stderr:2024-04-26T03:38:59.633+0000 7f5bb75d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 31968) UID: 0 2024-04-26T03:38:59.639 INFO:tasks.rados.rados.0.smithi092.stdout:116: done (2 left) 2024-04-26T03:38:59.639 INFO:tasks.rados.rados.0.smithi092.stdout:118: done (1 left) 2024-04-26T03:38:59.639 INFO:tasks.rados.rados.0.smithi092.stdout:120: done (0 left) 2024-04-26T03:38:59.639 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 24 current snap is 5 2024-04-26T03:38:59.639 INFO:tasks.rados.rados.0.smithi092.stdout:121: seq_num 72 ranges {393216=442368,835584=720896,1556480=49152} 2024-04-26T03:38:59.640 INFO:tasks.rados.rados.0.smithi092.stdout:121: writing smithi09235267-24 from 393216 to 835584 tid 1 2024-04-26T03:38:59.640 INFO:tasks.rados.rados.0.smithi092.stdout:121: writing smithi09235267-24 from 835584 to 1556480 tid 2 2024-04-26T03:38:59.640 INFO:tasks.rados.rados.0.smithi092.stdout:121: writing smithi09235267-24 from 1556480 to 1605632 tid 3 2024-04-26T03:38:59.640 INFO:tasks.rados.rados.0.smithi092.stdout:122: delete oid 43 current snap is 5 2024-04-26T03:38:59.640 INFO:tasks.rados.rados.0.smithi092.stdout:122: done (1 left) 2024-04-26T03:38:59.640 INFO:tasks.rados.rados.0.smithi092.stdout:123: rollback oid 8 current snap is 5 2024-04-26T03:38:59.640 INFO:tasks.rados.rados.0.smithi092.stdout:rollback oid 8 to 4 2024-04-26T03:38:59.640 INFO:tasks.rados.rados.0.smithi092.stdout:124: copy_from oid 25 from oid 49 current snap is 5 2024-04-26T03:38:59.640 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 30 current snap is 5 2024-04-26T03:38:59.640 INFO:tasks.rados.rados.0.smithi092.stdout:125: seq_num 74 ranges {1523712=475136,1998848=622592,2621440=147456} 2024-04-26T03:38:59.640 INFO:tasks.rados.rados.0.smithi092.stdout:125: writing smithi09235267-30 from 1523712 to 1998848 tid 1 2024-04-26T03:38:59.640 INFO:tasks.rados.rados.0.smithi092.stdout:125: writing smithi09235267-30 from 1998848 to 2621440 tid 2 2024-04-26T03:38:59.640 INFO:tasks.rados.rados.0.smithi092.stdout:125: writing smithi09235267-30 from 2621440 to 2768896 tid 3 2024-04-26T03:38:59.640 INFO:tasks.rados.rados.0.smithi092.stdout:126: snap_create 2024-04-26T03:38:59.640 INFO:tasks.rados.rados.0.smithi092.stdout:123: finishing rollback tid 0 to smithi09235267-8 2024-04-26T03:38:59.641 INFO:tasks.rados.rados.0.smithi092.stdout:123: finishing rollback tid 1 to smithi09235267-8 2024-04-26T03:38:59.641 INFO:tasks.rados.rados.0.smithi092.stdout:123: finishing rollback tid 2 to smithi09235267-8 2024-04-26T03:38:59.641 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 8 v 6 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-04-26T03:38:59.641 INFO:tasks.rados.rados.0.smithi092.stdout:121: finishing write tid 1 to smithi09235267-24 2024-04-26T03:38:59.641 INFO:tasks.rados.rados.0.smithi092.stdout:121: finishing write tid 2 to smithi09235267-24 2024-04-26T03:38:59.641 INFO:tasks.rados.rados.0.smithi092.stdout:121: finishing write tid 3 to smithi09235267-24 2024-04-26T03:38:59.641 INFO:tasks.rados.rados.0.smithi092.stdout:121: finishing write tid 4 to smithi09235267-24 2024-04-26T03:38:59.641 INFO:tasks.rados.rados.0.smithi092.stdout:121: finishing write tid 5 to smithi09235267-24 2024-04-26T03:38:59.641 INFO:tasks.rados.rados.0.smithi092.stdout:121: oid 24 updating version 0 to 19 2024-04-26T03:38:59.641 INFO:tasks.rados.rados.0.smithi092.stdout:121: oid 24 version 19 is already newer than 16 2024-04-26T03:38:59.641 INFO:tasks.rados.rados.0.smithi092.stdout:121: oid 24 version 19 is already newer than 18 2024-04-26T03:38:59.641 INFO:tasks.rados.rados.0.smithi092.stdout:121: oid 24 version 19 is already newer than 17 2024-04-26T03:38:59.641 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 24 v 19 (ObjNum 72 snap 5 seq_num 72) dirty exists 2024-04-26T03:38:59.641 INFO:tasks.rados.rados.0.smithi092.stdout:121: left oid 24 (ObjNum 72 snap 5 seq_num 72) 2024-04-26T03:38:59.641 INFO:tasks.rados.rados.0.smithi092.stdout:124: finishing copy_from racing read to smithi09235267-25 2024-04-26T03:38:59.642 INFO:tasks.rados.rados.0.smithi092.stdout:124: finishing copy_from to smithi09235267-25 2024-04-26T03:38:59.642 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 25 v 3 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-04-26T03:38:59.642 INFO:tasks.rados.rados.0.smithi092.stdout:125: finishing write tid 1 to smithi09235267-30 2024-04-26T03:38:59.642 INFO:tasks.rados.rados.0.smithi092.stdout:125: finishing write tid 2 to smithi09235267-30 2024-04-26T03:38:59.642 INFO:tasks.rados.rados.0.smithi092.stdout:125: finishing write tid 3 to smithi09235267-30 2024-04-26T03:38:59.642 INFO:tasks.rados.rados.0.smithi092.stdout:125: finishing write tid 4 to smithi09235267-30 2024-04-26T03:38:59.642 INFO:tasks.rados.rados.0.smithi092.stdout:125: finishing write tid 5 to smithi09235267-30 2024-04-26T03:38:59.642 INFO:tasks.rados.rados.0.smithi092.stdout:125: oid 30 updating version 0 to 17 2024-04-26T03:38:59.642 INFO:tasks.rados.rados.0.smithi092.stdout:125: oid 30 version 17 is already newer than 16 2024-04-26T03:38:59.642 INFO:tasks.rados.rados.0.smithi092.stdout:125: oid 30 updating version 17 to 19 2024-04-26T03:38:59.642 INFO:tasks.rados.rados.0.smithi092.stdout:125: oid 30 version 19 is already newer than 18 2024-04-26T03:38:59.642 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 30 v 19 (ObjNum 74 snap 5 seq_num 74) dirty exists 2024-04-26T03:38:59.642 INFO:tasks.rados.rados.0.smithi092.stdout:125: left oid 30 (ObjNum 74 snap 5 seq_num 74) 2024-04-26T03:38:59.735 INFO:tasks.ceph.osd.15.smithi092.stderr:2024-04-26T03:38:59.734+0000 7f6d12dec640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 31824) UID: 0 2024-04-26T03:38:59.783 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:38:59.784 INFO:teuthology.orchestra.run.smithi028.stdout: "ops": [], 2024-04-26T03:38:59.784 INFO:teuthology.orchestra.run.smithi028.stdout: "num_ops": 0 2024-04-26T03:38:59.784 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:38:59.794 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-26T03:38:59.836 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:38:59.835+0000 7fb78c544640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31956) UID: 0 2024-04-26T03:38:59.900 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:38:59.900 INFO:teuthology.orchestra.run.smithi028.stdout: "ops": [], 2024-04-26T03:38:59.900 INFO:teuthology.orchestra.run.smithi028.stdout: "complaint_time": 30, 2024-04-26T03:38:59.900 INFO:teuthology.orchestra.run.smithi028.stdout: "num_blocked_ops": 0 2024-04-26T03:38:59.901 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:38:59.909 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-26T03:38:59.937 INFO:tasks.ceph.osd.10.smithi066.stderr:2024-04-26T03:38:59.936+0000 7f130fcc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 31763) UID: 0 2024-04-26T03:39:00.019 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:39:00.019 INFO:teuthology.orchestra.run.smithi028.stdout: "size": 20, 2024-04-26T03:39:00.019 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 600, 2024-04-26T03:39:00.019 INFO:teuthology.orchestra.run.smithi028.stdout: "ops": [ 2024-04-26T03:39:00.019 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.019 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:7 3.bs0 3:dbf634d0:::smithi09235267-3:head [append 0~49152,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:00.019 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.481375+0000", 2024-04-26T03:39:00.019 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 8.5346726799999999, 2024-04-26T03:39:00.019 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.012016825, 2024-04-26T03:39:00.019 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:00.019 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:00.019 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:00.020 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:00.020 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:00.020 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:00.020 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 7 2024-04-26T03:39:00.020 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.020 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:00.020 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.020 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:00.020 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.481375+0000", 2024-04-26T03:39:00.020 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.020 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.020 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.020 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:00.020 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.481375+0000", 2024-04-26T03:39:00.021 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.021 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.021 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.021 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:00.021 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.481377+0000", 2024-04-26T03:39:00.021 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.3149999999999999e-06 2024-04-26T03:39:00.021 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.021 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.021 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:00.021 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.481501+0000", 2024-04-26T03:39:00.021 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000123872 2024-04-26T03:39:00.021 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.021 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.021 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:00.022 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.481519+0000", 2024-04-26T03:39:00.022 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.7974000000000001e-05 2024-04-26T03:39:00.022 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.022 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.022 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:00.022 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.481532+0000", 2024-04-26T03:39:00.022 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.2738000000000001e-05 2024-04-26T03:39:00.022 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.022 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.022 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.022 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.481628+0000", 2024-04-26T03:39:00.022 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.6408000000000003e-05 2024-04-26T03:39:00.022 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.022 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.023 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:00.023 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.481811+0000", 2024-04-26T03:39:00.023 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000182516 2024-04-26T03:39:00.023 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.023 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.023 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:00.023 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.482812+0000", 2024-04-26T03:39:00.023 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.001000641 2024-04-26T03:39:00.023 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.023 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.023 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:00.023 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.485739+0000", 2024-04-26T03:39:00.023 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0029270860000000002 2024-04-26T03:39:00.023 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.024 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.024 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:00.024 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.492917+0000", 2024-04-26T03:39:00.024 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0071787489999999999 2024-04-26T03:39:00.024 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.024 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.024 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:00.024 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493392+0000", 2024-04-26T03:39:00.024 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00047452600000000002 2024-04-26T03:39:00.024 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.024 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:00.024 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.024 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.024 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.024 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:8 3.bs0 3:dbf634d0:::smithi09235267-3:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:00.025 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.481563+0000", 2024-04-26T03:39:00.025 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 8.5344848100000004, 2024-04-26T03:39:00.025 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.012116942, 2024-04-26T03:39:00.025 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:00.025 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:00.025 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:00.025 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:00.025 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:00.025 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:00.025 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 8 2024-04-26T03:39:00.025 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.025 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:00.025 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.025 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:00.026 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.481563+0000", 2024-04-26T03:39:00.026 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.026 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.026 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.026 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:00.026 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.481563+0000", 2024-04-26T03:39:00.026 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.026 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.026 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.026 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:00.026 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.481564+0000", 2024-04-26T03:39:00.026 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.3300000000000001e-07 2024-04-26T03:39:00.026 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.026 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.027 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:00.027 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.481568+0000", 2024-04-26T03:39:00.027 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.7060000000000003e-06 2024-04-26T03:39:00.027 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.027 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.027 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:00.027 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.481577+0000", 2024-04-26T03:39:00.027 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.9239999999999996e-06 2024-04-26T03:39:00.027 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.027 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.027 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:00.027 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.481584+0000", 2024-04-26T03:39:00.027 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.0040000000000002e-06 2024-04-26T03:39:00.027 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.028 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.028 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.028 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.483702+0000", 2024-04-26T03:39:00.028 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0021172610000000001 2024-04-26T03:39:00.028 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.028 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.028 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:00.028 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.483806+0000", 2024-04-26T03:39:00.028 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000103836 2024-04-26T03:39:00.028 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.028 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.028 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:00.028 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.484343+0000", 2024-04-26T03:39:00.028 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00053700700000000004 2024-04-26T03:39:00.028 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.029 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.029 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:00.029 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.485741+0000", 2024-04-26T03:39:00.029 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0013980920000000001 2024-04-26T03:39:00.029 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.029 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.029 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:00.029 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493581+0000", 2024-04-26T03:39:00.029 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0078406120000000003 2024-04-26T03:39:00.029 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.029 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.029 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:00.029 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493680+0000", 2024-04-26T03:39:00.029 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.8666999999999996e-05 2024-04-26T03:39:00.030 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.030 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:00.030 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.030 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.030 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.030 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:9 3.bs0 3:dbf634d0:::smithi09235267-3:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:00.030 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.481597+0000", 2024-04-26T03:39:00.030 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 8.5344508870000002, 2024-04-26T03:39:00.030 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.024373156999999999, 2024-04-26T03:39:00.030 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:00.030 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:00.030 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "started", 2024-04-26T03:39:00.030 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:00.030 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:00.031 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:00.031 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 9 2024-04-26T03:39:00.031 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.031 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:00.031 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.031 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:00.031 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.481597+0000", 2024-04-26T03:39:00.031 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.031 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.031 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.031 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:00.031 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.481597+0000", 2024-04-26T03:39:00.031 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.031 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.031 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.032 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:00.032 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.481597+0000", 2024-04-26T03:39:00.032 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.5599999999999995e-07 2024-04-26T03:39:00.032 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.032 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.032 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:00.032 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.481610+0000", 2024-04-26T03:39:00.032 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.2330000000000001e-05 2024-04-26T03:39:00.032 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.032 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.032 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:00.032 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.481618+0000", 2024-04-26T03:39:00.032 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.7159999999999992e-06 2024-04-26T03:39:00.032 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.033 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.033 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:00.033 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.481624+0000", 2024-04-26T03:39:00.033 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.8989999999999998e-06 2024-04-26T03:39:00.033 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.033 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.033 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.033 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.484627+0000", 2024-04-26T03:39:00.033 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00300257 2024-04-26T03:39:00.033 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.033 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.033 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:00.033 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.484712+0000", 2024-04-26T03:39:00.033 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.4753000000000007e-05 2024-04-26T03:39:00.034 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.034 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.034 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.034 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.494632+0000", 2024-04-26T03:39:00.034 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0099199100000000005 2024-04-26T03:39:00.034 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.034 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.034 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:00.034 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.494899+0000", 2024-04-26T03:39:00.034 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00026711399999999999 2024-04-26T03:39:00.034 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.034 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.034 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:00.034 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.505970+0000", 2024-04-26T03:39:00.034 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.011071309 2024-04-26T03:39:00.035 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.035 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:00.035 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.035 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.035 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.035 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:22 3.7s0 3:e644bba7:::smithi09235267-8:head [append 0~131072,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:00.035 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.484923+0000", 2024-04-26T03:39:00.035 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 8.5311251739999996, 2024-04-26T03:39:00.035 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.010111055000000001, 2024-04-26T03:39:00.035 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:00.035 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:00.035 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:00.035 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:00.035 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:00.036 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:00.036 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 22 2024-04-26T03:39:00.036 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.036 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:00.036 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.036 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:00.036 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.484923+0000", 2024-04-26T03:39:00.036 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.036 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.036 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.036 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:00.036 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.484923+0000", 2024-04-26T03:39:00.036 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.036 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.036 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.037 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:00.037 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.484925+0000", 2024-04-26T03:39:00.037 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.0439999999999998e-06 2024-04-26T03:39:00.037 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.037 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.037 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:00.037 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.485026+0000", 2024-04-26T03:39:00.037 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000101322 2024-04-26T03:39:00.037 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.037 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.037 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:00.037 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.485041+0000", 2024-04-26T03:39:00.037 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.5529e-05 2024-04-26T03:39:00.037 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.038 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.038 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:00.038 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.485051+0000", 2024-04-26T03:39:00.038 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.3149999999999998e-06 2024-04-26T03:39:00.038 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.038 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.038 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.038 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.485114+0000", 2024-04-26T03:39:00.038 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.3307999999999998e-05 2024-04-26T03:39:00.038 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.038 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.038 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:00.038 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.485257+0000", 2024-04-26T03:39:00.038 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00014296400000000001 2024-04-26T03:39:00.039 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.039 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.039 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:00.039 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.486112+0000", 2024-04-26T03:39:00.039 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00085478099999999999 2024-04-26T03:39:00.039 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.039 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.039 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:00.039 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.488807+0000", 2024-04-26T03:39:00.039 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0026950590000000001 2024-04-26T03:39:00.039 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.039 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.039 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:00.039 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.494959+0000", 2024-04-26T03:39:00.039 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0061523200000000002 2024-04-26T03:39:00.040 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.040 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.040 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:00.040 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495034+0000", 2024-04-26T03:39:00.040 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.4412999999999994e-05 2024-04-26T03:39:00.040 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.040 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:00.040 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.040 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.040 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.040 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:23 3.7s0 3:e644bba7:::smithi09235267-8:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:00.040 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.485079+0000", 2024-04-26T03:39:00.040 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 8.5309691930000007, 2024-04-26T03:39:00.040 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.011530514, 2024-04-26T03:39:00.041 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:00.041 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:00.041 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:00.041 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:00.041 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:00.041 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:00.041 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 23 2024-04-26T03:39:00.041 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.041 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:00.041 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.041 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:00.041 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.485079+0000", 2024-04-26T03:39:00.041 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.041 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.042 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.042 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:00.042 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.485079+0000", 2024-04-26T03:39:00.042 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.042 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.042 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.042 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:00.042 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.485080+0000", 2024-04-26T03:39:00.042 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.4799999999999997e-07 2024-04-26T03:39:00.042 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.042 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.042 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:00.042 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.485084+0000", 2024-04-26T03:39:00.042 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.1470000000000003e-06 2024-04-26T03:39:00.043 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.043 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.043 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:00.043 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.485092+0000", 2024-04-26T03:39:00.043 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.7620000000000001e-06 2024-04-26T03:39:00.043 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.043 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.043 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:00.043 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.485100+0000", 2024-04-26T03:39:00.043 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.4320000000000002e-06 2024-04-26T03:39:00.043 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.043 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.043 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.043 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.487879+0000", 2024-04-26T03:39:00.044 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0027795649999999999 2024-04-26T03:39:00.044 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.044 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.044 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:00.044 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.487970+0000", 2024-04-26T03:39:00.044 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.0254999999999996e-05 2024-04-26T03:39:00.044 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.044 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.044 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:00.044 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.488491+0000", 2024-04-26T03:39:00.044 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00052132899999999995 2024-04-26T03:39:00.044 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.044 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.044 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:00.045 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.489060+0000", 2024-04-26T03:39:00.045 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00056855600000000001 2024-04-26T03:39:00.045 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.045 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.045 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:00.045 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.496521+0000", 2024-04-26T03:39:00.045 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0074614210000000002 2024-04-26T03:39:00.045 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.045 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.045 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:00.045 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.496609+0000", 2024-04-26T03:39:00.045 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.8098999999999998e-05 2024-04-26T03:39:00.045 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.045 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:00.045 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.046 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.046 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.046 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:24 3.7s0 3:e644bba7:::smithi09235267-8:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:00.046 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.485110+0000", 2024-04-26T03:39:00.046 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 8.53093784, 2024-04-26T03:39:00.046 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.015789734, 2024-04-26T03:39:00.046 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:00.046 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:00.046 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "started", 2024-04-26T03:39:00.046 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:00.046 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:00.046 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:00.046 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 24 2024-04-26T03:39:00.047 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.047 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:00.047 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.047 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:00.047 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.485110+0000", 2024-04-26T03:39:00.047 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.047 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.047 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.047 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:00.047 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.485110+0000", 2024-04-26T03:39:00.047 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.047 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.047 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.047 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:00.047 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.485110+0000", 2024-04-26T03:39:00.048 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.4099999999999999e-07 2024-04-26T03:39:00.048 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.048 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.048 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:00.048 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.485114+0000", 2024-04-26T03:39:00.048 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.2430000000000001e-06 2024-04-26T03:39:00.048 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.048 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.048 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:00.048 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.485120+0000", 2024-04-26T03:39:00.048 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.5810000000000002e-06 2024-04-26T03:39:00.048 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.048 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.048 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:00.049 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.485125+0000", 2024-04-26T03:39:00.049 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.0270000000000003e-06 2024-04-26T03:39:00.049 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.049 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.049 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.049 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.488721+0000", 2024-04-26T03:39:00.049 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.003595844 2024-04-26T03:39:00.049 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.049 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.049 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:00.049 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.488795+0000", 2024-04-26T03:39:00.049 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.3850000000000006e-05 2024-04-26T03:39:00.049 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.049 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.049 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.050 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.496954+0000", 2024-04-26T03:39:00.050 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0081585890000000008 2024-04-26T03:39:00.050 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.050 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.050 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:00.050 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.497038+0000", 2024-04-26T03:39:00.050 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.3934e-05 2024-04-26T03:39:00.050 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.050 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.050 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:00.050 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.500900+0000", 2024-04-26T03:39:00.050 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0038621250000000001 2024-04-26T03:39:00.050 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.050 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:00.051 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.051 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.051 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.051 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:37 3.bs0 3:d42e9299:::smithi09235267-13:head [append 0~212992,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:00.051 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.492880+0000", 2024-04-26T03:39:00.051 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 8.5231678049999999, 2024-04-26T03:39:00.051 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.019386614999999999, 2024-04-26T03:39:00.051 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:00.051 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:00.051 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:00.051 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:00.051 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:00.051 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:00.051 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 37 2024-04-26T03:39:00.051 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.052 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:00.052 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.052 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:00.052 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.492880+0000", 2024-04-26T03:39:00.052 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.052 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.052 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.052 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:00.052 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.492880+0000", 2024-04-26T03:39:00.052 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.052 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.052 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.052 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:00.052 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.492898+0000", 2024-04-26T03:39:00.053 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.7759000000000001e-05 2024-04-26T03:39:00.053 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.053 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.053 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:00.053 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493142+0000", 2024-04-26T03:39:00.053 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00024467400000000001 2024-04-26T03:39:00.053 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.053 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.053 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:00.053 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493155+0000", 2024-04-26T03:39:00.053 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.2862e-05 2024-04-26T03:39:00.053 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.053 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.053 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:00.053 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493197+0000", 2024-04-26T03:39:00.054 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.2239e-05 2024-04-26T03:39:00.054 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.054 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.054 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.054 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495213+0000", 2024-04-26T03:39:00.054 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0020159090000000002 2024-04-26T03:39:00.054 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.054 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.054 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:00.054 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495342+0000", 2024-04-26T03:39:00.054 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000128214 2024-04-26T03:39:00.054 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.054 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.054 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:00.055 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.501605+0000", 2024-04-26T03:39:00.055 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0062634530000000004 2024-04-26T03:39:00.055 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.055 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.055 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:00.055 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.505099+0000", 2024-04-26T03:39:00.055 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0034938859999999999 2024-04-26T03:39:00.055 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.055 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.055 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:00.055 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.512085+0000", 2024-04-26T03:39:00.055 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0069863369999999996 2024-04-26T03:39:00.055 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.055 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.056 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:00.056 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.512267+0000", 2024-04-26T03:39:00.056 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000181282 2024-04-26T03:39:00.056 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.056 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:00.056 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.056 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.056 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.056 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:38 3.bs0 3:d42e9299:::smithi09235267-13:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:00.056 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.493215+0000", 2024-04-26T03:39:00.056 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 8.5228325920000003, 2024-04-26T03:39:00.056 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.025985364, 2024-04-26T03:39:00.056 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:00.056 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:00.057 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:00.057 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:00.057 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:00.057 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:00.057 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 38 2024-04-26T03:39:00.057 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.057 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:00.057 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.057 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:00.057 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493215+0000", 2024-04-26T03:39:00.057 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.057 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.057 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.057 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:00.057 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493215+0000", 2024-04-26T03:39:00.058 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.058 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.058 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.058 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:00.058 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493216+0000", 2024-04-26T03:39:00.058 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.4700000000000001e-07 2024-04-26T03:39:00.058 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.058 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.058 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:00.058 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493261+0000", 2024-04-26T03:39:00.058 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.4604000000000002e-05 2024-04-26T03:39:00.058 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.058 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.058 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:00.059 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493271+0000", 2024-04-26T03:39:00.059 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0550999999999999e-05 2024-04-26T03:39:00.059 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.059 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.059 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:00.059 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493279+0000", 2024-04-26T03:39:00.059 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.3790000000000006e-06 2024-04-26T03:39:00.059 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.059 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.059 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.059 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.506027+0000", 2024-04-26T03:39:00.059 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.012747058 2024-04-26T03:39:00.059 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.059 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.059 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:00.060 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.506166+0000", 2024-04-26T03:39:00.060 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000139357 2024-04-26T03:39:00.060 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.060 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.060 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:00.060 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.506670+0000", 2024-04-26T03:39:00.060 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00050440100000000002 2024-04-26T03:39:00.060 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.060 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.060 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:00.060 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.509149+0000", 2024-04-26T03:39:00.060 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.002478839 2024-04-26T03:39:00.060 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.060 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.061 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:00.061 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.519099+0000", 2024-04-26T03:39:00.061 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0099501880000000004 2024-04-26T03:39:00.061 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.061 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.061 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:00.061 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.519201+0000", 2024-04-26T03:39:00.061 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00010124 2024-04-26T03:39:00.061 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.061 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:00.061 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.061 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.061 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.061 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:39 3.bs0 3:d42e9299:::smithi09235267-13:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:00.062 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.493292+0000", 2024-04-26T03:39:00.062 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 8.5227562169999995, 2024-04-26T03:39:00.062 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.029045519999999998, 2024-04-26T03:39:00.062 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:00.062 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:00.062 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "started", 2024-04-26T03:39:00.062 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:00.062 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:00.062 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:00.062 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 39 2024-04-26T03:39:00.062 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.062 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:00.062 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.062 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:00.062 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493292+0000", 2024-04-26T03:39:00.063 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.063 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.063 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.063 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:00.063 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493292+0000", 2024-04-26T03:39:00.063 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.063 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.063 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.063 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:00.063 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493292+0000", 2024-04-26T03:39:00.063 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.3099999999999997e-07 2024-04-26T03:39:00.063 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.063 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.063 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:00.064 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493296+0000", 2024-04-26T03:39:00.064 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.0820000000000001e-06 2024-04-26T03:39:00.064 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.064 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.064 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:00.064 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493304+0000", 2024-04-26T03:39:00.064 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.0269999999999995e-06 2024-04-26T03:39:00.064 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.064 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.064 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:00.064 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493311+0000", 2024-04-26T03:39:00.064 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.1140000000000002e-06 2024-04-26T03:39:00.064 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.064 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.064 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.065 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.506997+0000", 2024-04-26T03:39:00.065 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.013686218999999999 2024-04-26T03:39:00.065 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.065 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.065 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:00.065 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.507073+0000", 2024-04-26T03:39:00.065 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.6766000000000003e-05 2024-04-26T03:39:00.065 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.065 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.065 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.065 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.519285+0000", 2024-04-26T03:39:00.065 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.012211324000000001 2024-04-26T03:39:00.065 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.065 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.065 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:00.066 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.519498+0000", 2024-04-26T03:39:00.066 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000213348 2024-04-26T03:39:00.066 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.066 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.066 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:00.066 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.522337+0000", 2024-04-26T03:39:00.066 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0028389090000000001 2024-04-26T03:39:00.066 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.066 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:00.066 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.066 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.066 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.066 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWriteReply(3.bs0 35/32 ECSubWriteReply(tid=4, last_complete=34'2, committed=1, applied=1))", 2024-04-26T03:39:00.066 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.494760+0000", 2024-04-26T03:39:00.067 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 8.5212880609999999, 2024-04-26T03:39:00.067 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0096384239999999996, 2024-04-26T03:39:00.067 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:00.067 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:00.067 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:00.067 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:00.067 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.067 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:00.067 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.494760+0000", 2024-04-26T03:39:00.067 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.067 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.067 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.067 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:00.067 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.494760+0000", 2024-04-26T03:39:00.067 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.068 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.068 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.068 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:00.068 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.494760+0000", 2024-04-26T03:39:00.068 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.9699999999999996e-07 2024-04-26T03:39:00.068 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.068 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.068 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:00.068 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.494765+0000", 2024-04-26T03:39:00.068 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.7700000000000001e-06 2024-04-26T03:39:00.068 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.068 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.068 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:00.068 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.494775+0000", 2024-04-26T03:39:00.069 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.024e-05 2024-04-26T03:39:00.069 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.069 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.069 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:00.069 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.494782+0000", 2024-04-26T03:39:00.069 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.9399999999999996e-06 2024-04-26T03:39:00.069 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.069 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.069 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.069 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.504376+0000", 2024-04-26T03:39:00.069 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0095933549999999996 2024-04-26T03:39:00.069 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.069 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.069 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:00.069 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.504398+0000", 2024-04-26T03:39:00.070 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.2521999999999999e-05 2024-04-26T03:39:00.070 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.070 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:00.070 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.070 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.070 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.070 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpRead(3.bs0 35/32 ECSubRead(tid=5, to_read={3:dbf634d0:::smithi09235267-3:head=0,4096,0}, subchunks={3:dbf634d0:::smithi09235267-3:head=[0,1]}, attrs_to_read=))", 2024-04-26T03:39:00.070 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.495114+0000", 2024-04-26T03:39:00.070 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 8.5209334020000007, 2024-04-26T03:39:00.070 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0097676889999999995, 2024-04-26T03:39:00.070 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:00.070 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:00.070 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:00.070 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:00.070 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.071 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:00.071 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495114+0000", 2024-04-26T03:39:00.071 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.071 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.071 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.071 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:00.071 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495114+0000", 2024-04-26T03:39:00.071 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.071 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.071 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.071 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:00.071 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495114+0000", 2024-04-26T03:39:00.071 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.071 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.071 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.072 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:00.072 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495114+0000", 2024-04-26T03:39:00.072 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.072 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.072 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.072 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:00.072 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495164+0000", 2024-04-26T03:39:00.072 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.9267000000000001e-05 2024-04-26T03:39:00.072 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.072 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.072 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:00.072 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495176+0000", 2024-04-26T03:39:00.072 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1970000000000001e-05 2024-04-26T03:39:00.072 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.072 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.073 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.073 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.504563+0000", 2024-04-26T03:39:00.073 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0093872739999999993 2024-04-26T03:39:00.073 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.073 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.073 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:00.073 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.504882+0000", 2024-04-26T03:39:00.073 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00031917799999999999 2024-04-26T03:39:00.073 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.073 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:00.073 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.073 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.073 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.074 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWriteReply(3.bs0 35/32 ECSubWriteReply(tid=4, last_complete=34'2, committed=1, applied=1))", 2024-04-26T03:39:00.074 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.495143+0000", 2024-04-26T03:39:00.074 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 8.5209052649999997, 2024-04-26T03:39:00.074 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0098331189999999995, 2024-04-26T03:39:00.074 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:00.074 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:00.074 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:00.074 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:00.074 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.074 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:00.074 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495143+0000", 2024-04-26T03:39:00.074 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.074 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.074 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.074 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:00.075 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495143+0000", 2024-04-26T03:39:00.075 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.075 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.075 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.075 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:00.075 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495143+0000", 2024-04-26T03:39:00.075 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.3900000000000004e-07 2024-04-26T03:39:00.075 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.075 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.075 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:00.075 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495148+0000", 2024-04-26T03:39:00.075 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.0950000000000003e-06 2024-04-26T03:39:00.075 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.075 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.076 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:00.076 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495181+0000", 2024-04-26T03:39:00.076 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.2279e-05 2024-04-26T03:39:00.076 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.076 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.076 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:00.076 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495188+0000", 2024-04-26T03:39:00.076 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.3390000000000004e-06 2024-04-26T03:39:00.076 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.076 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.076 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.076 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.504952+0000", 2024-04-26T03:39:00.076 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0097639179999999999 2024-04-26T03:39:00.076 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.076 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.077 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:00.077 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.504976+0000", 2024-04-26T03:39:00.077 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.3649e-05 2024-04-26T03:39:00.077 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.077 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:00.077 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.077 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.077 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.077 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.3s2 35/32 ECSubWrite(tid=2, reqid=client.4556.0:17, at_version=0'0, trim_to=0'0, roll_forward_to=34'2))", 2024-04-26T03:39:00.077 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.499475+0000", 2024-04-26T03:39:00.077 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 8.5165726890000002, 2024-04-26T03:39:00.077 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0096714069999999999, 2024-04-26T03:39:00.077 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:00.077 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:00.078 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:00.078 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:00.078 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.078 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:00.078 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.499475+0000", 2024-04-26T03:39:00.078 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.078 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.078 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.078 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:00.078 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.499475+0000", 2024-04-26T03:39:00.078 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.078 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.078 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.078 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:00.079 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.499476+0000", 2024-04-26T03:39:00.079 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.4799999999999997e-07 2024-04-26T03:39:00.079 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.079 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.079 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:00.079 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.499489+0000", 2024-04-26T03:39:00.079 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.2812e-05 2024-04-26T03:39:00.079 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.079 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.079 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:00.079 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.499501+0000", 2024-04-26T03:39:00.079 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.2112e-05 2024-04-26T03:39:00.079 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.079 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.080 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:00.080 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.499510+0000", 2024-04-26T03:39:00.080 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.6889999999999993e-06 2024-04-26T03:39:00.080 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.080 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.080 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.080 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.505498+0000", 2024-04-26T03:39:00.080 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0059880569999999998 2024-04-26T03:39:00.080 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.080 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.080 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:00.080 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.505506+0000", 2024-04-26T03:39:00.080 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.4800000000000001e-06 2024-04-26T03:39:00.080 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.080 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.081 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:00.081 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.509084+0000", 2024-04-26T03:39:00.081 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0035780849999999999 2024-04-26T03:39:00.081 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.081 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.081 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:00.081 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.509146+0000", 2024-04-26T03:39:00.081 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.2323999999999995e-05 2024-04-26T03:39:00.081 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.081 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:00.081 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.081 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.081 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.081 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:120 3.1s0 3:8cd34b8e:::smithi09235267-40:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:00.082 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.584517+0000", 2024-04-26T03:39:00.082 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 8.4315304920000003, 2024-04-26T03:39:00.082 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0098389670000000005, 2024-04-26T03:39:00.082 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:00.082 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:00.082 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "started", 2024-04-26T03:39:00.082 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:00.082 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:00.082 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:00.082 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 120 2024-04-26T03:39:00.082 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.082 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:00.082 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.082 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:00.083 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.584517+0000", 2024-04-26T03:39:00.083 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.083 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.083 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.083 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:00.083 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.584517+0000", 2024-04-26T03:39:00.083 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.083 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.083 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.083 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:00.083 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.584518+0000", 2024-04-26T03:39:00.083 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.9299999999999998e-07 2024-04-26T03:39:00.083 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.083 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.083 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:00.084 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.584522+0000", 2024-04-26T03:39:00.084 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.1450000000000001e-06 2024-04-26T03:39:00.084 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.084 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.084 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:00.084 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.584531+0000", 2024-04-26T03:39:00.084 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.7909999999999993e-06 2024-04-26T03:39:00.084 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.084 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.084 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:00.084 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.584537+0000", 2024-04-26T03:39:00.084 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.4489999999999996e-06 2024-04-26T03:39:00.084 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.084 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.085 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.085 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.590303+0000", 2024-04-26T03:39:00.085 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0057655010000000001 2024-04-26T03:39:00.085 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.085 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.085 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:00.085 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.590395+0000", 2024-04-26T03:39:00.085 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.2585999999999998e-05 2024-04-26T03:39:00.085 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.085 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.085 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.085 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.593187+0000", 2024-04-26T03:39:00.085 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0027916859999999998 2024-04-26T03:39:00.085 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.085 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.086 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:00.086 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.593282+0000", 2024-04-26T03:39:00.086 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.4720000000000001e-05 2024-04-26T03:39:00.086 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.086 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.086 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:00.086 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.594356+0000", 2024-04-26T03:39:00.086 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0010744960000000001 2024-04-26T03:39:00.086 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.086 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:00.086 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.086 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.086 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.086 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:127 3.bs0 3:d339032c:::smithi09235267-43:head [append 0~720896,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:00.087 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.599452+0000", 2024-04-26T03:39:00.087 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 8.4165957480000007, 2024-04-26T03:39:00.087 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0095046049999999993, 2024-04-26T03:39:00.087 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:00.087 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:00.087 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:00.087 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:00.087 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:00.087 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:00.087 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 127 2024-04-26T03:39:00.087 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.087 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:00.087 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.087 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:00.087 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.599452+0000", 2024-04-26T03:39:00.088 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.088 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.088 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.088 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:00.088 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.599452+0000", 2024-04-26T03:39:00.088 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.088 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.088 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.088 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:00.088 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.599453+0000", 2024-04-26T03:39:00.088 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.02e-06 2024-04-26T03:39:00.088 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.088 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.088 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:00.089 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.600092+0000", 2024-04-26T03:39:00.089 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00063885000000000005 2024-04-26T03:39:00.089 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.089 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.089 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:00.089 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.600103+0000", 2024-04-26T03:39:00.089 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0883999999999999e-05 2024-04-26T03:39:00.089 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.089 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.089 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:00.089 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.600110+0000", 2024-04-26T03:39:00.089 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.0609999999999998e-06 2024-04-26T03:39:00.089 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.089 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.089 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.090 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.600173+0000", 2024-04-26T03:39:00.090 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.2749000000000005e-05 2024-04-26T03:39:00.090 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.090 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.090 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:00.090 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.600323+0000", 2024-04-26T03:39:00.090 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00014995300000000001 2024-04-26T03:39:00.090 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.090 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.090 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:00.090 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601244+0000", 2024-04-26T03:39:00.090 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000920969 2024-04-26T03:39:00.090 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.090 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.091 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:00.091 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.608654+0000", 2024-04-26T03:39:00.091 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0074103930000000004 2024-04-26T03:39:00.091 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.091 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.091 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:00.091 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.608890+0000", 2024-04-26T03:39:00.091 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000236352 2024-04-26T03:39:00.091 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.091 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.091 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:00.091 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.608957+0000", 2024-04-26T03:39:00.091 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.6373999999999998e-05 2024-04-26T03:39:00.091 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.091 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:00.092 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.092 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.092 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.092 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:128 3.bs0 3:d339032c:::smithi09235267-43:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:00.092 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.600125+0000", 2024-04-26T03:39:00.092 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 8.4159226539999992, 2024-04-26T03:39:00.092 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.018721534000000001, 2024-04-26T03:39:00.092 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:00.092 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:00.092 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:00.092 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:00.092 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:00.092 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:00.092 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 128 2024-04-26T03:39:00.093 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.093 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:00.093 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.093 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:00.093 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.600125+0000", 2024-04-26T03:39:00.093 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.093 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.093 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.093 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:00.093 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.600125+0000", 2024-04-26T03:39:00.093 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.093 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.093 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.093 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:00.093 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.600126+0000", 2024-04-26T03:39:00.094 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.2100000000000002e-07 2024-04-26T03:39:00.094 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.094 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.094 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:00.094 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.600128+0000", 2024-04-26T03:39:00.094 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.5150000000000001e-06 2024-04-26T03:39:00.094 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.094 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.094 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:00.094 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.600133+0000", 2024-04-26T03:39:00.094 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.4240000000000001e-06 2024-04-26T03:39:00.094 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.094 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.094 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:00.095 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.600137+0000", 2024-04-26T03:39:00.095 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.4300000000000002e-06 2024-04-26T03:39:00.095 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.095 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.095 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.095 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.607830+0000", 2024-04-26T03:39:00.095 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0076931029999999997 2024-04-26T03:39:00.095 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.095 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.095 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:00.095 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.607888+0000", 2024-04-26T03:39:00.095 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.8260999999999997e-05 2024-04-26T03:39:00.095 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.095 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.095 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:00.096 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.608325+0000", 2024-04-26T03:39:00.096 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00043624700000000001 2024-04-26T03:39:00.096 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.096 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.096 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:00.096 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.608965+0000", 2024-04-26T03:39:00.096 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00064069400000000003 2024-04-26T03:39:00.096 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.096 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.096 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:00.096 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.618751+0000", 2024-04-26T03:39:00.096 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0097859209999999995 2024-04-26T03:39:00.096 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.096 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.096 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:00.097 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.618847+0000", 2024-04-26T03:39:00.097 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.5518000000000003e-05 2024-04-26T03:39:00.097 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.097 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:00.097 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.097 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.097 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.097 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:129 3.bs0 3:d339032c:::smithi09235267-43:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:00.097 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.600159+0000", 2024-04-26T03:39:00.097 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 8.4158886539999997, 2024-04-26T03:39:00.097 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.020601199000000001, 2024-04-26T03:39:00.097 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:00.097 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:00.097 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "started", 2024-04-26T03:39:00.098 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:00.098 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:00.098 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:00.098 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 129 2024-04-26T03:39:00.098 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.098 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:00.098 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.098 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:00.098 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.600159+0000", 2024-04-26T03:39:00.098 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.098 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.098 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.098 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:00.098 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.600159+0000", 2024-04-26T03:39:00.098 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.099 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.099 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.099 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:00.099 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.600159+0000", 2024-04-26T03:39:00.099 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.2000000000000001e-07 2024-04-26T03:39:00.099 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.099 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.099 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:00.099 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.600162+0000", 2024-04-26T03:39:00.099 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.3070000000000001e-06 2024-04-26T03:39:00.099 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.099 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.099 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:00.099 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.600170+0000", 2024-04-26T03:39:00.100 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.568e-06 2024-04-26T03:39:00.100 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.100 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.100 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:00.100 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.600174+0000", 2024-04-26T03:39:00.100 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.241e-06 2024-04-26T03:39:00.100 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.100 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.100 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.100 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.608991+0000", 2024-04-26T03:39:00.100 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0088175800000000002 2024-04-26T03:39:00.100 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.100 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.100 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:00.100 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.609072+0000", 2024-04-26T03:39:00.101 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.1075000000000006e-05 2024-04-26T03:39:00.101 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.101 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.101 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.101 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.618897+0000", 2024-04-26T03:39:00.101 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0098250460000000005 2024-04-26T03:39:00.101 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.101 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.101 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:00.101 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.618974+0000", 2024-04-26T03:39:00.101 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.7075000000000003e-05 2024-04-26T03:39:00.101 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.101 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.101 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:00.102 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.620760+0000", 2024-04-26T03:39:00.102 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0017859869999999999 2024-04-26T03:39:00.102 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.102 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:00.102 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.102 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.102 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.102 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:133 3.bs0 3:d66ffd47:::smithi09235267-45:head [append 0~753664,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:00.102 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.608848+0000", 2024-04-26T03:39:00.102 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 8.4072000340000006, 2024-04-26T03:39:00.102 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.010660602, 2024-04-26T03:39:00.102 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:00.102 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:00.102 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:00.103 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:00.103 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:00.103 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:00.103 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 133 2024-04-26T03:39:00.103 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.103 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:00.103 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.103 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:00.103 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.608848+0000", 2024-04-26T03:39:00.103 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.103 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.103 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.103 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:00.103 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.608848+0000", 2024-04-26T03:39:00.103 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.104 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.104 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.104 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:00.104 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.608849+0000", 2024-04-26T03:39:00.104 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.5799999999999999e-06 2024-04-26T03:39:00.104 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.104 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.104 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:00.104 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.609482+0000", 2024-04-26T03:39:00.104 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00063272800000000004 2024-04-26T03:39:00.104 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.104 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.104 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:00.104 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.609494+0000", 2024-04-26T03:39:00.104 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1562e-05 2024-04-26T03:39:00.105 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.105 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.105 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:00.105 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.609499+0000", 2024-04-26T03:39:00.105 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.5790000000000003e-06 2024-04-26T03:39:00.105 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.105 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.105 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.105 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.609557+0000", 2024-04-26T03:39:00.105 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.7346999999999998e-05 2024-04-26T03:39:00.105 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.105 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.105 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:00.105 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.609723+0000", 2024-04-26T03:39:00.106 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00016625099999999999 2024-04-26T03:39:00.106 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.106 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.106 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:00.106 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.610750+0000", 2024-04-26T03:39:00.106 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0010276619999999999 2024-04-26T03:39:00.106 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.106 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.106 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:00.106 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.619317+0000", 2024-04-26T03:39:00.106 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.008566789 2024-04-26T03:39:00.106 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.106 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.106 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:00.107 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.619445+0000", 2024-04-26T03:39:00.107 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00012778999999999999 2024-04-26T03:39:00.107 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.107 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.107 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:00.107 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.619508+0000", 2024-04-26T03:39:00.107 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.3313999999999997e-05 2024-04-26T03:39:00.107 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.107 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:00.107 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.107 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.107 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.107 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:134 3.bs0 3:d66ffd47:::smithi09235267-45:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:00.107 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.609513+0000", 2024-04-26T03:39:00.107 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 8.4065349119999997, 2024-04-26T03:39:00.108 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.010479423, 2024-04-26T03:39:00.108 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:00.108 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:00.108 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:00.108 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:00.108 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:00.108 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:00.108 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 134 2024-04-26T03:39:00.108 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.108 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:00.108 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.108 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:00.108 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.609513+0000", 2024-04-26T03:39:00.108 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.109 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.109 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.109 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:00.109 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.609513+0000", 2024-04-26T03:39:00.109 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.109 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.109 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.109 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:00.109 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.609513+0000", 2024-04-26T03:39:00.109 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.7399999999999998e-07 2024-04-26T03:39:00.109 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.109 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.109 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:00.109 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.609517+0000", 2024-04-26T03:39:00.109 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.2710000000000001e-06 2024-04-26T03:39:00.110 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.110 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.110 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:00.110 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.609529+0000", 2024-04-26T03:39:00.110 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.2879e-05 2024-04-26T03:39:00.110 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.110 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.110 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:00.110 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.609540+0000", 2024-04-26T03:39:00.110 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0054000000000001e-05 2024-04-26T03:39:00.110 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.110 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.110 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.110 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.617732+0000", 2024-04-26T03:39:00.110 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0081920719999999999 2024-04-26T03:39:00.111 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.111 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.111 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:00.111 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.617852+0000", 2024-04-26T03:39:00.111 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000120254 2024-04-26T03:39:00.111 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.111 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.111 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:00.111 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.618349+0000", 2024-04-26T03:39:00.111 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000497286 2024-04-26T03:39:00.111 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.111 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.111 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:00.111 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.619318+0000", 2024-04-26T03:39:00.111 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00096878800000000005 2024-04-26T03:39:00.112 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.112 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.112 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:00.112 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.619922+0000", 2024-04-26T03:39:00.112 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00060439200000000004 2024-04-26T03:39:00.112 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.112 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.112 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:00.112 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.619992+0000", 2024-04-26T03:39:00.112 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.9952999999999999e-05 2024-04-26T03:39:00.112 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.112 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:00.112 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.112 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.112 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.113 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:135 3.bs0 3:d66ffd47:::smithi09235267-45:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:00.113 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.609553+0000", 2024-04-26T03:39:00.113 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 8.4064947580000009, 2024-04-26T03:39:00.113 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.012333168, 2024-04-26T03:39:00.113 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:00.113 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:00.113 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "started", 2024-04-26T03:39:00.113 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:00.113 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:00.113 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:00.113 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 135 2024-04-26T03:39:00.113 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.113 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:00.113 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.114 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:00.114 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.609553+0000", 2024-04-26T03:39:00.114 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.114 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.114 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.114 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:00.114 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.609553+0000", 2024-04-26T03:39:00.114 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.114 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.114 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.114 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:00.114 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.609554+0000", 2024-04-26T03:39:00.114 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.7400000000000002e-07 2024-04-26T03:39:00.114 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.114 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.115 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:00.115 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.609558+0000", 2024-04-26T03:39:00.115 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.6110000000000004e-06 2024-04-26T03:39:00.115 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.115 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.115 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:00.115 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.609567+0000", 2024-04-26T03:39:00.115 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.6589999999999996e-06 2024-04-26T03:39:00.115 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.115 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.115 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:00.115 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.609573+0000", 2024-04-26T03:39:00.115 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.7370000000000003e-06 2024-04-26T03:39:00.115 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.115 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.116 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.116 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.620811+0000", 2024-04-26T03:39:00.116 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.011238689 2024-04-26T03:39:00.116 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.116 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.116 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:00.116 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.620864+0000", 2024-04-26T03:39:00.116 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.2441000000000002e-05 2024-04-26T03:39:00.116 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.116 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.116 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:00.116 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.621886+0000", 2024-04-26T03:39:00.116 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0010222569999999999 2024-04-26T03:39:00.116 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.116 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:00.117 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.117 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.117 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:00.117 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:39:00.118 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-04-26T03:39:00.119 INFO:tasks.ceph.osd.12.smithi028.stderr:2024-04-26T03:39:00.117+0000 7f114fe14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 31967) UID: 0 2024-04-26T03:39:00.218 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:39:00.217+0000 7fba0d2ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31961) UID: 0 2024-04-26T03:39:00.267 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:39:00.267 INFO:teuthology.orchestra.run.smithi028.stdout: "ops": [], 2024-04-26T03:39:00.267 INFO:teuthology.orchestra.run.smithi028.stdout: "num_ops": 0 2024-04-26T03:39:00.267 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:39:00.275 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-04-26T03:39:00.322 INFO:tasks.ceph.osd.10.smithi066.stderr:2024-04-26T03:39:00.318+0000 7f130fcc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 31763) UID: 0 2024-04-26T03:39:00.420 INFO:tasks.ceph.osd.9.smithi042.stderr:2024-04-26T03:39:00.419+0000 7fdac63b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 31960) UID: 0 2024-04-26T03:39:00.423 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:39:00.423 INFO:teuthology.orchestra.run.smithi028.stdout: "ops": [], 2024-04-26T03:39:00.423 INFO:teuthology.orchestra.run.smithi028.stdout: "complaint_time": 30, 2024-04-26T03:39:00.423 INFO:teuthology.orchestra.run.smithi028.stdout: "num_blocked_ops": 0 2024-04-26T03:39:00.423 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:39:00.432 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-04-26T03:39:00.521 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-04-26T03:39:00.519+0000 7f745118d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 31821) UID: 0 2024-04-26T03:39:00.528 INFO:tasks.rados.rados.0.smithi092.stdout:121: done (4 left) 2024-04-26T03:39:00.528 INFO:tasks.rados.rados.0.smithi092.stdout:123: done (3 left) 2024-04-26T03:39:00.528 INFO:tasks.rados.rados.0.smithi092.stdout:124: done (2 left) 2024-04-26T03:39:00.529 INFO:tasks.rados.rados.0.smithi092.stdout:125: done (1 left) 2024-04-26T03:39:00.529 INFO:tasks.rados.rados.0.smithi092.stdout:126: done (0 left) 2024-04-26T03:39:00.529 INFO:tasks.rados.rados.0.smithi092.stdout:127: delete oid 23 current snap is 6 2024-04-26T03:39:00.549 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:39:00.549 INFO:teuthology.orchestra.run.smithi028.stdout: "size": 20, 2024-04-26T03:39:00.549 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 600, 2024-04-26T03:39:00.549 INFO:teuthology.orchestra.run.smithi028.stdout: "ops": [ 2024-04-26T03:39:00.549 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.549 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:34 3.4s0 3:2ee2abb1:::smithi09235267-12:head [append 0~196608,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:00.549 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.494587+0000", 2024-04-26T03:39:00.550 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 9.0517673520000006, 2024-04-26T03:39:00.550 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.014012205, 2024-04-26T03:39:00.550 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:00.550 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:00.550 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:00.550 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:00.550 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:00.550 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:00.550 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 34 2024-04-26T03:39:00.550 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.550 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:00.550 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.550 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:00.550 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.494587+0000", 2024-04-26T03:39:00.551 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.551 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.551 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.551 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:00.551 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.494587+0000", 2024-04-26T03:39:00.551 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.551 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.551 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.551 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:00.551 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.494589+0000", 2024-04-26T03:39:00.551 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.3530000000000002e-06 2024-04-26T03:39:00.551 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.552 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.552 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:00.552 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.494944+0000", 2024-04-26T03:39:00.552 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00035535200000000002 2024-04-26T03:39:00.552 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.552 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.552 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:00.552 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.494963+0000", 2024-04-26T03:39:00.552 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.8745000000000001e-05 2024-04-26T03:39:00.552 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.552 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.552 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:00.552 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.494972+0000", 2024-04-26T03:39:00.552 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.0629999999999995e-06 2024-04-26T03:39:00.552 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.553 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.553 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.553 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495054+0000", 2024-04-26T03:39:00.553 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.1810000000000004e-05 2024-04-26T03:39:00.553 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.553 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.553 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:00.553 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495184+0000", 2024-04-26T03:39:00.553 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00012965399999999999 2024-04-26T03:39:00.553 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.553 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.553 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:00.553 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495943+0000", 2024-04-26T03:39:00.553 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00075938199999999996 2024-04-26T03:39:00.553 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.554 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.554 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:00.554 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.500314+0000", 2024-04-26T03:39:00.554 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0043711669999999996 2024-04-26T03:39:00.554 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.554 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.554 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:00.554 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.508524+0000", 2024-04-26T03:39:00.554 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0082094520000000008 2024-04-26T03:39:00.554 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.554 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.554 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:00.554 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.508599+0000", 2024-04-26T03:39:00.555 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.5227000000000003e-05 2024-04-26T03:39:00.555 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.555 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:00.555 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.555 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.555 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.555 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:35 3.4s0 3:2ee2abb1:::smithi09235267-12:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:00.555 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.495003+0000", 2024-04-26T03:39:00.555 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 9.0513511050000002, 2024-04-26T03:39:00.555 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.013766825999999999, 2024-04-26T03:39:00.555 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:00.555 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:00.555 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:00.555 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:00.555 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:00.555 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:00.556 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 35 2024-04-26T03:39:00.556 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.556 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:00.556 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.556 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:00.556 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495003+0000", 2024-04-26T03:39:00.556 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.556 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.556 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.556 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:00.556 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495003+0000", 2024-04-26T03:39:00.556 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.556 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.556 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.557 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:00.557 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495004+0000", 2024-04-26T03:39:00.557 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.9899999999999999e-07 2024-04-26T03:39:00.557 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.557 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.557 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:00.557 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495008+0000", 2024-04-26T03:39:00.557 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.3499999999999999e-06 2024-04-26T03:39:00.557 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.557 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.558 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:00.558 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495017+0000", 2024-04-26T03:39:00.558 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.3219999999999999e-06 2024-04-26T03:39:00.558 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.558 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.558 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:00.558 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495025+0000", 2024-04-26T03:39:00.558 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.7049999999999999e-06 2024-04-26T03:39:00.558 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.558 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.558 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.558 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.498423+0000", 2024-04-26T03:39:00.558 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0033980220000000001 2024-04-26T03:39:00.558 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.559 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.559 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:00.559 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.498519+0000", 2024-04-26T03:39:00.559 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.6070999999999995e-05 2024-04-26T03:39:00.559 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.559 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.559 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:00.559 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.499008+0000", 2024-04-26T03:39:00.559 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000488982 2024-04-26T03:39:00.559 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.559 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.559 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:00.559 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.500315+0000", 2024-04-26T03:39:00.559 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00130697 2024-04-26T03:39:00.560 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.560 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.560 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:00.560 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.508694+0000", 2024-04-26T03:39:00.560 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0083790870000000003 2024-04-26T03:39:00.560 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.560 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.560 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:00.560 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.508770+0000", 2024-04-26T03:39:00.560 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.5518000000000004e-05 2024-04-26T03:39:00.560 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.560 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:00.560 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.560 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.560 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.561 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:36 3.4s0 3:2ee2abb1:::smithi09235267-12:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:00.561 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.495035+0000", 2024-04-26T03:39:00.561 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 9.0513185279999995, 2024-04-26T03:39:00.561 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.018906236, 2024-04-26T03:39:00.561 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:00.561 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:00.561 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "started", 2024-04-26T03:39:00.561 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:00.561 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:00.561 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:00.561 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 36 2024-04-26T03:39:00.561 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.561 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:00.562 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.562 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:00.562 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495035+0000", 2024-04-26T03:39:00.562 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.562 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.562 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.562 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:00.562 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495035+0000", 2024-04-26T03:39:00.562 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.562 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.562 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.562 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:00.563 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495036+0000", 2024-04-26T03:39:00.563 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.5199999999999997e-07 2024-04-26T03:39:00.563 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.563 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.563 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:00.563 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495039+0000", 2024-04-26T03:39:00.563 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.5770000000000001e-06 2024-04-26T03:39:00.563 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.563 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.563 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:00.563 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495045+0000", 2024-04-26T03:39:00.563 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.8199999999999999e-06 2024-04-26T03:39:00.563 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.563 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.564 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:00.564 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495050+0000", 2024-04-26T03:39:00.564 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.8520000000000003e-06 2024-04-26T03:39:00.564 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.564 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.564 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.564 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.500349+0000", 2024-04-26T03:39:00.564 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0052990069999999997 2024-04-26T03:39:00.564 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.564 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.564 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:00.564 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.500436+0000", 2024-04-26T03:39:00.564 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.6327000000000002e-05 2024-04-26T03:39:00.564 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.565 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.565 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.565 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.508818+0000", 2024-04-26T03:39:00.565 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0083819900000000006 2024-04-26T03:39:00.565 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.565 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.565 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:00.565 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.508897+0000", 2024-04-26T03:39:00.565 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.9580999999999996e-05 2024-04-26T03:39:00.565 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.565 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.565 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:00.565 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.513942+0000", 2024-04-26T03:39:00.565 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0050445300000000002 2024-04-26T03:39:00.565 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.566 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:00.566 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.567 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.567 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.567 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:49 3.4s0 3:287ad008:::smithi09235267-17:head [append 0~278528,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:00.567 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.503026+0000", 2024-04-26T03:39:00.567 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 9.0433277719999996, 2024-04-26T03:39:00.567 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0096820680000000003, 2024-04-26T03:39:00.567 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:00.567 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:00.567 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:00.567 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:00.567 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:00.567 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:00.568 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 49 2024-04-26T03:39:00.568 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.568 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:00.568 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.568 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:00.568 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.503026+0000", 2024-04-26T03:39:00.568 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.568 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.568 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.568 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:00.568 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.503026+0000", 2024-04-26T03:39:00.568 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.568 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.569 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.569 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:00.569 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.503028+0000", 2024-04-26T03:39:00.569 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.1849999999999998e-06 2024-04-26T03:39:00.569 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.569 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.569 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:00.569 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.503400+0000", 2024-04-26T03:39:00.569 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00037169500000000001 2024-04-26T03:39:00.569 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.569 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.569 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:00.569 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.503414+0000", 2024-04-26T03:39:00.569 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.4168999999999999e-05 2024-04-26T03:39:00.570 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.570 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.570 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:00.570 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.503422+0000", 2024-04-26T03:39:00.570 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.8699999999999992e-06 2024-04-26T03:39:00.570 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.570 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.570 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.570 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.503621+0000", 2024-04-26T03:39:00.570 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0001985 2024-04-26T03:39:00.570 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.570 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.570 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:00.571 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.503775+0000", 2024-04-26T03:39:00.571 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00015452700000000001 2024-04-26T03:39:00.571 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.571 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.571 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:00.571 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.504517+0000", 2024-04-26T03:39:00.571 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00074210400000000005 2024-04-26T03:39:00.571 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.571 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.571 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:00.571 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.509965+0000", 2024-04-26T03:39:00.571 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0054473270000000001 2024-04-26T03:39:00.571 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.571 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.572 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:00.572 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.512650+0000", 2024-04-26T03:39:00.572 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0026857640000000002 2024-04-26T03:39:00.572 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.572 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.572 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:00.572 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.512708+0000", 2024-04-26T03:39:00.572 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.7927000000000003e-05 2024-04-26T03:39:00.572 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.572 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:00.572 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.572 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.572 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.572 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:50 3.4s0 3:287ad008:::smithi09235267-17:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:00.573 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.503442+0000", 2024-04-26T03:39:00.573 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 9.042911492, 2024-04-26T03:39:00.573 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0095066019999999994, 2024-04-26T03:39:00.573 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:00.573 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:00.573 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:00.573 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:00.573 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:00.573 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:00.573 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 50 2024-04-26T03:39:00.573 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.573 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:00.573 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.573 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:00.574 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.503442+0000", 2024-04-26T03:39:00.574 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.574 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.574 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.574 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:00.574 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.503442+0000", 2024-04-26T03:39:00.574 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.574 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.574 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.574 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:00.574 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.503443+0000", 2024-04-26T03:39:00.574 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.8299999999999996e-07 2024-04-26T03:39:00.574 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.574 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.575 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:00.575 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.503448+0000", 2024-04-26T03:39:00.575 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.169e-06 2024-04-26T03:39:00.575 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.575 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.575 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:00.575 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.503456+0000", 2024-04-26T03:39:00.575 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.3459999999999996e-06 2024-04-26T03:39:00.575 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.575 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.575 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:00.575 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.503461+0000", 2024-04-26T03:39:00.575 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.6409999999999999e-06 2024-04-26T03:39:00.575 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.575 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.576 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.576 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.507657+0000", 2024-04-26T03:39:00.576 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0041960189999999996 2024-04-26T03:39:00.576 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.576 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.576 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:00.576 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.507745+0000", 2024-04-26T03:39:00.576 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.7113000000000004e-05 2024-04-26T03:39:00.576 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.576 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.576 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:00.576 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.508206+0000", 2024-04-26T03:39:00.576 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000461352 2024-04-26T03:39:00.576 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.577 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.577 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:00.577 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.509966+0000", 2024-04-26T03:39:00.577 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0017604179999999999 2024-04-26T03:39:00.577 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.577 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.577 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:00.577 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.512843+0000", 2024-04-26T03:39:00.577 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0028768499999999998 2024-04-26T03:39:00.577 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.577 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.577 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:00.577 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.512949+0000", 2024-04-26T03:39:00.577 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000105911 2024-04-26T03:39:00.577 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.578 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:00.578 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.578 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.578 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.578 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:51 3.4s0 3:287ad008:::smithi09235267-17:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:00.578 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.503471+0000", 2024-04-26T03:39:00.578 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 9.0428827040000002, 2024-04-26T03:39:00.578 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.013832507000000001, 2024-04-26T03:39:00.578 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:00.578 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:00.578 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "started", 2024-04-26T03:39:00.578 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:00.579 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:00.579 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:00.579 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 51 2024-04-26T03:39:00.579 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.579 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:00.579 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.579 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:00.579 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.503471+0000", 2024-04-26T03:39:00.579 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.579 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.579 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.579 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:00.579 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.503471+0000", 2024-04-26T03:39:00.579 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.580 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.580 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.580 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:00.580 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.503472+0000", 2024-04-26T03:39:00.580 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.82e-07 2024-04-26T03:39:00.580 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.580 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.580 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:00.580 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.503475+0000", 2024-04-26T03:39:00.580 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.033e-06 2024-04-26T03:39:00.580 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.580 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.580 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:00.581 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.503482+0000", 2024-04-26T03:39:00.581 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.4900000000000003e-06 2024-04-26T03:39:00.581 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.581 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.581 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:00.581 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.503488+0000", 2024-04-26T03:39:00.581 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.0179999999999998e-06 2024-04-26T03:39:00.581 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.581 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.581 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.581 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.509711+0000", 2024-04-26T03:39:00.581 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0062223959999999998 2024-04-26T03:39:00.581 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.582 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.582 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:00.582 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.509883+0000", 2024-04-26T03:39:00.582 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00017213600000000001 2024-04-26T03:39:00.582 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.582 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.582 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.582 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.513351+0000", 2024-04-26T03:39:00.582 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.003467908 2024-04-26T03:39:00.582 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.582 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.582 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:00.583 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.513431+0000", 2024-04-26T03:39:00.583 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.9917000000000001e-05 2024-04-26T03:39:00.583 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.583 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.583 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:00.583 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.517304+0000", 2024-04-26T03:39:00.583 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0038730269999999998 2024-04-26T03:39:00.583 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.583 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:00.583 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.583 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.583 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.584 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:83 3.4s0 3:2d740be2:::smithi09235267-28:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:00.584 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.537691+0000", 2024-04-26T03:39:00.584 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 9.0086625280000003, 2024-04-26T03:39:00.584 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.010715817000000001, 2024-04-26T03:39:00.584 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:00.584 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:00.584 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:00.584 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:00.584 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:00.584 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:00.584 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 83 2024-04-26T03:39:00.584 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.584 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:00.585 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.585 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:00.585 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.537691+0000", 2024-04-26T03:39:00.585 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.585 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.585 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.585 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:00.585 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.537691+0000", 2024-04-26T03:39:00.585 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.585 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.585 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.585 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:00.585 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.537692+0000", 2024-04-26T03:39:00.585 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.0699999999999997e-07 2024-04-26T03:39:00.586 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.586 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.586 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:00.586 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.537695+0000", 2024-04-26T03:39:00.586 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.706e-06 2024-04-26T03:39:00.586 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.586 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.586 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:00.586 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.537699+0000", 2024-04-26T03:39:00.586 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.6600000000000003e-06 2024-04-26T03:39:00.586 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.586 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.586 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:00.587 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.537705+0000", 2024-04-26T03:39:00.587 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.1329999999999998e-06 2024-04-26T03:39:00.587 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.587 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.587 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.587 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.543951+0000", 2024-04-26T03:39:00.587 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0062463750000000002 2024-04-26T03:39:00.587 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.587 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.587 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:00.587 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.544010+0000", 2024-04-26T03:39:00.587 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.8950000000000003e-05 2024-04-26T03:39:00.587 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.587 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.587 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:00.588 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.544384+0000", 2024-04-26T03:39:00.588 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000374562 2024-04-26T03:39:00.588 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.588 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.588 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:00.588 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.545050+0000", 2024-04-26T03:39:00.588 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00066610300000000003 2024-04-26T03:39:00.588 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.588 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.588 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:00.588 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.548340+0000", 2024-04-26T03:39:00.588 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0032892450000000001 2024-04-26T03:39:00.588 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.588 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.589 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:00.589 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.548407+0000", 2024-04-26T03:39:00.589 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.7476000000000002e-05 2024-04-26T03:39:00.589 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.589 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:00.589 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.589 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.589 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.589 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:84 3.4s0 3:2d740be2:::smithi09235267-28:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:00.589 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.537728+0000", 2024-04-26T03:39:00.589 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 9.0086263819999992, 2024-04-26T03:39:00.589 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.012278776999999999, 2024-04-26T03:39:00.590 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:00.590 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:00.590 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "started", 2024-04-26T03:39:00.590 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:00.590 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:00.590 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:00.590 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 84 2024-04-26T03:39:00.590 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.590 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:00.590 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.590 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:00.590 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.537728+0000", 2024-04-26T03:39:00.590 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.590 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.591 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.591 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:00.591 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.537728+0000", 2024-04-26T03:39:00.591 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.591 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.591 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.591 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:00.591 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.537728+0000", 2024-04-26T03:39:00.591 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.8999999999999997e-07 2024-04-26T03:39:00.591 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.591 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.591 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:00.591 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.537731+0000", 2024-04-26T03:39:00.591 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.6299999999999998e-06 2024-04-26T03:39:00.592 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.592 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.592 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:00.592 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.537743+0000", 2024-04-26T03:39:00.592 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.2449999999999999e-05 2024-04-26T03:39:00.592 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.592 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.592 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:00.592 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.537750+0000", 2024-04-26T03:39:00.592 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.7839999999999998e-06 2024-04-26T03:39:00.592 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.592 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.592 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.593 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.544950+0000", 2024-04-26T03:39:00.593 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.007199754 2024-04-26T03:39:00.593 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.593 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.593 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:00.593 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.545037+0000", 2024-04-26T03:39:00.593 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.7668000000000002e-05 2024-04-26T03:39:00.593 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.593 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.593 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.593 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.548678+0000", 2024-04-26T03:39:00.593 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.003640524 2024-04-26T03:39:00.593 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.594 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.594 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:00.594 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.548759+0000", 2024-04-26T03:39:00.594 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.0827000000000004e-05 2024-04-26T03:39:00.594 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.594 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.594 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:00.594 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.550006+0000", 2024-04-26T03:39:00.594 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0012476500000000001 2024-04-26T03:39:00.594 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.594 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:00.594 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.594 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.595 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.595 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:100 3.4s0 3:2869fefd:::smithi09235267-34:head [append 0~557056,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:00.595 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.558706+0000", 2024-04-26T03:39:00.595 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 8.9876474729999991, 2024-04-26T03:39:00.595 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.013658327, 2024-04-26T03:39:00.595 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:00.595 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:00.595 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:00.595 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:00.595 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:00.595 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:00.595 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 100 2024-04-26T03:39:00.595 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.596 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:00.596 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.596 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:00.596 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.558706+0000", 2024-04-26T03:39:00.596 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.596 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.596 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.596 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:00.596 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.558706+0000", 2024-04-26T03:39:00.596 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.596 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.596 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.596 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:00.596 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.558708+0000", 2024-04-26T03:39:00.597 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.9290000000000001e-06 2024-04-26T03:39:00.597 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.597 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.597 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:00.597 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.559223+0000", 2024-04-26T03:39:00.597 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00051454199999999999 2024-04-26T03:39:00.597 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.597 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.597 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:00.597 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.559241+0000", 2024-04-26T03:39:00.597 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.7972e-05 2024-04-26T03:39:00.597 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.597 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.598 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:00.598 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.559258+0000", 2024-04-26T03:39:00.598 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.7487999999999999e-05 2024-04-26T03:39:00.598 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.598 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.598 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.598 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.559321+0000", 2024-04-26T03:39:00.598 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.2799000000000006e-05 2024-04-26T03:39:00.598 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.598 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.598 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:00.598 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.559480+0000", 2024-04-26T03:39:00.598 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00015929799999999999 2024-04-26T03:39:00.599 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.599 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.599 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:00.599 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.560384+0000", 2024-04-26T03:39:00.599 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00090365499999999995 2024-04-26T03:39:00.599 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.599 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.599 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:00.599 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.564827+0000", 2024-04-26T03:39:00.599 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0044424099999999999 2024-04-26T03:39:00.599 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.599 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.599 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:00.599 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.572305+0000", 2024-04-26T03:39:00.600 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0074782540000000002 2024-04-26T03:39:00.600 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.600 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.600 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:00.600 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.572365+0000", 2024-04-26T03:39:00.600 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.9979999999999998e-05 2024-04-26T03:39:00.600 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.600 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:00.600 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.600 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.600 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.600 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:101 3.4s0 3:2869fefd:::smithi09235267-34:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:00.600 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.559280+0000", 2024-04-26T03:39:00.601 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 8.9870738729999999, 2024-04-26T03:39:00.601 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.015441082, 2024-04-26T03:39:00.601 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:00.601 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:00.601 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:00.601 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:00.601 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:00.601 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:00.601 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 101 2024-04-26T03:39:00.601 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.601 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:00.602 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.602 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:00.602 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.559280+0000", 2024-04-26T03:39:00.602 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.602 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.602 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.602 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:00.602 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.559280+0000", 2024-04-26T03:39:00.602 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.602 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.602 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.602 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:00.602 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.559281+0000", 2024-04-26T03:39:00.602 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.4600000000000004e-07 2024-04-26T03:39:00.603 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.603 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.603 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:00.603 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.559285+0000", 2024-04-26T03:39:00.603 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.5129999999999998e-06 2024-04-26T03:39:00.603 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.603 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.603 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:00.603 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.559294+0000", 2024-04-26T03:39:00.603 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.0059999999999998e-06 2024-04-26T03:39:00.603 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.603 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.603 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:00.603 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.559303+0000", 2024-04-26T03:39:00.603 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.1999999999999994e-06 2024-04-26T03:39:00.604 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.604 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.604 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.604 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.564299+0000", 2024-04-26T03:39:00.604 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0049963150000000003 2024-04-26T03:39:00.604 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.604 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.604 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:00.604 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.564369+0000", 2024-04-26T03:39:00.604 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.0238000000000001e-05 2024-04-26T03:39:00.604 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.604 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.604 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:00.604 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.564677+0000", 2024-04-26T03:39:00.604 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00030746099999999998 2024-04-26T03:39:00.605 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.605 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.605 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:00.605 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.565235+0000", 2024-04-26T03:39:00.605 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00055854500000000003 2024-04-26T03:39:00.605 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.605 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.605 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:00.605 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.574673+0000", 2024-04-26T03:39:00.605 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0094382830000000004 2024-04-26T03:39:00.605 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.605 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.605 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:00.605 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.574721+0000", 2024-04-26T03:39:00.606 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.7775000000000003e-05 2024-04-26T03:39:00.606 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.606 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:00.606 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.606 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.606 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.606 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:102 3.4s0 3:2869fefd:::smithi09235267-34:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:00.606 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.559325+0000", 2024-04-26T03:39:00.606 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 8.9870291939999998, 2024-04-26T03:39:00.606 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.017964909000000001, 2024-04-26T03:39:00.606 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:00.606 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:00.606 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "started", 2024-04-26T03:39:00.607 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:00.607 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:00.607 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:00.607 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 102 2024-04-26T03:39:00.607 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.607 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:00.607 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.607 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:00.607 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.559325+0000", 2024-04-26T03:39:00.607 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.607 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.607 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.608 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:00.608 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.559325+0000", 2024-04-26T03:39:00.608 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.608 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.608 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.608 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:00.608 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.559325+0000", 2024-04-26T03:39:00.608 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.2900000000000003e-07 2024-04-26T03:39:00.608 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.608 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.608 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:00.608 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.559330+0000", 2024-04-26T03:39:00.608 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.7029999999999997e-06 2024-04-26T03:39:00.608 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.609 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.609 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:00.609 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.559339+0000", 2024-04-26T03:39:00.609 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.8969999999999997e-06 2024-04-26T03:39:00.609 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.609 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.609 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:00.609 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.559346+0000", 2024-04-26T03:39:00.609 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.8560000000000001e-06 2024-04-26T03:39:00.609 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.609 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.609 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.609 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.564881+0000", 2024-04-26T03:39:00.609 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0055355830000000002 2024-04-26T03:39:00.609 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.610 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.610 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:00.610 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.564965+0000", 2024-04-26T03:39:00.610 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.3248e-05 2024-04-26T03:39:00.610 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.610 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.610 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.610 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.574764+0000", 2024-04-26T03:39:00.610 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0097988769999999992 2024-04-26T03:39:00.610 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.610 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.610 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:00.610 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.574811+0000", 2024-04-26T03:39:00.611 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.7466999999999998e-05 2024-04-26T03:39:00.611 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.611 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.611 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:00.611 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.577290+0000", 2024-04-26T03:39:00.611 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0024786489999999999 2024-04-26T03:39:00.611 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.611 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:00.611 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.611 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.611 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.611 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:106 3.4s0 3:2aba41c0:::smithi09235267-36:head [append 0~589824,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:00.611 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.565818+0000", 2024-04-26T03:39:00.611 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 8.9805356429999996, 2024-04-26T03:39:00.611 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.009397987, 2024-04-26T03:39:00.612 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:00.612 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:00.612 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:00.612 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:00.612 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:00.612 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:00.612 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 106 2024-04-26T03:39:00.612 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.612 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:00.612 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.612 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:00.612 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.565818+0000", 2024-04-26T03:39:00.612 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.612 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.613 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.613 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:00.613 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.565818+0000", 2024-04-26T03:39:00.613 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.613 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.613 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.613 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:00.613 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.565819+0000", 2024-04-26T03:39:00.613 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0160000000000001e-06 2024-04-26T03:39:00.613 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.613 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.613 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:00.613 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.566371+0000", 2024-04-26T03:39:00.613 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00055153000000000005 2024-04-26T03:39:00.614 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.614 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.614 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:00.614 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.566381+0000", 2024-04-26T03:39:00.614 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0159999999999999e-05 2024-04-26T03:39:00.614 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.614 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.614 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:00.614 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.566386+0000", 2024-04-26T03:39:00.614 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.5759999999999999e-06 2024-04-26T03:39:00.614 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.614 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.614 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.614 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.566416+0000", 2024-04-26T03:39:00.615 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.0528000000000003e-05 2024-04-26T03:39:00.615 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.615 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.615 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:00.615 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.566499+0000", 2024-04-26T03:39:00.615 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.3220999999999995e-05 2024-04-26T03:39:00.615 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.615 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.615 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:00.615 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.567063+0000", 2024-04-26T03:39:00.615 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00056343400000000003 2024-04-26T03:39:00.615 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.615 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.615 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:00.616 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.571224+0000", 2024-04-26T03:39:00.616 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.004161021 2024-04-26T03:39:00.616 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.616 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.616 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:00.616 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.575159+0000", 2024-04-26T03:39:00.616 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0039354489999999997 2024-04-26T03:39:00.616 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.616 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.616 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:00.616 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.575216+0000", 2024-04-26T03:39:00.616 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.7052000000000002e-05 2024-04-26T03:39:00.616 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.616 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:00.616 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.617 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.617 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.617 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:107 3.4s0 3:2aba41c0:::smithi09235267-36:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:00.617 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.566396+0000", 2024-04-26T03:39:00.617 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 8.9799581659999994, 2024-04-26T03:39:00.617 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0091045770000000008, 2024-04-26T03:39:00.617 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:00.617 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:00.617 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:00.617 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:00.617 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:00.617 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:00.617 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 107 2024-04-26T03:39:00.617 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.618 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:00.618 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.618 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:00.618 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.566396+0000", 2024-04-26T03:39:00.618 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.618 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.618 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.618 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:00.618 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.566396+0000", 2024-04-26T03:39:00.618 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.618 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.618 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.618 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:00.618 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.566397+0000", 2024-04-26T03:39:00.618 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.8700000000000004e-07 2024-04-26T03:39:00.619 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.619 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.619 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:00.619 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.566400+0000", 2024-04-26T03:39:00.619 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.3720000000000001e-06 2024-04-26T03:39:00.619 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.619 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.619 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:00.619 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.566407+0000", 2024-04-26T03:39:00.619 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.5390000000000004e-06 2024-04-26T03:39:00.619 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.619 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.619 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:00.619 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.566412+0000", 2024-04-26T03:39:00.620 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.2220000000000001e-06 2024-04-26T03:39:00.620 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.620 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.620 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.620 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.570514+0000", 2024-04-26T03:39:00.620 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0041026109999999999 2024-04-26T03:39:00.620 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.620 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.620 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:00.620 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.570567+0000", 2024-04-26T03:39:00.620 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.2834000000000003e-05 2024-04-26T03:39:00.620 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.620 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.620 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:00.621 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.570955+0000", 2024-04-26T03:39:00.621 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00038789800000000002 2024-04-26T03:39:00.621 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.621 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.621 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:00.621 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.571468+0000", 2024-04-26T03:39:00.621 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00051256000000000001 2024-04-26T03:39:00.621 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.621 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.621 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:00.621 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.575409+0000", 2024-04-26T03:39:00.621 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0039416449999999997 2024-04-26T03:39:00.621 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.621 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.621 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:00.622 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.575500+0000", 2024-04-26T03:39:00.622 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.1008999999999995e-05 2024-04-26T03:39:00.622 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.622 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:00.622 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.622 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.622 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.622 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:108 3.4s0 3:2aba41c0:::smithi09235267-36:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:00.622 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.566421+0000", 2024-04-26T03:39:00.622 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 8.9799329019999998, 2024-04-26T03:39:00.623 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.011131057, 2024-04-26T03:39:00.623 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:00.623 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:00.623 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "started", 2024-04-26T03:39:00.623 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:00.623 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:00.623 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:00.623 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 108 2024-04-26T03:39:00.623 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.623 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:00.623 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.623 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:00.623 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.566421+0000", 2024-04-26T03:39:00.623 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.624 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.624 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.624 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:00.624 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.566421+0000", 2024-04-26T03:39:00.624 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.624 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.624 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.624 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:00.624 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.566422+0000", 2024-04-26T03:39:00.624 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.0299999999999998e-07 2024-04-26T03:39:00.624 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.624 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.624 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:00.624 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.566425+0000", 2024-04-26T03:39:00.624 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.2179999999999999e-06 2024-04-26T03:39:00.625 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.625 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.625 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:00.625 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.566432+0000", 2024-04-26T03:39:00.625 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.6889999999999998e-06 2024-04-26T03:39:00.625 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.625 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.625 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:00.625 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.566436+0000", 2024-04-26T03:39:00.625 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.0629999999999999e-06 2024-04-26T03:39:00.625 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.625 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.625 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.625 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.571133+0000", 2024-04-26T03:39:00.626 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0046974829999999997 2024-04-26T03:39:00.626 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.626 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.626 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:00.626 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.571215+0000", 2024-04-26T03:39:00.626 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.1749000000000006e-05 2024-04-26T03:39:00.626 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.626 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.626 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.626 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.575872+0000", 2024-04-26T03:39:00.626 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0046567930000000002 2024-04-26T03:39:00.626 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.626 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.627 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:00.627 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.575918+0000", 2024-04-26T03:39:00.627 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.5970000000000002e-05 2024-04-26T03:39:00.627 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.627 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.627 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:00.627 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.577552+0000", 2024-04-26T03:39:00.627 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.001634389 2024-04-26T03:39:00.627 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.627 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:00.627 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.627 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.627 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.627 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:197 3.4s0 3:2aba41c0:::smithi09235267-36:head [copy-from ver 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:00.627 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.753505+0000", 2024-04-26T03:39:00.627 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 8.7928487579999999, 2024-04-26T03:39:00.628 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0096165839999999992, 2024-04-26T03:39:00.628 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:00.628 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:00.628 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:00.628 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:00.628 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:00.628 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:00.628 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 197 2024-04-26T03:39:00.628 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.628 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:00.628 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.628 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:00.628 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.753505+0000", 2024-04-26T03:39:00.628 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.629 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.629 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.629 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:00.629 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.753505+0000", 2024-04-26T03:39:00.629 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.629 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.629 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.629 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:00.629 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.753506+0000", 2024-04-26T03:39:00.629 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.7600000000000006e-07 2024-04-26T03:39:00.629 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.629 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.629 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:00.629 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.753511+0000", 2024-04-26T03:39:00.629 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.42e-06 2024-04-26T03:39:00.630 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.630 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.630 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:00.630 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.753523+0000", 2024-04-26T03:39:00.630 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.2119999999999999e-05 2024-04-26T03:39:00.630 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.630 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.630 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:00.630 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.753530+0000", 2024-04-26T03:39:00.630 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.6329999999999996e-06 2024-04-26T03:39:00.630 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.630 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.630 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.630 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.753662+0000", 2024-04-26T03:39:00.631 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00013192600000000001 2024-04-26T03:39:00.631 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.631 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.631 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:00.631 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.753768+0000", 2024-04-26T03:39:00.631 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000106118 2024-04-26T03:39:00.631 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.631 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.631 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:00.631 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.758068+0000", 2024-04-26T03:39:00.631 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0042996550000000003 2024-04-26T03:39:00.631 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.631 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.631 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:00.631 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.762911+0000", 2024-04-26T03:39:00.632 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0048426650000000003 2024-04-26T03:39:00.632 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.632 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.632 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:00.632 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.763039+0000", 2024-04-26T03:39:00.632 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00012850700000000001 2024-04-26T03:39:00.632 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.632 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.632 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:00.632 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.763122+0000", 2024-04-26T03:39:00.632 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.2564000000000004e-05 2024-04-26T03:39:00.632 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.632 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:00.632 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.632 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.633 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.633 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:198 3.4s0 3:2aba41c0:::smithi09235267-36:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:00.633 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.753550+0000", 2024-04-26T03:39:00.633 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 8.7928036380000005, 2024-04-26T03:39:00.633 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0092093060000000004, 2024-04-26T03:39:00.633 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:00.633 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:00.633 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "started", 2024-04-26T03:39:00.633 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:00.633 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:00.633 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:00.633 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 198 2024-04-26T03:39:00.633 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.633 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:00.634 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.634 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:00.634 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.753550+0000", 2024-04-26T03:39:00.634 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.634 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.634 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.634 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:00.634 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.753550+0000", 2024-04-26T03:39:00.634 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.634 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.634 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.634 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:00.634 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.753551+0000", 2024-04-26T03:39:00.635 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.5899999999999996e-07 2024-04-26T03:39:00.635 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.635 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.635 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:00.635 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.753554+0000", 2024-04-26T03:39:00.635 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.2200000000000001e-06 2024-04-26T03:39:00.635 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.635 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.635 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:00.635 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.753563+0000", 2024-04-26T03:39:00.635 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.5560000000000008e-06 2024-04-26T03:39:00.635 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.635 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.635 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:00.635 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.753569+0000", 2024-04-26T03:39:00.636 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.1260000000000003e-06 2024-04-26T03:39:00.636 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.636 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.636 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.636 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.753938+0000", 2024-04-26T03:39:00.636 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00036876900000000002 2024-04-26T03:39:00.636 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.636 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.636 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "waiting for blocked object", 2024-04-26T03:39:00.636 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.753976+0000", 2024-04-26T03:39:00.636 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.8785000000000002e-05 2024-04-26T03:39:00.636 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.636 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.636 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.637 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.762561+0000", 2024-04-26T03:39:00.637 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0085849480000000002 2024-04-26T03:39:00.637 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.637 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.637 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:00.637 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.762655+0000", 2024-04-26T03:39:00.637 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.3691999999999997e-05 2024-04-26T03:39:00.637 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.637 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.637 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:00.637 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.762760+0000", 2024-04-26T03:39:00.637 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000104551 2024-04-26T03:39:00.637 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.637 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:00.637 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.638 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.638 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.638 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:262 3.4s0 3:2d740be2:::smithi09235267-28:head [append 0~720896] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T03:39:00.638 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:56.518030+0000", 2024-04-26T03:39:00.638 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 4.028323769, 2024-04-26T03:39:00.638 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.018690736999999999, 2024-04-26T03:39:00.638 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:00.638 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:00.638 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:00.638 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:00.638 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:00.638 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:00.638 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 262 2024-04-26T03:39:00.638 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.639 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:00.639 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.639 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:00.639 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.518030+0000", 2024-04-26T03:39:00.639 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.639 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.639 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.639 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:00.639 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.518030+0000", 2024-04-26T03:39:00.639 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.639 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.639 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.639 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:00.639 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.518039+0000", 2024-04-26T03:39:00.639 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.3999999999999992e-06 2024-04-26T03:39:00.640 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.640 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.640 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:00.640 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.519077+0000", 2024-04-26T03:39:00.640 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.001038593 2024-04-26T03:39:00.640 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.640 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.640 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:00.640 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.519089+0000", 2024-04-26T03:39:00.640 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.2092e-05 2024-04-26T03:39:00.640 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.640 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.640 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:00.640 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.519097+0000", 2024-04-26T03:39:00.640 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.7680000000000008e-06 2024-04-26T03:39:00.641 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.641 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.641 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.641 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.519162+0000", 2024-04-26T03:39:00.641 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.4517999999999995e-05 2024-04-26T03:39:00.641 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.641 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.641 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:00.641 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.519303+0000", 2024-04-26T03:39:00.641 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000141602 2024-04-26T03:39:00.641 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.641 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.641 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:00.641 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.520994+0000", 2024-04-26T03:39:00.641 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0016909780000000001 2024-04-26T03:39:00.642 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.642 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.642 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:00.642 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.531345+0000", 2024-04-26T03:39:00.642 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.010350471999999999 2024-04-26T03:39:00.642 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.642 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.642 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:00.642 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.536659+0000", 2024-04-26T03:39:00.642 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0053146649999999997 2024-04-26T03:39:00.642 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.642 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.642 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:00.642 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.536721+0000", 2024-04-26T03:39:00.642 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.1649000000000006e-05 2024-04-26T03:39:00.643 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.643 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:00.643 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.643 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.643 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.643 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:263 3.4s0 3:2d740be2:::smithi09235267-28:head [append 0~376832] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T03:39:00.643 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:56.520937+0000", 2024-04-26T03:39:00.643 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 4.0254172520000004, 2024-04-26T03:39:00.643 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.016100099999999999, 2024-04-26T03:39:00.643 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:00.643 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:00.643 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:00.643 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:00.643 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:00.643 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:00.644 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 263 2024-04-26T03:39:00.644 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.644 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:00.644 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.644 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:00.644 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.520937+0000", 2024-04-26T03:39:00.644 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.644 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.644 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.644 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:00.644 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.520937+0000", 2024-04-26T03:39:00.644 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.644 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.644 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.644 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:00.645 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.520939+0000", 2024-04-26T03:39:00.645 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.0669999999999999e-06 2024-04-26T03:39:00.645 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.645 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.645 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:00.645 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.521258+0000", 2024-04-26T03:39:00.645 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00031960300000000001 2024-04-26T03:39:00.645 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.645 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.645 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:00.645 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.521281+0000", 2024-04-26T03:39:00.645 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.2600999999999999e-05 2024-04-26T03:39:00.645 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.645 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.645 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:00.646 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.521290+0000", 2024-04-26T03:39:00.646 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.1169999999999993e-06 2024-04-26T03:39:00.646 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.646 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.646 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.646 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.528401+0000", 2024-04-26T03:39:00.646 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0071106499999999996 2024-04-26T03:39:00.646 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.646 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.646 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:00.646 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.528472+0000", 2024-04-26T03:39:00.646 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.1081999999999995e-05 2024-04-26T03:39:00.646 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.647 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.647 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:00.647 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.529015+0000", 2024-04-26T03:39:00.647 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00054347300000000005 2024-04-26T03:39:00.647 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.647 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.647 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:00.647 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.531961+0000", 2024-04-26T03:39:00.647 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.002945526 2024-04-26T03:39:00.647 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.647 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.647 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:00.647 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.536998+0000", 2024-04-26T03:39:00.647 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0050376960000000004 2024-04-26T03:39:00.647 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.648 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.648 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:00.648 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.537037+0000", 2024-04-26T03:39:00.648 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.8284999999999997e-05 2024-04-26T03:39:00.648 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.648 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:00.648 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.648 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.648 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.648 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:264 3.4s0 3:2d740be2:::smithi09235267-28:head [setxattr _header (56) in=7b] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T03:39:00.648 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:56.521315+0000", 2024-04-26T03:39:00.648 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 4.0250386520000001, 2024-04-26T03:39:00.648 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.015842726000000001, 2024-04-26T03:39:00.648 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:00.649 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:00.649 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:00.649 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:00.649 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:00.649 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:00.649 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 264 2024-04-26T03:39:00.649 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.649 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:00.649 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.649 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:00.649 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.521315+0000", 2024-04-26T03:39:00.649 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.649 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.649 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.649 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:00.650 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.521315+0000", 2024-04-26T03:39:00.650 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.650 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.650 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.650 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:00.650 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.521316+0000", 2024-04-26T03:39:00.650 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.2799999999999995e-07 2024-04-26T03:39:00.650 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.650 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.650 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:00.650 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.521321+0000", 2024-04-26T03:39:00.650 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.9230000000000001e-06 2024-04-26T03:39:00.650 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.650 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.651 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:00.651 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.521330+0000", 2024-04-26T03:39:00.651 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.1859999999999998e-06 2024-04-26T03:39:00.651 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.651 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.651 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:00.651 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.521336+0000", 2024-04-26T03:39:00.651 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.8499999999999999e-06 2024-04-26T03:39:00.651 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.651 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.651 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.651 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.531370+0000", 2024-04-26T03:39:00.651 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.010033767000000001 2024-04-26T03:39:00.651 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.652 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.652 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:00.652 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.531431+0000", 2024-04-26T03:39:00.652 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.1359000000000007e-05 2024-04-26T03:39:00.652 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.652 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.652 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:00.652 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.531718+0000", 2024-04-26T03:39:00.652 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00028652000000000001 2024-04-26T03:39:00.652 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.652 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.652 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:00.652 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.532128+0000", 2024-04-26T03:39:00.652 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00041016700000000002 2024-04-26T03:39:00.652 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.653 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.653 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:00.653 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.537102+0000", 2024-04-26T03:39:00.653 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0049742520000000002 2024-04-26T03:39:00.653 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.653 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.653 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:00.653 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.537158+0000", 2024-04-26T03:39:00.653 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.5874000000000001e-05 2024-04-26T03:39:00.653 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.653 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:00.653 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.653 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.653 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.653 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:265 3.4s0 3:2d740be2:::smithi09235267-28:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e40)", 2024-04-26T03:39:00.653 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:56.521345+0000", 2024-04-26T03:39:00.654 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 4.0250093429999998, 2024-04-26T03:39:00.654 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.017065469999999999, 2024-04-26T03:39:00.654 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:00.654 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:00.654 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "started", 2024-04-26T03:39:00.654 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:00.654 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:00.654 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:00.654 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 265 2024-04-26T03:39:00.654 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.654 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:00.654 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.654 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:00.654 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.521345+0000", 2024-04-26T03:39:00.655 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.655 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.655 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.655 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:00.655 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.521345+0000", 2024-04-26T03:39:00.655 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:00.655 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.655 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.655 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:00.655 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.521345+0000", 2024-04-26T03:39:00.655 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.1799999999999995e-07 2024-04-26T03:39:00.655 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.655 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.655 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:00.655 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.521347+0000", 2024-04-26T03:39:00.656 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.2199999999999999e-06 2024-04-26T03:39:00.656 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.656 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.656 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:00.656 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.521355+0000", 2024-04-26T03:39:00.656 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.9140000000000008e-06 2024-04-26T03:39:00.656 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.656 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.656 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:00.656 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.521360+0000", 2024-04-26T03:39:00.656 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.9989999999999999e-06 2024-04-26T03:39:00.656 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.656 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.656 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.656 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.531908+0000", 2024-04-26T03:39:00.657 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.010547707 2024-04-26T03:39:00.657 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.657 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.657 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:00.657 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.531953+0000", 2024-04-26T03:39:00.657 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.5543000000000001e-05 2024-04-26T03:39:00.657 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.657 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.657 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:00.657 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.537458+0000", 2024-04-26T03:39:00.657 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0055043699999999997 2024-04-26T03:39:00.657 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.657 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.657 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:00.658 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.537538+0000", 2024-04-26T03:39:00.658 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.9798999999999999e-05 2024-04-26T03:39:00.658 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:00.658 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:00.658 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:00.658 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.538410+0000", 2024-04-26T03:39:00.658 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00087239999999999996 2024-04-26T03:39:00.658 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.658 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:00.658 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.658 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:00.658 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:00.658 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:39:00.660 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-04-26T03:39:00.661 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:39:00.658+0000 7fb78c544640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31956) UID: 0 2024-04-26T03:39:00.761 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:39:00.759+0000 7fb78c544640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31956) UID: 0 2024-04-26T03:39:00.767 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:39:00.768 INFO:teuthology.orchestra.run.smithi028.stdout: "ops": [], 2024-04-26T03:39:00.768 INFO:teuthology.orchestra.run.smithi028.stdout: "num_ops": 0 2024-04-26T03:39:00.768 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:39:00.776 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-04-26T03:39:00.862 INFO:tasks.ceph.osd.9.smithi042.stderr:2024-04-26T03:39:00.861+0000 7fdac63b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 31960) UID: 0 2024-04-26T03:39:00.883 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:39:00.883 INFO:teuthology.orchestra.run.smithi028.stdout: "ops": [], 2024-04-26T03:39:00.883 INFO:teuthology.orchestra.run.smithi028.stdout: "complaint_time": 30, 2024-04-26T03:39:00.883 INFO:teuthology.orchestra.run.smithi028.stdout: "num_blocked_ops": 0 2024-04-26T03:39:00.883 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:39:00.891 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-04-26T03:39:00.963 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:39:00.961+0000 7fb78c544640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31956) UID: 0 2024-04-26T03:39:00.999 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:39:01.000 INFO:teuthology.orchestra.run.smithi028.stdout: "size": 20, 2024-04-26T03:39:01.000 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 600, 2024-04-26T03:39:01.000 INFO:teuthology.orchestra.run.smithi028.stdout: "ops": [ 2024-04-26T03:39:01.000 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.000 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.cs5 34/32 ECSubWrite(tid=0, reqid=client.4556.0:10, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:39:01.000 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.482740+0000", 2024-04-26T03:39:01.000 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 9.5146788769999997, 2024-04-26T03:39:01.000 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.003014287, 2024-04-26T03:39:01.000 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:01.000 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:01.000 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:01.000 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:01.001 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.001 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:01.001 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.482740+0000", 2024-04-26T03:39:01.001 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:01.001 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.001 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.001 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:01.001 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.482740+0000", 2024-04-26T03:39:01.001 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:01.001 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.001 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.002 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:01.002 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.482740+0000", 2024-04-26T03:39:01.002 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.1600000000000002e-07 2024-04-26T03:39:01.002 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.003 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.003 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:01.003 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.482783+0000", 2024-04-26T03:39:01.003 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.3130999999999998e-05 2024-04-26T03:39:01.003 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.003 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.004 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:01.004 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.482805+0000", 2024-04-26T03:39:01.004 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.1441999999999999e-05 2024-04-26T03:39:01.004 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.004 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.004 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:01.004 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.482816+0000", 2024-04-26T03:39:01.004 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1364e-05 2024-04-26T03:39:01.004 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.004 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.004 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:01.004 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.482903+0000", 2024-04-26T03:39:01.004 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.6735000000000002e-05 2024-04-26T03:39:01.005 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.005 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.005 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:01.005 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.482913+0000", 2024-04-26T03:39:01.005 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0145e-05 2024-04-26T03:39:01.005 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.005 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.005 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:01.005 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.485707+0000", 2024-04-26T03:39:01.005 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.002794 2024-04-26T03:39:01.005 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.005 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.005 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:01.005 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.485754+0000", 2024-04-26T03:39:01.006 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.7154000000000003e-05 2024-04-26T03:39:01.006 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:01.006 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:01.006 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:01.006 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.006 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.006 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.bs3 34/32 ECSubWrite(tid=0, reqid=client.4556.0:7, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:39:01.006 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.483031+0000", 2024-04-26T03:39:01.006 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 9.5143883860000003, 2024-04-26T03:39:01.006 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0035141780000000002, 2024-04-26T03:39:01.006 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:01.006 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:01.006 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:01.007 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:01.007 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.007 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:01.007 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.483031+0000", 2024-04-26T03:39:01.007 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:01.007 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.007 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.007 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:01.007 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.483031+0000", 2024-04-26T03:39:01.007 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:01.007 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.007 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.007 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:01.007 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.483031+0000", 2024-04-26T03:39:01.008 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.2999999999999999e-07 2024-04-26T03:39:01.008 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.008 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.008 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:01.008 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.483071+0000", 2024-04-26T03:39:01.008 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.9969999999999998e-05 2024-04-26T03:39:01.008 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.008 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.008 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:01.008 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.483083+0000", 2024-04-26T03:39:01.008 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.2482e-05 2024-04-26T03:39:01.008 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.009 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.009 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:01.009 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.483091+0000", 2024-04-26T03:39:01.009 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.2570000000000007e-06 2024-04-26T03:39:01.009 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.009 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.009 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:01.009 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.483202+0000", 2024-04-26T03:39:01.009 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000110368 2024-04-26T03:39:01.009 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.009 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.009 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:01.010 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.483211+0000", 2024-04-26T03:39:01.010 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.0000000000000002e-06 2024-04-26T03:39:01.010 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.010 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.010 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:01.010 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.486506+0000", 2024-04-26T03:39:01.010 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0032956320000000002 2024-04-26T03:39:01.010 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.010 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.010 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:01.010 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.486545+0000", 2024-04-26T03:39:01.010 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.8238999999999998e-05 2024-04-26T03:39:01.010 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:01.011 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:01.011 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:01.011 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.011 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.011 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.cs5 35/32 ECSubWrite(tid=4, reqid=client.4556.0:31, at_version=35'3, trim_to=0'0, roll_forward_to=34'1))", 2024-04-26T03:39:01.011 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.496173+0000", 2024-04-26T03:39:01.011 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 9.5012456659999991, 2024-04-26T03:39:01.011 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0034904060000000002, 2024-04-26T03:39:01.011 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:01.011 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:01.011 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:01.011 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:01.012 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.012 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:01.012 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.496173+0000", 2024-04-26T03:39:01.012 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:01.012 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.012 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.012 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:01.012 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.496173+0000", 2024-04-26T03:39:01.012 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:01.012 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.012 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.012 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:01.012 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.496174+0000", 2024-04-26T03:39:01.012 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.1199999999999998e-07 2024-04-26T03:39:01.013 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.013 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.013 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:01.013 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.496329+0000", 2024-04-26T03:39:01.013 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00015506799999999999 2024-04-26T03:39:01.013 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.013 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.013 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:01.013 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.496340+0000", 2024-04-26T03:39:01.013 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1399e-05 2024-04-26T03:39:01.013 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.013 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.013 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:01.013 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.496348+0000", 2024-04-26T03:39:01.014 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.0989999999999999e-06 2024-04-26T03:39:01.014 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.014 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.014 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:01.014 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.496414+0000", 2024-04-26T03:39:01.014 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.6070999999999998e-05 2024-04-26T03:39:01.014 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.014 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.014 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:01.014 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.496423+0000", 2024-04-26T03:39:01.014 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.6039999999999993e-06 2024-04-26T03:39:01.014 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.014 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.015 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:01.015 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.499624+0000", 2024-04-26T03:39:01.015 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0032013570000000002 2024-04-26T03:39:01.015 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.015 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.015 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:01.015 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.499664+0000", 2024-04-26T03:39:01.015 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.9396e-05 2024-04-26T03:39:01.015 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:01.015 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:01.015 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:01.015 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.015 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.015 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.bs3 35/32 ECSubWrite(tid=6, reqid=client.4556.0:37, at_version=35'3, trim_to=0'0, roll_forward_to=34'2))", 2024-04-26T03:39:01.016 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.501740+0000", 2024-04-26T03:39:01.016 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 9.4956789350000008, 2024-04-26T03:39:01.016 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.003262956, 2024-04-26T03:39:01.016 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:01.016 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:01.016 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:01.016 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:01.016 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.017 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:01.017 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.501740+0000", 2024-04-26T03:39:01.018 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:01.018 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.018 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.018 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:01.018 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.501740+0000", 2024-04-26T03:39:01.018 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:01.018 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.018 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.018 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:01.018 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.501741+0000", 2024-04-26T03:39:01.018 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1319999999999999e-06 2024-04-26T03:39:01.018 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.018 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.019 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:01.019 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.501815+0000", 2024-04-26T03:39:01.019 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.4122000000000006e-05 2024-04-26T03:39:01.019 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.019 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.019 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:01.019 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.501836+0000", 2024-04-26T03:39:01.019 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.0356e-05 2024-04-26T03:39:01.019 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.019 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.020 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:01.020 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.501845+0000", 2024-04-26T03:39:01.020 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.1619999999999997e-06 2024-04-26T03:39:01.020 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.020 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.020 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:01.020 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.501922+0000", 2024-04-26T03:39:01.020 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.7724000000000003e-05 2024-04-26T03:39:01.020 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.020 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.020 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:01.020 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.501932+0000", 2024-04-26T03:39:01.020 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.3600000000000002e-06 2024-04-26T03:39:01.020 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.021 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.021 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:01.021 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.504962+0000", 2024-04-26T03:39:01.021 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0030298899999999999 2024-04-26T03:39:01.021 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.021 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.021 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:01.021 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.505003+0000", 2024-04-26T03:39:01.021 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.121e-05 2024-04-26T03:39:01.021 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:01.021 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:01.021 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:01.021 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.021 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.021 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.es6 35/32 ECSubWrite(tid=2, reqid=client.4556.0:58, at_version=35'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:39:01.022 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.512937+0000", 2024-04-26T03:39:01.022 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 9.4844818600000007, 2024-04-26T03:39:01.022 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0046369269999999999, 2024-04-26T03:39:01.022 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:01.022 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:01.022 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:01.022 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:01.022 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.022 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:01.022 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.512937+0000", 2024-04-26T03:39:01.022 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:01.022 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.022 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.022 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:01.022 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.512937+0000", 2024-04-26T03:39:01.023 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:01.023 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.023 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.023 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:01.023 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.512937+0000", 2024-04-26T03:39:01.023 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.8099999999999999e-07 2024-04-26T03:39:01.023 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.023 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.023 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:01.023 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.513175+0000", 2024-04-26T03:39:01.023 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000237851 2024-04-26T03:39:01.023 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.023 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.024 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:01.024 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.513196+0000", 2024-04-26T03:39:01.024 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.0344999999999999e-05 2024-04-26T03:39:01.024 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.024 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.024 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:01.024 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.513207+0000", 2024-04-26T03:39:01.024 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1083999999999999e-05 2024-04-26T03:39:01.024 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.024 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.024 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:01.025 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.513309+0000", 2024-04-26T03:39:01.025 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000102321 2024-04-26T03:39:01.025 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.025 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.025 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:01.025 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.513318+0000", 2024-04-26T03:39:01.025 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.5219999999999996e-06 2024-04-26T03:39:01.025 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.025 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.025 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:01.025 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.517543+0000", 2024-04-26T03:39:01.025 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0042245160000000002 2024-04-26T03:39:01.026 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.026 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.026 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:01.026 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.517574+0000", 2024-04-26T03:39:01.026 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.1007000000000003e-05 2024-04-26T03:39:01.026 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:01.026 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:01.026 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:01.026 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.026 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.026 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.bs3 35/32 ECSubWrite(tid=10, reqid=client.4556.0:61, at_version=35'5, trim_to=0'0, roll_forward_to=35'3))", 2024-04-26T03:39:01.026 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.514935+0000", 2024-04-26T03:39:01.026 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 9.4824841249999992, 2024-04-26T03:39:01.027 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0043992700000000003, 2024-04-26T03:39:01.027 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:01.027 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:01.027 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:01.027 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:01.027 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.027 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:01.027 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.514935+0000", 2024-04-26T03:39:01.027 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:01.027 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.027 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.027 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:01.027 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.514935+0000", 2024-04-26T03:39:01.027 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:01.027 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.028 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.028 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:01.028 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.514936+0000", 2024-04-26T03:39:01.028 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1170000000000001e-06 2024-04-26T03:39:01.028 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.028 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.028 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:01.028 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.515012+0000", 2024-04-26T03:39:01.028 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.5631999999999997e-05 2024-04-26T03:39:01.028 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.028 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.028 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:01.028 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.515030+0000", 2024-04-26T03:39:01.028 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.8881000000000001e-05 2024-04-26T03:39:01.029 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.029 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.029 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:01.029 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.515040+0000", 2024-04-26T03:39:01.029 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.6039999999999999e-06 2024-04-26T03:39:01.029 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.029 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.029 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:01.029 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.515118+0000", 2024-04-26T03:39:01.029 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.8121000000000006e-05 2024-04-26T03:39:01.030 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.030 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.030 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:01.030 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.515129+0000", 2024-04-26T03:39:01.030 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.04e-05 2024-04-26T03:39:01.030 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.030 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.030 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:01.030 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.519303+0000", 2024-04-26T03:39:01.030 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0041743129999999998 2024-04-26T03:39:01.030 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.030 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.030 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:01.030 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.519334+0000", 2024-04-26T03:39:01.031 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.1201999999999997e-05 2024-04-26T03:39:01.031 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:01.031 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:01.031 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:01.031 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.031 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.031 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.cs5 35/32 ECSubWrite(tid=8, reqid=client.4556.0:32, at_version=35'4, trim_to=0'0, roll_forward_to=34'2))", 2024-04-26T03:39:01.031 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.519044+0000", 2024-04-26T03:39:01.031 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 9.4783750639999997, 2024-04-26T03:39:01.031 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.003351896, 2024-04-26T03:39:01.031 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:01.031 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:01.032 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:01.032 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:01.032 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.032 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:01.032 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.519044+0000", 2024-04-26T03:39:01.032 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:01.032 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.032 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.032 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:01.032 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.519044+0000", 2024-04-26T03:39:01.032 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:01.033 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.033 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.033 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:01.033 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.519045+0000", 2024-04-26T03:39:01.033 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.6300000000000004e-07 2024-04-26T03:39:01.033 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.033 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.033 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:01.033 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.519567+0000", 2024-04-26T03:39:01.033 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00052223500000000002 2024-04-26T03:39:01.033 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.033 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.033 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:01.033 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.519579+0000", 2024-04-26T03:39:01.033 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.2184999999999999e-05 2024-04-26T03:39:01.034 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.034 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.034 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:01.034 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.519587+0000", 2024-04-26T03:39:01.034 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.2530000000000004e-06 2024-04-26T03:39:01.034 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.034 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.034 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:01.034 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.519646+0000", 2024-04-26T03:39:01.034 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.9077999999999999e-05 2024-04-26T03:39:01.034 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.034 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.034 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:01.035 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.519656+0000", 2024-04-26T03:39:01.035 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.2199999999999998e-06 2024-04-26T03:39:01.035 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.035 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.035 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:01.035 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.522359+0000", 2024-04-26T03:39:01.035 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0027033790000000001 2024-04-26T03:39:01.035 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.035 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.035 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:01.035 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.522396+0000", 2024-04-26T03:39:01.035 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.6683000000000001e-05 2024-04-26T03:39:01.035 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:01.035 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:01.035 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:01.036 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.036 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.036 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.es6 35/32 ECSubWrite(tid=8, reqid=client.4556.0:70, at_version=35'3, trim_to=0'0, roll_forward_to=35'2))", 2024-04-26T03:39:01.036 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.523457+0000", 2024-04-26T03:39:01.036 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 9.4739620040000005, 2024-04-26T03:39:01.036 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0042674139999999998, 2024-04-26T03:39:01.036 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:01.036 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:01.036 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:01.036 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:01.036 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.036 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:01.036 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.523457+0000", 2024-04-26T03:39:01.036 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:01.036 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.037 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.037 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:01.037 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.523457+0000", 2024-04-26T03:39:01.037 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:01.037 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.037 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.037 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:01.037 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.523458+0000", 2024-04-26T03:39:01.037 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.78e-07 2024-04-26T03:39:01.037 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.037 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.037 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:01.037 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.523548+0000", 2024-04-26T03:39:01.037 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.0366999999999997e-05 2024-04-26T03:39:01.038 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.038 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.038 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:01.038 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.523563+0000", 2024-04-26T03:39:01.038 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.4450999999999999e-05 2024-04-26T03:39:01.038 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.038 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.038 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:01.038 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.523571+0000", 2024-04-26T03:39:01.038 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.4659999999999999e-06 2024-04-26T03:39:01.038 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.038 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.038 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:01.038 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.523664+0000", 2024-04-26T03:39:01.038 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.3312000000000004e-05 2024-04-26T03:39:01.039 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.039 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.039 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:01.039 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.523673+0000", 2024-04-26T03:39:01.039 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.9660000000000002e-06 2024-04-26T03:39:01.039 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.039 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.039 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:01.039 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.527682+0000", 2024-04-26T03:39:01.039 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0040086540000000004 2024-04-26T03:39:01.039 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.039 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.039 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:01.040 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.527724+0000", 2024-04-26T03:39:01.040 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.2320000000000001e-05 2024-04-26T03:39:01.040 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:01.040 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:01.040 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:01.040 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.040 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.040 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.es6 35/32 ECSubWrite(tid=16, reqid=client.4556.0:85, at_version=35'5, trim_to=0'0, roll_forward_to=35'4))", 2024-04-26T03:39:01.040 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.542343+0000", 2024-04-26T03:39:01.040 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 9.4550762959999997, 2024-04-26T03:39:01.040 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0051706219999999997, 2024-04-26T03:39:01.040 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:01.040 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:01.041 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:01.041 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:01.041 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.041 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:01.041 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.542343+0000", 2024-04-26T03:39:01.041 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:01.041 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.041 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.041 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:01.041 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.542343+0000", 2024-04-26T03:39:01.041 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:01.041 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.041 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.041 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:01.041 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.542344+0000", 2024-04-26T03:39:01.042 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.488e-06 2024-04-26T03:39:01.042 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.042 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.042 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:01.042 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.542511+0000", 2024-04-26T03:39:01.042 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000166844 2024-04-26T03:39:01.042 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.042 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.042 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:01.042 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.542528+0000", 2024-04-26T03:39:01.042 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.6548000000000001e-05 2024-04-26T03:39:01.042 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.042 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.042 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:01.042 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.542535+0000", 2024-04-26T03:39:01.043 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.7610000000000007e-06 2024-04-26T03:39:01.043 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.043 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.043 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:01.043 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.542610+0000", 2024-04-26T03:39:01.043 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.4632e-05 2024-04-26T03:39:01.043 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.043 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.043 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:01.043 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.542620+0000", 2024-04-26T03:39:01.043 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0108e-05 2024-04-26T03:39:01.043 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.043 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.043 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:01.043 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.547469+0000", 2024-04-26T03:39:01.044 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0048486129999999999 2024-04-26T03:39:01.044 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.044 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.044 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:01.044 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.547513+0000", 2024-04-26T03:39:01.044 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.4628e-05 2024-04-26T03:39:01.044 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:01.044 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:01.044 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:01.044 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.044 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.044 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.es6 35/32 ECSubWrite(tid=24, reqid=client.4556.0:124, at_version=35'7, trim_to=0'0, roll_forward_to=35'6))", 2024-04-26T03:39:01.044 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.596283+0000", 2024-04-26T03:39:01.044 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 9.4011359970000008, 2024-04-26T03:39:01.044 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0047651710000000003, 2024-04-26T03:39:01.045 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:01.045 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:01.045 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:01.045 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:01.045 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.045 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:01.045 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.596283+0000", 2024-04-26T03:39:01.045 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:01.045 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.045 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.045 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:01.045 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.596283+0000", 2024-04-26T03:39:01.045 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:01.045 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.045 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.046 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:01.046 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.596284+0000", 2024-04-26T03:39:01.046 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0950000000000001e-06 2024-04-26T03:39:01.046 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.046 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.046 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:01.046 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.596595+0000", 2024-04-26T03:39:01.046 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00031093400000000002 2024-04-26T03:39:01.046 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.046 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.046 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:01.046 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.596610+0000", 2024-04-26T03:39:01.046 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.5264000000000002e-05 2024-04-26T03:39:01.046 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.047 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.047 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:01.047 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.596617+0000", 2024-04-26T03:39:01.047 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.0859999999999996e-06 2024-04-26T03:39:01.047 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.047 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.047 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:01.047 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.596669+0000", 2024-04-26T03:39:01.047 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.1436e-05 2024-04-26T03:39:01.047 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.047 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.047 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:01.047 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.596675+0000", 2024-04-26T03:39:01.047 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.4030000000000004e-06 2024-04-26T03:39:01.047 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.048 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.048 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:01.048 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601010+0000", 2024-04-26T03:39:01.048 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0043352030000000001 2024-04-26T03:39:01.048 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.048 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.048 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:01.048 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601048+0000", 2024-04-26T03:39:01.048 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.7750000000000003e-05 2024-04-26T03:39:01.048 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:01.048 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:01.048 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:01.048 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.048 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.048 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.bs3 35/32 ECSubWrite(tid=20, reqid=client.4556.0:127, at_version=35'7, trim_to=0'0, roll_forward_to=35'6))", 2024-04-26T03:39:01.049 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.601363+0000", 2024-04-26T03:39:01.049 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 9.3960560399999995, 2024-04-26T03:39:01.049 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.007091913, 2024-04-26T03:39:01.049 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:01.049 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:01.049 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:01.049 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:01.049 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.049 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:01.049 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601363+0000", 2024-04-26T03:39:01.049 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:01.049 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.049 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.049 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:01.049 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601363+0000", 2024-04-26T03:39:01.050 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:01.050 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.050 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.050 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:01.050 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601364+0000", 2024-04-26T03:39:01.050 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.54e-07 2024-04-26T03:39:01.050 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.050 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.050 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:01.050 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601498+0000", 2024-04-26T03:39:01.050 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00013417600000000001 2024-04-26T03:39:01.051 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.051 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.051 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:01.051 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601514+0000", 2024-04-26T03:39:01.051 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.6371999999999998e-05 2024-04-26T03:39:01.051 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.051 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.051 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:01.051 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601523+0000", 2024-04-26T03:39:01.051 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.0189999999999995e-06 2024-04-26T03:39:01.051 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.051 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.051 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:01.051 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601589+0000", 2024-04-26T03:39:01.051 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.5924000000000001e-05 2024-04-26T03:39:01.052 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.052 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.052 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:01.052 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601598+0000", 2024-04-26T03:39:01.052 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.6940000000000001e-06 2024-04-26T03:39:01.052 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.052 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.052 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:01.052 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.608417+0000", 2024-04-26T03:39:01.052 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.006819121 2024-04-26T03:39:01.052 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.052 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.052 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:01.052 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.608455+0000", 2024-04-26T03:39:01.053 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.7753000000000003e-05 2024-04-26T03:39:01.053 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:01.053 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:01.053 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:01.053 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.053 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.053 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.bs3 35/32 ECSubWrite(tid=22, reqid=client.4556.0:133, at_version=35'9, trim_to=0'0, roll_forward_to=35'7))", 2024-04-26T03:39:01.053 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.610898+0000", 2024-04-26T03:39:01.053 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 9.3865208320000004, 2024-04-26T03:39:01.053 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0048565170000000003, 2024-04-26T03:39:01.053 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:01.053 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:01.053 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:01.053 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:01.053 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.053 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:01.054 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.610898+0000", 2024-04-26T03:39:01.054 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:01.054 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.054 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.054 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:01.054 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.610898+0000", 2024-04-26T03:39:01.054 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:01.054 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.054 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.054 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:01.054 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.610899+0000", 2024-04-26T03:39:01.054 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.9500000000000001e-07 2024-04-26T03:39:01.054 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.054 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.055 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:01.055 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.611030+0000", 2024-04-26T03:39:01.055 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00013066099999999999 2024-04-26T03:39:01.055 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.055 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.055 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:01.055 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.611046+0000", 2024-04-26T03:39:01.055 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.6056e-05 2024-04-26T03:39:01.055 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.055 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.055 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:01.055 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.611053+0000", 2024-04-26T03:39:01.055 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.1999999999999997e-06 2024-04-26T03:39:01.055 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.055 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.056 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:01.056 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.611119+0000", 2024-04-26T03:39:01.056 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.5839000000000004e-05 2024-04-26T03:39:01.056 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.056 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.056 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:01.056 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.611136+0000", 2024-04-26T03:39:01.056 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.7070000000000001e-05 2024-04-26T03:39:01.056 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.056 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.056 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:01.056 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.615722+0000", 2024-04-26T03:39:01.056 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.004586499 2024-04-26T03:39:01.056 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.056 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.056 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:01.057 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.615755+0000", 2024-04-26T03:39:01.057 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.2296999999999999e-05 2024-04-26T03:39:01.057 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:01.057 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:01.057 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:01.057 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.057 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.057 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.6s4 35/32 ECSubWrite(tid=0, reqid=client.4556.0:146, at_version=35'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:39:01.057 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.630338+0000", 2024-04-26T03:39:01.057 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 9.3670806280000001, 2024-04-26T03:39:01.057 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0032862170000000001, 2024-04-26T03:39:01.058 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:01.058 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:01.058 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:01.058 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:01.058 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.058 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:01.058 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.630338+0000", 2024-04-26T03:39:01.058 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:01.058 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.058 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.058 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:01.058 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.630338+0000", 2024-04-26T03:39:01.058 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:01.058 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.059 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.059 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:01.059 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.630339+0000", 2024-04-26T03:39:01.059 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.9899999999999999e-07 2024-04-26T03:39:01.059 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.059 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.059 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:01.059 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.630430+0000", 2024-04-26T03:39:01.059 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.1057999999999995e-05 2024-04-26T03:39:01.059 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.059 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.059 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:01.059 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.630445+0000", 2024-04-26T03:39:01.059 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.5024999999999999e-05 2024-04-26T03:39:01.059 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.060 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.060 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:01.060 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.630478+0000", 2024-04-26T03:39:01.060 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.2302999999999999e-05 2024-04-26T03:39:01.060 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.060 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.060 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:01.060 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.630554+0000", 2024-04-26T03:39:01.060 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.6544000000000004e-05 2024-04-26T03:39:01.060 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.060 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.060 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:01.061 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.630563+0000", 2024-04-26T03:39:01.061 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.4700000000000002e-06 2024-04-26T03:39:01.061 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.061 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.061 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:01.061 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.633596+0000", 2024-04-26T03:39:01.061 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0030333339999999999 2024-04-26T03:39:01.061 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.061 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.061 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:01.061 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.633625+0000", 2024-04-26T03:39:01.061 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.8583999999999999e-05 2024-04-26T03:39:01.061 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:01.062 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:01.062 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:01.062 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.062 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.062 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.6s4 35/32 ECSubWrite(tid=1, reqid=client.4556.0:147, at_version=35'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:39:01.062 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.634031+0000", 2024-04-26T03:39:01.062 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 9.3633882199999992, 2024-04-26T03:39:01.062 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.002869726, 2024-04-26T03:39:01.062 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:01.062 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:01.062 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:01.062 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:01.062 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.062 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:01.062 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.634031+0000", 2024-04-26T03:39:01.063 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:01.063 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.063 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.063 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:01.063 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.634031+0000", 2024-04-26T03:39:01.063 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:01.063 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.063 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.063 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:01.063 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.634032+0000", 2024-04-26T03:39:01.063 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.035e-06 2024-04-26T03:39:01.063 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.063 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.063 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:01.063 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.634119+0000", 2024-04-26T03:39:01.064 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.6831999999999998e-05 2024-04-26T03:39:01.064 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.064 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.064 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:01.064 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.634134+0000", 2024-04-26T03:39:01.064 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.5051e-05 2024-04-26T03:39:01.064 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.064 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.064 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:01.064 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.634152+0000", 2024-04-26T03:39:01.064 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.8153000000000001e-05 2024-04-26T03:39:01.064 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.064 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.064 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:01.065 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.634235+0000", 2024-04-26T03:39:01.065 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.2787000000000005e-05 2024-04-26T03:39:01.065 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.065 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.065 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:01.065 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.634240+0000", 2024-04-26T03:39:01.065 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.8939999999999998e-06 2024-04-26T03:39:01.065 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.065 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.065 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:01.065 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.636870+0000", 2024-04-26T03:39:01.065 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0026300030000000001 2024-04-26T03:39:01.065 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.065 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.065 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:01.066 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.636900+0000", 2024-04-26T03:39:01.066 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.9971000000000002e-05 2024-04-26T03:39:01.066 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:01.066 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:01.066 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:01.066 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.066 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.066 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.es6 35/32 ECSubWrite(tid=34, reqid=client.4556.0:177, at_version=35'9, trim_to=0'0, roll_forward_to=35'8))", 2024-04-26T03:39:01.066 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.731787+0000", 2024-04-26T03:39:01.066 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 9.2656315520000003, 2024-04-26T03:39:01.066 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0075575629999999998, 2024-04-26T03:39:01.066 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:01.066 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:01.066 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:01.066 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:01.067 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.067 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:01.067 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.731787+0000", 2024-04-26T03:39:01.067 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:01.067 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.067 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.067 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:01.067 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.731787+0000", 2024-04-26T03:39:01.067 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:01.067 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.067 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.067 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:01.067 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.731788+0000", 2024-04-26T03:39:01.067 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1200000000000001e-06 2024-04-26T03:39:01.067 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.068 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.068 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:01.068 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.732056+0000", 2024-04-26T03:39:01.068 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00026767099999999998 2024-04-26T03:39:01.068 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.068 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.068 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:01.068 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.732072+0000", 2024-04-26T03:39:01.068 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.6242000000000001e-05 2024-04-26T03:39:01.068 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.068 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.068 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:01.068 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.732083+0000", 2024-04-26T03:39:01.068 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0135e-05 2024-04-26T03:39:01.069 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.069 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.069 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:01.069 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.732152+0000", 2024-04-26T03:39:01.069 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.9325000000000005e-05 2024-04-26T03:39:01.069 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.069 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.069 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:01.069 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.732163+0000", 2024-04-26T03:39:01.069 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0728e-05 2024-04-26T03:39:01.069 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.069 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.069 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:01.069 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.739313+0000", 2024-04-26T03:39:01.069 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0071506599999999997 2024-04-26T03:39:01.070 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.070 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.070 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:01.070 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.739345+0000", 2024-04-26T03:39:01.070 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.1681999999999999e-05 2024-04-26T03:39:01.070 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:01.070 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:01.070 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:01.070 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.070 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.070 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.es6 35/32 ECSubWrite(tid=35, reqid=client.4556.0:178, at_version=35'10, trim_to=0'0, roll_forward_to=35'8))", 2024-04-26T03:39:01.070 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.736069+0000", 2024-04-26T03:39:01.070 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 9.2613499340000001, 2024-04-26T03:39:01.070 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0040055810000000002, 2024-04-26T03:39:01.070 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:01.071 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:01.071 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:01.071 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:01.071 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.071 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:01.071 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.736069+0000", 2024-04-26T03:39:01.071 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:01.071 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.071 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.071 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:01.071 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.736069+0000", 2024-04-26T03:39:01.071 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:01.071 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.071 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.071 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:01.072 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.736070+0000", 2024-04-26T03:39:01.072 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.4799999999999998e-07 2024-04-26T03:39:01.072 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.072 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.072 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:01.072 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.736255+0000", 2024-04-26T03:39:01.072 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00018521999999999999 2024-04-26T03:39:01.072 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.072 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.072 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:01.072 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.736266+0000", 2024-04-26T03:39:01.072 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0886999999999999e-05 2024-04-26T03:39:01.072 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.072 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.072 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:01.073 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.736271+0000", 2024-04-26T03:39:01.073 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.1490000000000002e-06 2024-04-26T03:39:01.073 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.073 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.073 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:01.073 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.736970+0000", 2024-04-26T03:39:01.073 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00069932999999999996 2024-04-26T03:39:01.073 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.073 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.073 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:01.073 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.736982+0000", 2024-04-26T03:39:01.073 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1642e-05 2024-04-26T03:39:01.073 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.073 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.074 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:01.074 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.740045+0000", 2024-04-26T03:39:01.074 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0030632720000000001 2024-04-26T03:39:01.074 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.074 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.074 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:01.074 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.740075+0000", 2024-04-26T03:39:01.074 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.9533e-05 2024-04-26T03:39:01.074 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:01.074 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:01.074 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:01.074 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.074 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.074 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.es6 35/32 ECSubWrite(tid=39, reqid=client.4556.0:207, at_version=35'12, trim_to=0'0, roll_forward_to=35'11))", 2024-04-26T03:39:01.075 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.807524+0000", 2024-04-26T03:39:01.075 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 9.1898951259999997, 2024-04-26T03:39:01.075 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0081817109999999995, 2024-04-26T03:39:01.075 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:01.075 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:01.075 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:01.075 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:01.075 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.075 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:01.075 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.807524+0000", 2024-04-26T03:39:01.075 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:01.075 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.075 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.075 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:01.075 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.807524+0000", 2024-04-26T03:39:01.075 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:01.076 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.076 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.076 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:01.076 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.807525+0000", 2024-04-26T03:39:01.076 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.6700000000000002e-07 2024-04-26T03:39:01.076 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.076 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.076 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:01.076 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.808020+0000", 2024-04-26T03:39:01.076 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00049521799999999996 2024-04-26T03:39:01.076 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.076 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.076 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:01.076 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.808031+0000", 2024-04-26T03:39:01.076 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0862999999999999e-05 2024-04-26T03:39:01.077 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.077 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.077 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:01.077 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.808037+0000", 2024-04-26T03:39:01.077 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.438e-06 2024-04-26T03:39:01.077 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.077 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.077 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:01.077 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.808102+0000", 2024-04-26T03:39:01.077 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.4388999999999996e-05 2024-04-26T03:39:01.077 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.077 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.077 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:01.077 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.808114+0000", 2024-04-26T03:39:01.077 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.2211e-05 2024-04-26T03:39:01.078 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.078 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.078 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:01.078 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.815665+0000", 2024-04-26T03:39:01.078 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0075510259999999997 2024-04-26T03:39:01.078 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.078 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.078 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:01.078 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.815706+0000", 2024-04-26T03:39:01.078 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.0698999999999998e-05 2024-04-26T03:39:01.078 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:01.078 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:01.078 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:01.078 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.078 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.079 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.es6 35/32 ECSubWrite(tid=40, reqid=client.4556.0:208, at_version=35'13, trim_to=0'0, roll_forward_to=35'11))", 2024-04-26T03:39:01.079 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.811551+0000", 2024-04-26T03:39:01.079 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 9.1858681329999996, 2024-04-26T03:39:01.079 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0045656179999999996, 2024-04-26T03:39:01.079 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:01.079 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:01.079 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:01.079 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:01.079 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.079 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:01.079 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.811551+0000", 2024-04-26T03:39:01.079 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:01.079 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.079 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.079 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:01.080 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.811551+0000", 2024-04-26T03:39:01.080 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:01.080 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.080 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.080 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:01.080 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.811551+0000", 2024-04-26T03:39:01.080 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.3900000000000005e-07 2024-04-26T03:39:01.080 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.080 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.080 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:01.080 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.811779+0000", 2024-04-26T03:39:01.080 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000227541 2024-04-26T03:39:01.080 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.080 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.080 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:01.081 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.811790+0000", 2024-04-26T03:39:01.081 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1076e-05 2024-04-26T03:39:01.081 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.081 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.081 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:01.081 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.811796+0000", 2024-04-26T03:39:01.081 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.643e-06 2024-04-26T03:39:01.081 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.081 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.081 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:01.081 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.813542+0000", 2024-04-26T03:39:01.081 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0017465969999999999 2024-04-26T03:39:01.081 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.081 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.082 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:01.082 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.813549+0000", 2024-04-26T03:39:01.082 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.5359999999999998e-06 2024-04-26T03:39:01.082 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.082 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.082 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:01.082 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.816088+0000", 2024-04-26T03:39:01.082 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0025395750000000001 2024-04-26T03:39:01.082 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.082 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.082 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:01.082 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.816116+0000", 2024-04-26T03:39:01.082 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.8110999999999999e-05 2024-04-26T03:39:01.082 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:01.083 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:01.083 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:01.083 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.083 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.083 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.es6 43/32 ECSubWrite(tid=48, reqid=client.4556.0:301, at_version=43'16, trim_to=0'0, roll_forward_to=35'14))", 2024-04-26T03:39:01.083 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:59.531146+0000", 2024-04-26T03:39:01.083 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 1.466272843, 2024-04-26T03:39:01.083 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.004955175, 2024-04-26T03:39:01.083 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:01.083 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:01.083 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:01.083 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:01.083 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.083 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:01.083 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.531146+0000", 2024-04-26T03:39:01.084 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:01.084 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.084 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.084 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:01.084 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.531146+0000", 2024-04-26T03:39:01.084 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:01.084 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.084 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.084 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:01.084 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.531146+0000", 2024-04-26T03:39:01.084 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.49e-07 2024-04-26T03:39:01.084 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.084 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.084 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:01.084 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.531483+0000", 2024-04-26T03:39:01.085 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00033656099999999998 2024-04-26T03:39:01.085 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.085 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.085 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:01.085 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.531497+0000", 2024-04-26T03:39:01.085 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.378e-05 2024-04-26T03:39:01.085 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.085 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.085 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:01.085 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.531506+0000", 2024-04-26T03:39:01.085 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.2159999999999995e-06 2024-04-26T03:39:01.085 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.085 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.085 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:01.085 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.531587+0000", 2024-04-26T03:39:01.085 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.1180999999999994e-05 2024-04-26T03:39:01.086 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.086 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.086 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:01.086 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.531599+0000", 2024-04-26T03:39:01.086 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.2169e-05 2024-04-26T03:39:01.086 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.086 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.086 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:01.086 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.536067+0000", 2024-04-26T03:39:01.086 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0044680099999999997 2024-04-26T03:39:01.086 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.086 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.086 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:01.086 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.536101+0000", 2024-04-26T03:39:01.087 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.4109000000000002e-05 2024-04-26T03:39:01.087 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:01.087 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:01.087 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:01.087 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.087 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.087 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.es6 43/32 ECSubWrite(tid=49, reqid=client.4556.0:302, at_version=43'17, trim_to=0'0, roll_forward_to=35'14))", 2024-04-26T03:39:01.087 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:59.537150+0000", 2024-04-26T03:39:01.087 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 1.460269144, 2024-04-26T03:39:01.087 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0058389130000000003, 2024-04-26T03:39:01.087 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:01.087 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:01.087 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:01.087 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:01.087 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.088 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:01.088 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.537150+0000", 2024-04-26T03:39:01.088 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:01.088 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.088 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.088 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:01.088 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.537150+0000", 2024-04-26T03:39:01.088 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:01.088 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.088 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.088 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:01.088 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.537151+0000", 2024-04-26T03:39:01.088 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.92e-07 2024-04-26T03:39:01.088 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.088 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.089 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:01.089 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.537707+0000", 2024-04-26T03:39:01.089 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00055628199999999996 2024-04-26T03:39:01.089 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.089 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.089 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:01.089 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.537720+0000", 2024-04-26T03:39:01.089 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.3434999999999999e-05 2024-04-26T03:39:01.089 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.089 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.089 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:01.089 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.537728+0000", 2024-04-26T03:39:01.089 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.8290000000000008e-06 2024-04-26T03:39:01.089 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.089 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.090 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:01.090 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.537787+0000", 2024-04-26T03:39:01.090 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.9249000000000001e-05 2024-04-26T03:39:01.090 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.090 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.090 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:01.090 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.537798+0000", 2024-04-26T03:39:01.090 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0997e-05 2024-04-26T03:39:01.090 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.090 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.090 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:01.090 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.542961+0000", 2024-04-26T03:39:01.090 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0051625320000000001 2024-04-26T03:39:01.090 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:01.091 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:01.091 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:01.091 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.542989+0000", 2024-04-26T03:39:01.091 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.7797000000000002e-05 2024-04-26T03:39:01.091 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:01.091 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:01.091 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:01.091 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:01.091 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:01.091 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:39:01.093 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-26T03:39:01.093 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:39:01.091+0000 7f60d3c4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31757) UID: 0 2024-04-26T03:39:01.194 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-04-26T03:39:01.192+0000 7fdc2d21c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31819) UID: 0 2024-04-26T03:39:01.213 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-26T03:39:01.213 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-04-26T03:39:01.213 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-04-26T03:39:01.213 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-26T03:39:01.222 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-26T03:39:01.294 INFO:tasks.ceph.osd.9.smithi042.stderr:2024-04-26T03:39:01.293+0000 7fdac63b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 31960) UID: 0 2024-04-26T03:39:01.330 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-26T03:39:01.330 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-04-26T03:39:01.330 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-04-26T03:39:01.330 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-04-26T03:39:01.331 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-26T03:39:01.339 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-26T03:39:01.395 INFO:tasks.ceph.osd.12.smithi028.stderr:2024-04-26T03:39:01.394+0000 7f114fe14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 31967) UID: 0 2024-04-26T03:39:01.489 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-26T03:39:01.489 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-04-26T03:39:01.489 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-04-26T03:39:01.489 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-04-26T03:39:01.489 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.490 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:13 3.0s0 3:0a8403ef:::smithi09235267-5:head [append 0~81920,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:01.490 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.484308+0000", 2024-04-26T03:39:01.490 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.00245194, 2024-04-26T03:39:01.490 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017722486999999999, 2024-04-26T03:39:01.490 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:01.490 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:01.490 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:01.490 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:01.490 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:01.490 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:01.490 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 13 2024-04-26T03:39:01.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.491 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:01.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.491 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:01.491 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484308+0000", 2024-04-26T03:39:01.491 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:01.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.491 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:01.491 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484308+0000", 2024-04-26T03:39:01.491 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:01.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.492 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:01.492 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484311+0000", 2024-04-26T03:39:01.492 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8890000000000002e-06 2024-04-26T03:39:01.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.492 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.492 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:01.492 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484792+0000", 2024-04-26T03:39:01.492 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00048166600000000001 2024-04-26T03:39:01.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.492 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.492 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:01.492 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484806+0000", 2024-04-26T03:39:01.492 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3709999999999999e-05 2024-04-26T03:39:01.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.492 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.493 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:01.493 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484816+0000", 2024-04-26T03:39:01.493 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7000000000000003e-06 2024-04-26T03:39:01.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.493 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.493 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:01.493 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484975+0000", 2024-04-26T03:39:01.493 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015943199999999999 2024-04-26T03:39:01.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.493 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.494 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:01.494 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.485147+0000", 2024-04-26T03:39:01.494 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000171888 2024-04-26T03:39:01.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.494 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:01.494 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.486224+0000", 2024-04-26T03:39:01.494 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010766720000000001 2024-04-26T03:39:01.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.494 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:01.494 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.489541+0000", 2024-04-26T03:39:01.494 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033176210000000002 2024-04-26T03:39:01.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.495 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:39:01.495 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.501934+0000", 2024-04-26T03:39:01.495 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012392633 2024-04-26T03:39:01.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.495 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:01.495 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.502030+0000", 2024-04-26T03:39:01.495 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6275999999999998e-05 2024-04-26T03:39:01.495 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:01.495 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:01.496 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:01.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.496 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:14 3.0s0 3:0a8403ef:::smithi09235267-5:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:01.496 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.484901+0000", 2024-04-26T03:39:01.496 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.001858833, 2024-04-26T03:39:01.496 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017354086000000001, 2024-04-26T03:39:01.496 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:01.496 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:01.496 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:01.496 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:01.496 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:01.496 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:01.496 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 14 2024-04-26T03:39:01.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.497 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:01.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.497 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:01.497 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484901+0000", 2024-04-26T03:39:01.497 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:01.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.497 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:01.497 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484901+0000", 2024-04-26T03:39:01.497 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:01.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.497 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:01.497 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484902+0000", 2024-04-26T03:39:01.497 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1069999999999999e-06 2024-04-26T03:39:01.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:01.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484909+0000", 2024-04-26T03:39:01.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.323e-06 2024-04-26T03:39:01.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:01.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484919+0000", 2024-04-26T03:39:01.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0069e-05 2024-04-26T03:39:01.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:01.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484930+0000", 2024-04-26T03:39:01.499 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1092e-05 2024-04-26T03:39:01.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.499 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.499 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:01.499 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.487836+0000", 2024-04-26T03:39:01.499 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0029053569999999999 2024-04-26T03:39:01.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.499 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.499 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:01.499 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.487949+0000", 2024-04-26T03:39:01.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000112822 2024-04-26T03:39:01.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.500 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:01.500 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.488660+0000", 2024-04-26T03:39:01.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00071102899999999996 2024-04-26T03:39:01.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:01.501 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.490973+0000", 2024-04-26T03:39:01.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023132510000000001 2024-04-26T03:39:01.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:39:01.501 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.502166+0000", 2024-04-26T03:39:01.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011193639999999999 2024-04-26T03:39:01.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:01.501 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.502255+0000", 2024-04-26T03:39:01.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8395999999999999e-05 2024-04-26T03:39:01.501 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:01.502 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:01.502 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:01.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.502 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:15 3.0s0 3:0a8403ef:::smithi09235267-5:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:01.502 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.484947+0000", 2024-04-26T03:39:01.502 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.00181306, 2024-04-26T03:39:01.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024468274000000002, 2024-04-26T03:39:01.502 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:01.503 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:01.503 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-04-26T03:39:01.503 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:01.503 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:01.503 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:01.503 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15 2024-04-26T03:39:01.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.503 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:01.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.503 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:01.503 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484947+0000", 2024-04-26T03:39:01.503 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:01.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.503 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:01.504 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484947+0000", 2024-04-26T03:39:01.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:01.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.504 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:01.504 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484947+0000", 2024-04-26T03:39:01.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0999999999999997e-07 2024-04-26T03:39:01.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.504 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:01.504 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484952+0000", 2024-04-26T03:39:01.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.549e-06 2024-04-26T03:39:01.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.505 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:01.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484962+0000", 2024-04-26T03:39:01.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0064999999999999e-05 2024-04-26T03:39:01.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.505 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:01.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484972+0000", 2024-04-26T03:39:01.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7529999999999996e-06 2024-04-26T03:39:01.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.505 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:01.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.489714+0000", 2024-04-26T03:39:01.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0047418649999999996 2024-04-26T03:39:01.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:01.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.489836+0000", 2024-04-26T03:39:01.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012253399999999999 2024-04-26T03:39:01.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:01.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.502300+0000", 2024-04-26T03:39:01.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012464098 2024-04-26T03:39:01.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:01.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.502390+0000", 2024-04-26T03:39:01.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9954e-05 2024-04-26T03:39:01.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.507 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:01.507 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.509415+0000", 2024-04-26T03:39:01.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0070245459999999996 2024-04-26T03:39:01.507 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:01.507 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:01.507 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:01.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.507 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:40 3.0s0 3:0e9b8628:::smithi09235267-14:head [append 0~229376,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:01.507 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.495863+0000", 2024-04-26T03:39:01.507 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.9908965670000001, 2024-04-26T03:39:01.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013720777999999999, 2024-04-26T03:39:01.508 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:01.508 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:01.508 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:01.508 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:01.508 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:01.508 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:01.508 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 40 2024-04-26T03:39:01.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.508 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:01.508 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.508 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:01.508 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.495863+0000", 2024-04-26T03:39:01.508 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:01.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.509 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:01.509 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.495863+0000", 2024-04-26T03:39:01.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:01.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.509 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:01.509 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.495865+0000", 2024-04-26T03:39:01.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4090000000000001e-06 2024-04-26T03:39:01.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.509 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:01.509 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496307+0000", 2024-04-26T03:39:01.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00044159800000000002 2024-04-26T03:39:01.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:01.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496322+0000", 2024-04-26T03:39:01.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4935e-05 2024-04-26T03:39:01.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:01.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496338+0000", 2024-04-26T03:39:01.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6155e-05 2024-04-26T03:39:01.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:01.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496408+0000", 2024-04-26T03:39:01.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0183000000000002e-05 2024-04-26T03:39:01.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:01.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496716+0000", 2024-04-26T03:39:01.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000308024 2024-04-26T03:39:01.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:01.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.497681+0000", 2024-04-26T03:39:01.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00096483300000000001 2024-04-26T03:39:01.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:01.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.505243+0000", 2024-04-26T03:39:01.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0075613329999999999 2024-04-26T03:39:01.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.512 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:39:01.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.509516+0000", 2024-04-26T03:39:01.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0042732480000000003 2024-04-26T03:39:01.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.512 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:01.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.509584+0000", 2024-04-26T03:39:01.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8059999999999996e-05 2024-04-26T03:39:01.512 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:01.512 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:01.512 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:01.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.513 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:41 3.0s0 3:0e9b8628:::smithi09235267-14:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:01.513 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.496478+0000", 2024-04-26T03:39:01.513 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.9902813070000001, 2024-04-26T03:39:01.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018225094000000001, 2024-04-26T03:39:01.513 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:01.513 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:01.513 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:01.513 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:01.513 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:01.513 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:01.513 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 41 2024-04-26T03:39:01.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.514 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:01.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:01.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496478+0000", 2024-04-26T03:39:01.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:01.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:01.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496478+0000", 2024-04-26T03:39:01.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:01.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:01.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496479+0000", 2024-04-26T03:39:01.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7700000000000003e-07 2024-04-26T03:39:01.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:01.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496563+0000", 2024-04-26T03:39:01.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3778999999999999e-05 2024-04-26T03:39:01.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:01.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496574+0000", 2024-04-26T03:39:01.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1144999999999999e-05 2024-04-26T03:39:01.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:01.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496586+0000", 2024-04-26T03:39:01.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1457999999999999e-05 2024-04-26T03:39:01.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:01.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.503975+0000", 2024-04-26T03:39:01.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0073896480000000004 2024-04-26T03:39:01.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:01.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504089+0000", 2024-04-26T03:39:01.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011359299999999999 2024-04-26T03:39:01.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:01.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504968+0000", 2024-04-26T03:39:01.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00087954500000000002 2024-04-26T03:39:01.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:01.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.508613+0000", 2024-04-26T03:39:01.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0036446500000000001 2024-04-26T03:39:01.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:39:01.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.514613+0000", 2024-04-26T03:39:01.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0060000000000000001 2024-04-26T03:39:01.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:01.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.514703+0000", 2024-04-26T03:39:01.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0399e-05 2024-04-26T03:39:01.518 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:01.518 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:01.518 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:01.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.519 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:42 3.0s0 3:0e9b8628:::smithi09235267-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:01.519 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.496943+0000", 2024-04-26T03:39:01.519 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.9898165310000007, 2024-04-26T03:39:01.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021178255999999999, 2024-04-26T03:39:01.519 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:01.519 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:01.519 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-04-26T03:39:01.519 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:01.519 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:01.519 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:01.519 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 42 2024-04-26T03:39:01.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.519 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:01.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:01.520 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496943+0000", 2024-04-26T03:39:01.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:01.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:01.520 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496943+0000", 2024-04-26T03:39:01.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:01.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:01.520 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496944+0000", 2024-04-26T03:39:01.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9000000000000005e-07 2024-04-26T03:39:01.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:01.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496951+0000", 2024-04-26T03:39:01.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6470000000000001e-06 2024-04-26T03:39:01.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:01.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496962+0000", 2024-04-26T03:39:01.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1618e-05 2024-04-26T03:39:01.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:01.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496971+0000", 2024-04-26T03:39:01.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6170000000000007e-06 2024-04-26T03:39:01.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:01.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.505538+0000", 2024-04-26T03:39:01.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0085666530000000005 2024-04-26T03:39:01.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:01.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.505620+0000", 2024-04-26T03:39:01.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2669000000000004e-05 2024-04-26T03:39:01.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:01.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.515261+0000", 2024-04-26T03:39:01.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0096405210000000009 2024-04-26T03:39:01.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:01.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.515342+0000", 2024-04-26T03:39:01.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1642000000000002e-05 2024-04-26T03:39:01.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:01.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.518121+0000", 2024-04-26T03:39:01.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002778899 2024-04-26T03:39:01.523 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:01.523 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:01.524 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:01.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.524 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds3 35/32 ECSubWrite(tid=0, reqid=client.4556.0:43, at_version=35'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:39:01.524 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.499996+0000", 2024-04-26T03:39:01.524 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.9867633649999998, 2024-04-26T03:39:01.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0057023960000000002, 2024-04-26T03:39:01.524 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:01.524 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:01.524 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:01.524 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:01.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:01.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.499996+0000", 2024-04-26T03:39:01.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:01.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:01.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.499996+0000", 2024-04-26T03:39:01.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:01.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:01.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.499997+0000", 2024-04-26T03:39:01.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.89e-07 2024-04-26T03:39:01.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:01.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.500102+0000", 2024-04-26T03:39:01.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000105369 2024-04-26T03:39:01.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:01.526 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.500120+0000", 2024-04-26T03:39:01.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7813e-05 2024-04-26T03:39:01.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:01.526 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.500130+0000", 2024-04-26T03:39:01.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6229999999999992e-06 2024-04-26T03:39:01.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:01.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.500195+0000", 2024-04-26T03:39:01.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5394000000000004e-05 2024-04-26T03:39:01.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:01.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.500203+0000", 2024-04-26T03:39:01.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2530000000000004e-06 2024-04-26T03:39:01.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:01.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.505663+0000", 2024-04-26T03:39:01.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0054600910000000002 2024-04-26T03:39:01.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:01.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.505699+0000", 2024-04-26T03:39:01.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5364000000000002e-05 2024-04-26T03:39:01.528 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:01.528 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:01.528 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:01.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.528 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.bs4 35/32 ECSubWrite(tid=6, reqid=client.4556.0:37, at_version=35'3, trim_to=0'0, roll_forward_to=34'2))", 2024-04-26T03:39:01.528 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.501702+0000", 2024-04-26T03:39:01.528 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.9850577319999996, 2024-04-26T03:39:01.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0052680410000000002, 2024-04-26T03:39:01.529 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:01.529 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:01.529 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:01.529 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:01.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:01.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.501702+0000", 2024-04-26T03:39:01.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:01.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:01.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.501702+0000", 2024-04-26T03:39:01.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:01.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:01.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.501703+0000", 2024-04-26T03:39:01.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2799999999999995e-07 2024-04-26T03:39:01.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:01.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.501876+0000", 2024-04-26T03:39:01.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000173318 2024-04-26T03:39:01.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:01.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.501888+0000", 2024-04-26T03:39:01.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2114e-05 2024-04-26T03:39:01.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:01.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.501894+0000", 2024-04-26T03:39:01.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.964e-06 2024-04-26T03:39:01.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:01.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.501951+0000", 2024-04-26T03:39:01.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6764000000000001e-05 2024-04-26T03:39:01.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:01.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.501961+0000", 2024-04-26T03:39:01.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7319999999999993e-06 2024-04-26T03:39:01.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:01.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.506931+0000", 2024-04-26T03:39:01.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0049704759999999997 2024-04-26T03:39:01.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:01.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.506970+0000", 2024-04-26T03:39:01.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8844999999999998e-05 2024-04-26T03:39:01.533 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:01.533 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:01.533 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:01.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.533 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds3 35/32 ECSubWrite(tid=1, reqid=client.4556.0:44, at_version=35'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:39:01.533 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.504056+0000", 2024-04-26T03:39:01.533 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.9827034430000001, 2024-04-26T03:39:01.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0085269190000000009, 2024-04-26T03:39:01.533 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:01.533 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:01.533 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:01.533 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:01.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:01.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504056+0000", 2024-04-26T03:39:01.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:01.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:01.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504056+0000", 2024-04-26T03:39:01.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:01.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:01.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504057+0000", 2024-04-26T03:39:01.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2600000000000001e-07 2024-04-26T03:39:01.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:01.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504078+0000", 2024-04-26T03:39:01.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1250000000000002e-05 2024-04-26T03:39:01.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:01.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504092+0000", 2024-04-26T03:39:01.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3370000000000001e-05 2024-04-26T03:39:01.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:01.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504102+0000", 2024-04-26T03:39:01.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0047000000000001e-05 2024-04-26T03:39:01.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:01.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504408+0000", 2024-04-26T03:39:01.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030663799999999997 2024-04-26T03:39:01.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:01.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504417+0000", 2024-04-26T03:39:01.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8400000000000001e-06 2024-04-26T03:39:01.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:01.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.512551+0000", 2024-04-26T03:39:01.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0081339870000000005 2024-04-26T03:39:01.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:01.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.512583+0000", 2024-04-26T03:39:01.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1961000000000001e-05 2024-04-26T03:39:01.537 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:01.537 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:01.537 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:01.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.537 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds3 35/32 ECSubWrite(tid=2, reqid=client.4556.0:52, at_version=35'3, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:39:01.537 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.506681+0000", 2024-04-26T03:39:01.537 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.9800782599999991, 2024-04-26T03:39:01.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0070048899999999997, 2024-04-26T03:39:01.537 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:01.538 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:01.538 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:01.538 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:01.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:01.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.506681+0000", 2024-04-26T03:39:01.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:01.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:01.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.506681+0000", 2024-04-26T03:39:01.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:01.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:01.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.506682+0000", 2024-04-26T03:39:01.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.04e-06 2024-04-26T03:39:01.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:01.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.506853+0000", 2024-04-26T03:39:01.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017012599999999999 2024-04-26T03:39:01.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:01.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.506866+0000", 2024-04-26T03:39:01.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3244999999999999e-05 2024-04-26T03:39:01.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:01.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.506874+0000", 2024-04-26T03:39:01.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0229999999999992e-06 2024-04-26T03:39:01.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:01.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.508635+0000", 2024-04-26T03:39:01.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0017609380000000001 2024-04-26T03:39:01.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:01.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.508644+0000", 2024-04-26T03:39:01.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0569999999999999e-06 2024-04-26T03:39:01.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:01.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.513648+0000", 2024-04-26T03:39:01.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0050038920000000002 2024-04-26T03:39:01.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:01.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.513686+0000", 2024-04-26T03:39:01.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8569000000000003e-05 2024-04-26T03:39:01.541 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:01.541 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:01.541 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:01.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.542 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 35/32 ECSubWriteReply(tid=4, last_complete=35'4, committed=1, applied=1))", 2024-04-26T03:39:01.542 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.507927+0000", 2024-04-26T03:39:01.542 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.9788323979999998, 2024-04-26T03:39:01.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0053162010000000004, 2024-04-26T03:39:01.542 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:01.542 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:01.542 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:01.542 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:01.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:01.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.507927+0000", 2024-04-26T03:39:01.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:01.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:01.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.507927+0000", 2024-04-26T03:39:01.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:01.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:01.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.507928+0000", 2024-04-26T03:39:01.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.083e-06 2024-04-26T03:39:01.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:01.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.507938+0000", 2024-04-26T03:39:01.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7880000000000001e-06 2024-04-26T03:39:01.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:01.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.512598+0000", 2024-04-26T03:39:01.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0046601029999999996 2024-04-26T03:39:01.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:01.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.512607+0000", 2024-04-26T03:39:01.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.619e-06 2024-04-26T03:39:01.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:01.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.513211+0000", 2024-04-26T03:39:01.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00060419900000000003 2024-04-26T03:39:01.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:01.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.513243+0000", 2024-04-26T03:39:01.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2409000000000001e-05 2024-04-26T03:39:01.545 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:01.545 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:01.545 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:01.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.545 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.8s2 35/32 ECSubWrite(tid=161, reqid=client.4556.0:55, at_version=35'3, trim_to=0'0, roll_forward_to=34'2))", 2024-04-26T03:39:01.545 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.509799+0000", 2024-04-26T03:39:01.545 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.9769607879999995, 2024-04-26T03:39:01.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.005403705, 2024-04-26T03:39:01.545 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:01.546 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:01.546 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:01.546 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:01.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:01.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.509799+0000", 2024-04-26T03:39:01.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:01.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:01.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.509799+0000", 2024-04-26T03:39:01.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:01.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:01.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.509800+0000", 2024-04-26T03:39:01.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5300000000000003e-07 2024-04-26T03:39:01.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:01.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.509859+0000", 2024-04-26T03:39:01.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9777e-05 2024-04-26T03:39:01.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:01.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.509873+0000", 2024-04-26T03:39:01.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3259e-05 2024-04-26T03:39:01.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:01.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.509881+0000", 2024-04-26T03:39:01.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0239999999999997e-06 2024-04-26T03:39:01.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:01.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.510016+0000", 2024-04-26T03:39:01.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000135809 2024-04-26T03:39:01.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:01.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.510037+0000", 2024-04-26T03:39:01.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0395e-05 2024-04-26T03:39:01.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:01.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.515171+0000", 2024-04-26T03:39:01.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0051341199999999998 2024-04-26T03:39:01.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:01.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.515203+0000", 2024-04-26T03:39:01.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1568e-05 2024-04-26T03:39:01.549 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:01.549 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:01.549 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:01.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.549 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.as5 35/32 ECSubWrite(tid=4, reqid=client.4556.0:73, at_version=35'3, trim_to=0'0, roll_forward_to=34'2))", 2024-04-26T03:39:01.549 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.525773+0000", 2024-04-26T03:39:01.550 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.9609870619999992, 2024-04-26T03:39:01.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0073138420000000001, 2024-04-26T03:39:01.550 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:01.550 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:01.550 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:01.550 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:01.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:01.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.525773+0000", 2024-04-26T03:39:01.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:01.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:01.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.525773+0000", 2024-04-26T03:39:01.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:01.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:01.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.525773+0000", 2024-04-26T03:39:01.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0699999999999996e-07 2024-04-26T03:39:01.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:01.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.525972+0000", 2024-04-26T03:39:01.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019857100000000001 2024-04-26T03:39:01.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:01.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.525984+0000", 2024-04-26T03:39:01.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2099999999999999e-05 2024-04-26T03:39:01.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:01.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.525990+0000", 2024-04-26T03:39:01.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7320000000000003e-06 2024-04-26T03:39:01.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:01.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.526063+0000", 2024-04-26T03:39:01.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3499999999999998e-05 2024-04-26T03:39:01.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:01.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.526074+0000", 2024-04-26T03:39:01.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0899e-05 2024-04-26T03:39:01.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:01.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.533045+0000", 2024-04-26T03:39:01.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0069703400000000002 2024-04-26T03:39:01.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:01.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.533086+0000", 2024-04-26T03:39:01.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1893e-05 2024-04-26T03:39:01.554 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:01.554 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:01.554 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:01.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.555 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds3 35/32 ECSubWrite(tid=7, reqid=client.4556.0:64, at_version=35'5, trim_to=0'0, roll_forward_to=35'4))", 2024-04-26T03:39:01.555 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.528525+0000", 2024-04-26T03:39:01.555 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.9582350989999995, 2024-04-26T03:39:01.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.006326946, 2024-04-26T03:39:01.555 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:01.555 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:01.555 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:01.555 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:01.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:01.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528525+0000", 2024-04-26T03:39:01.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:01.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:01.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528525+0000", 2024-04-26T03:39:01.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:01.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:01.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528526+0000", 2024-04-26T03:39:01.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9199999999999999e-07 2024-04-26T03:39:01.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:01.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528649+0000", 2024-04-26T03:39:01.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012333600000000001 2024-04-26T03:39:01.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:01.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.529481+0000", 2024-04-26T03:39:01.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00083257199999999996 2024-04-26T03:39:01.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:01.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.529492+0000", 2024-04-26T03:39:01.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0769e-05 2024-04-26T03:39:01.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:01.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.529566+0000", 2024-04-26T03:39:01.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3756000000000003e-05 2024-04-26T03:39:01.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:01.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.529578+0000", 2024-04-26T03:39:01.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2513e-05 2024-04-26T03:39:01.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:01.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.534815+0000", 2024-04-26T03:39:01.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0052369510000000001 2024-04-26T03:39:01.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:01.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.534851+0000", 2024-04-26T03:39:01.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6056999999999997e-05 2024-04-26T03:39:01.559 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:01.559 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:01.559 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:01.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.559 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.3s4 35/32 ECSubWrite(tid=4, reqid=client.4556.0:88, at_version=35'3, trim_to=0'0, roll_forward_to=34'2))", 2024-04-26T03:39:01.559 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.545174+0000", 2024-04-26T03:39:01.559 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.9415853930000004, 2024-04-26T03:39:01.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0052690339999999997, 2024-04-26T03:39:01.559 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:01.559 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:01.559 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:01.559 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:01.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:01.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.545174+0000", 2024-04-26T03:39:01.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:01.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:01.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.545174+0000", 2024-04-26T03:39:01.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:01.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:01.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.545176+0000", 2024-04-26T03:39:01.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3009999999999999e-06 2024-04-26T03:39:01.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:01.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.545582+0000", 2024-04-26T03:39:01.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00040631100000000002 2024-04-26T03:39:01.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:01.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.545596+0000", 2024-04-26T03:39:01.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4365000000000001e-05 2024-04-26T03:39:01.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:01.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.545607+0000", 2024-04-26T03:39:01.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0774999999999999e-05 2024-04-26T03:39:01.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:01.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.545680+0000", 2024-04-26T03:39:01.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2818999999999995e-05 2024-04-26T03:39:01.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:01.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.545692+0000", 2024-04-26T03:39:01.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1769e-05 2024-04-26T03:39:01.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:01.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.550405+0000", 2024-04-26T03:39:01.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0047133890000000001 2024-04-26T03:39:01.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:01.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.550443+0000", 2024-04-26T03:39:01.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8305e-05 2024-04-26T03:39:01.563 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:01.563 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:01.563 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:01.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.563 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.as5 35/32 ECSubWrite(tid=12, reqid=client.4556.0:115, at_version=35'7, trim_to=0'0, roll_forward_to=35'6))", 2024-04-26T03:39:01.563 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.579868+0000", 2024-04-26T03:39:01.563 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.9068914340000003, 2024-04-26T03:39:01.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0052991979999999998, 2024-04-26T03:39:01.563 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:01.563 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:01.564 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:01.564 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:01.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:01.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.579868+0000", 2024-04-26T03:39:01.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:01.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:01.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.579868+0000", 2024-04-26T03:39:01.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:01.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:01.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.579869+0000", 2024-04-26T03:39:01.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4600000000000003e-07 2024-04-26T03:39:01.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:01.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.580319+0000", 2024-04-26T03:39:01.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00045016799999999998 2024-04-26T03:39:01.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:01.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.580332+0000", 2024-04-26T03:39:01.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2639e-05 2024-04-26T03:39:01.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:01.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.580340+0000", 2024-04-26T03:39:01.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8480000000000001e-06 2024-04-26T03:39:01.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:01.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.580399+0000", 2024-04-26T03:39:01.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9122000000000001e-05 2024-04-26T03:39:01.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:01.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.580404+0000", 2024-04-26T03:39:01.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6699999999999999e-06 2024-04-26T03:39:01.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:01.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.585141+0000", 2024-04-26T03:39:01.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0047369270000000002 2024-04-26T03:39:01.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:01.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.585167+0000", 2024-04-26T03:39:01.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5978000000000001e-05 2024-04-26T03:39:01.567 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:01.567 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:01.567 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:01.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.567 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.bs4 35/32 ECSubWrite(tid=20, reqid=client.4556.0:127, at_version=35'7, trim_to=0'0, roll_forward_to=35'6))", 2024-04-26T03:39:01.567 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.601259+0000", 2024-04-26T03:39:01.567 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.8855003809999999, 2024-04-26T03:39:01.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0054673129999999997, 2024-04-26T03:39:01.570 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:01.570 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:01.570 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:01.570 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:01.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:01.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.601259+0000", 2024-04-26T03:39:01.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:01.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:01.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.601259+0000", 2024-04-26T03:39:01.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:01.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:01.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.601260+0000", 2024-04-26T03:39:01.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3900000000000004e-07 2024-04-26T03:39:01.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:01.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.601863+0000", 2024-04-26T03:39:01.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00060349500000000001 2024-04-26T03:39:01.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:01.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.601879+0000", 2024-04-26T03:39:01.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5471000000000001e-05 2024-04-26T03:39:01.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:01.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.601888+0000", 2024-04-26T03:39:01.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8540000000000003e-06 2024-04-26T03:39:01.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:01.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.601961+0000", 2024-04-26T03:39:01.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3249000000000003e-05 2024-04-26T03:39:01.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:01.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.601971+0000", 2024-04-26T03:39:01.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0197000000000001e-05 2024-04-26T03:39:01.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:01.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.606694+0000", 2024-04-26T03:39:01.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0047231010000000004 2024-04-26T03:39:01.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:01.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.606727+0000", 2024-04-26T03:39:01.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2206999999999998e-05 2024-04-26T03:39:01.573 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:01.573 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:01.573 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:01.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.574 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.fs5 35/32 ECSubWrite(tid=15, reqid=client.4556.0:136, at_version=35'5, trim_to=0'0, roll_forward_to=35'4))", 2024-04-26T03:39:01.574 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.616287+0000", 2024-04-26T03:39:01.574 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.8704729090000001, 2024-04-26T03:39:01.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0056710830000000004, 2024-04-26T03:39:01.574 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:01.574 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:01.574 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:01.574 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:01.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:01.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.616287+0000", 2024-04-26T03:39:01.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:01.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:01.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.616287+0000", 2024-04-26T03:39:01.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:01.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:01.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.616288+0000", 2024-04-26T03:39:01.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3600000000000002e-07 2024-04-26T03:39:01.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:01.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.616564+0000", 2024-04-26T03:39:01.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00027615700000000002 2024-04-26T03:39:01.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:01.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.616578+0000", 2024-04-26T03:39:01.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3895e-05 2024-04-26T03:39:01.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:01.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.616585+0000", 2024-04-26T03:39:01.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8960000000000003e-06 2024-04-26T03:39:01.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:01.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.616649+0000", 2024-04-26T03:39:01.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3744000000000005e-05 2024-04-26T03:39:01.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:01.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.616658+0000", 2024-04-26T03:39:01.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9570000000000008e-06 2024-04-26T03:39:01.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:01.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.621924+0000", 2024-04-26T03:39:01.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0052660459999999999 2024-04-26T03:39:01.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:01.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.621958+0000", 2024-04-26T03:39:01.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3552e-05 2024-04-26T03:39:01.579 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:01.579 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:01.579 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:01.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.579 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.8s2 40/32 ECSubWrite(tid=178, reqid=client.4556.0:266, at_version=40'8, trim_to=0'0, roll_forward_to=35'6))", 2024-04-26T03:39:01.579 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:56.534993+0000", 2024-04-26T03:39:01.579 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.9517671090000004, 2024-04-26T03:39:01.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0072588619999999996, 2024-04-26T03:39:01.580 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:01.580 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:01.580 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:01.580 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:01.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:01.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.534993+0000", 2024-04-26T03:39:01.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:01.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:01.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.534993+0000", 2024-04-26T03:39:01.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:01.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:01.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.534993+0000", 2024-04-26T03:39:01.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4799999999999998e-07 2024-04-26T03:39:01.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:01.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.535281+0000", 2024-04-26T03:39:01.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00028805200000000001 2024-04-26T03:39:01.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:01.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.535292+0000", 2024-04-26T03:39:01.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.045e-05 2024-04-26T03:39:01.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:01.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.535298+0000", 2024-04-26T03:39:01.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0959999999999997e-06 2024-04-26T03:39:01.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:01.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.535358+0000", 2024-04-26T03:39:01.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0448000000000001e-05 2024-04-26T03:39:01.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:01.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.535366+0000", 2024-04-26T03:39:01.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9419999999999995e-06 2024-04-26T03:39:01.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:01.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.542217+0000", 2024-04-26T03:39:01.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0068503519999999997 2024-04-26T03:39:01.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:01.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.542251+0000", 2024-04-26T03:39:01.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4873999999999998e-05 2024-04-26T03:39:01.583 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:01.584 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:01.584 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:01.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.584 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.as5 43/32 ECSubWrite(tid=54, reqid=client.4556.0:310, at_version=43'28, trim_to=0'0, roll_forward_to=42'26))", 2024-04-26T03:39:01.584 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:59.549519+0000", 2024-04-26T03:39:01.584 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 1.9372410260000001, 2024-04-26T03:39:01.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0069345320000000002, 2024-04-26T03:39:01.584 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:01.585 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:01.585 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:01.585 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:01.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:01.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.549519+0000", 2024-04-26T03:39:01.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:01.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:01.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.549519+0000", 2024-04-26T03:39:01.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:01.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:01.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.549519+0000", 2024-04-26T03:39:01.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4700000000000001e-07 2024-04-26T03:39:01.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:01.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.550000+0000", 2024-04-26T03:39:01.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00048121699999999998 2024-04-26T03:39:01.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:01.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.550028+0000", 2024-04-26T03:39:01.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7840000000000001e-05 2024-04-26T03:39:01.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:01.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.550053+0000", 2024-04-26T03:39:01.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5690999999999998e-05 2024-04-26T03:39:01.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:01.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.550134+0000", 2024-04-26T03:39:01.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0686000000000006e-05 2024-04-26T03:39:01.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:01.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.550144+0000", 2024-04-26T03:39:01.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0242e-05 2024-04-26T03:39:01.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:01.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.556415+0000", 2024-04-26T03:39:01.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0062707470000000001 2024-04-26T03:39:01.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:01.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:01.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:01.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.556453+0000", 2024-04-26T03:39:01.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7962e-05 2024-04-26T03:39:01.588 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:01.588 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:01.588 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:01.588 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:01.588 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:01.588 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-26T03:39:01.589 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-26T03:39:01.591 INFO:tasks.ceph.osd.12.smithi028.stderr:2024-04-26T03:39:01.588+0000 7f114fe14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 31967) UID: 0 2024-04-26T03:39:01.592 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-26T03:39:01.594 INFO:tasks.rados.rados.0.smithi092.stdout:127: done (0 left) 2024-04-26T03:39:01.594 INFO:tasks.rados.rados.0.smithi092.stdout:128: read oid 48 snap -1 2024-04-26T03:39:01.594 INFO:tasks.rados.rados.0.smithi092.stdout:128: expect (ObjNum 47 snap 0 seq_num 47) 2024-04-26T03:39:01.594 INFO:tasks.rados.rados.0.smithi092.stdout:129: read oid 45 snap -1 2024-04-26T03:39:01.594 INFO:tasks.rados.rados.0.smithi092.stdout:129: expect (ObjNum 44 snap 0 seq_num 44) 2024-04-26T03:39:01.594 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 9 current snap is 6 2024-04-26T03:39:01.595 INFO:tasks.rados.rados.0.smithi092.stdout:130: seq_num 76 ranges {147456=507904,655360=507904,1163264=262144} 2024-04-26T03:39:01.595 INFO:tasks.rados.rados.0.smithi092.stdout:130: writing smithi09235267-9 from 147456 to 655360 tid 1 2024-04-26T03:39:01.595 INFO:tasks.rados.rados.0.smithi092.stdout:130: writing smithi09235267-9 from 655360 to 1163264 tid 2 2024-04-26T03:39:01.595 INFO:tasks.rados.rados.0.smithi092.stdout:130: writing smithi09235267-9 from 1163264 to 1425408 tid 3 2024-04-26T03:39:01.595 INFO:tasks.rados.rados.0.smithi092.stdout:131: rollback oid 19 current snap is 6 2024-04-26T03:39:01.595 INFO:tasks.rados.rados.0.smithi092.stdout:rollback oid 19 to 5 2024-04-26T03:39:01.595 INFO:tasks.rados.rados.0.smithi092.stdout:132: copy_from oid 47 from oid 33 current snap is 6 2024-04-26T03:39:01.595 INFO:tasks.rados.rados.0.smithi092.stdout:133: read oid 23 snap -1 2024-04-26T03:39:01.595 INFO:tasks.rados.rados.0.smithi092.stdout:133: expect deleted 2024-04-26T03:39:01.595 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 11 current snap is 6 2024-04-26T03:39:01.595 INFO:tasks.rados.rados.0.smithi092.stdout:134: seq_num 77 ranges {1179648=524288,1703936=524288,2228224=245760} 2024-04-26T03:39:01.595 INFO:tasks.rados.rados.0.smithi092.stdout:134: writing smithi09235267-11 from 1179648 to 1703936 tid 1 2024-04-26T03:39:01.595 INFO:tasks.rados.rados.0.smithi092.stdout:134: writing smithi09235267-11 from 1703936 to 2228224 tid 2 2024-04-26T03:39:01.595 INFO:tasks.rados.rados.0.smithi092.stdout:134: writing smithi09235267-11 from 2228224 to 2473984 tid 3 2024-04-26T03:39:01.596 INFO:tasks.rados.rados.0.smithi092.stdout:135: rollback oid 26 current snap is 6 2024-04-26T03:39:01.596 INFO:tasks.rados.rados.0.smithi092.stdout:rollback oid 26 to 5 2024-04-26T03:39:01.596 INFO:tasks.rados.rados.0.smithi092.stdout:129: expect (ObjNum 44 snap 0 seq_num 44) 2024-04-26T03:39:01.598 INFO:tasks.rados.rados.0.smithi092.stdout:128: expect (ObjNum 47 snap 0 seq_num 47) 2024-04-26T03:39:01.620 INFO:tasks.rados.rados.0.smithi092.stdout:131: finishing rollback tid 0 to smithi09235267-19 2024-04-26T03:39:01.620 INFO:tasks.rados.rados.0.smithi092.stdout:131: finishing rollback tid 1 to smithi09235267-19 2024-04-26T03:39:01.620 INFO:tasks.rados.rados.0.smithi092.stdout:131: finishing rollback tid 2 to smithi09235267-19 2024-04-26T03:39:01.621 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 19 v 16 (ObjNum 66 snap 3 seq_num 66) dirty exists 2024-04-26T03:39:01.621 INFO:tasks.rados.rados.0.smithi092.stdout:130: finishing write tid 1 to smithi09235267-9 2024-04-26T03:39:01.621 INFO:tasks.rados.rados.0.smithi092.stdout:130: finishing write tid 2 to smithi09235267-9 2024-04-26T03:39:01.621 INFO:tasks.rados.rados.0.smithi092.stdout:132: finishing copy_from racing read to smithi09235267-47 2024-04-26T03:39:01.621 INFO:tasks.rados.rados.0.smithi092.stdout:130: finishing write tid 3 to smithi09235267-9 2024-04-26T03:39:01.621 INFO:tasks.rados.rados.0.smithi092.stdout:130: finishing write tid 4 to smithi09235267-9 2024-04-26T03:39:01.621 INFO:tasks.rados.rados.0.smithi092.stdout:130: finishing write tid 5 to smithi09235267-9 2024-04-26T03:39:01.621 INFO:tasks.rados.rados.0.smithi092.stdout:130: oid 9 updating version 0 to 20 2024-04-26T03:39:01.621 INFO:tasks.rados.rados.0.smithi092.stdout:130: oid 9 version 20 is already newer than 18 2024-04-26T03:39:01.621 INFO:tasks.rados.rados.0.smithi092.stdout:130: oid 9 version 20 is already newer than 17 2024-04-26T03:39:01.621 INFO:tasks.rados.rados.0.smithi092.stdout:130: oid 9 version 20 is already newer than 19 2024-04-26T03:39:01.621 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 9 v 20 (ObjNum 76 snap 6 seq_num 76) dirty exists 2024-04-26T03:39:01.622 INFO:tasks.rados.rados.0.smithi092.stdout:130: left oid 9 (ObjNum 76 snap 6 seq_num 76) 2024-04-26T03:39:01.622 INFO:tasks.rados.rados.0.smithi092.stdout:132: finishing copy_from to smithi09235267-47 2024-04-26T03:39:01.622 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 47 v 6 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-04-26T03:39:01.622 INFO:tasks.rados.rados.0.smithi092.stdout:135: finishing rollback tid 0 to smithi09235267-26 2024-04-26T03:39:01.622 INFO:tasks.rados.rados.0.smithi092.stdout:135: finishing rollback tid 1 to smithi09235267-26 2024-04-26T03:39:01.622 INFO:tasks.rados.rados.0.smithi092.stdout:128: done (7 left) 2024-04-26T03:39:01.622 INFO:tasks.rados.rados.0.smithi092.stdout:129: done (6 left) 2024-04-26T03:39:01.622 INFO:tasks.rados.rados.0.smithi092.stdout:130: done (5 left) 2024-04-26T03:39:01.622 INFO:tasks.rados.rados.0.smithi092.stdout:131: done (4 left) 2024-04-26T03:39:01.622 INFO:tasks.rados.rados.0.smithi092.stdout:132: done (3 left) 2024-04-26T03:39:01.622 INFO:tasks.rados.rados.0.smithi092.stdout:133: done (2 left) 2024-04-26T03:39:01.622 INFO:tasks.rados.rados.0.smithi092.stdout:136: read oid 29 snap -1 2024-04-26T03:39:01.623 INFO:tasks.rados.rados.0.smithi092.stdout:136: expect (ObjNum 28 snap 0 seq_num 28) 2024-04-26T03:39:01.623 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 24 current snap is 6 2024-04-26T03:39:01.623 INFO:tasks.rados.rados.0.smithi092.stdout:135: finishing rollback tid 2 to smithi09235267-26 2024-04-26T03:39:01.623 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 26 v 10 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-04-26T03:39:01.623 INFO:tasks.rados.rados.0.smithi092.stdout:134: finishing write tid 1 to smithi09235267-11 2024-04-26T03:39:01.623 INFO:tasks.rados.rados.0.smithi092.stdout:137: seq_num 78 ranges {1605632=557056,2162688=671744,2834432=98304} 2024-04-26T03:39:01.631 INFO:tasks.rados.rados.0.smithi092.stdout:137: writing smithi09235267-24 from 1605632 to 2162688 tid 1 2024-04-26T03:39:01.635 INFO:tasks.rados.rados.0.smithi092.stdout:137: writing smithi09235267-24 from 2162688 to 2834432 tid 2 2024-04-26T03:39:01.636 INFO:tasks.rados.rados.0.smithi092.stdout:137: writing smithi09235267-24 from 2834432 to 2932736 tid 3 2024-04-26T03:39:01.636 INFO:tasks.rados.rados.0.smithi092.stdout:135: done (3 left) 2024-04-26T03:39:01.636 INFO:tasks.rados.rados.0.smithi092.stdout:138: read oid 30 snap -1 2024-04-26T03:39:01.636 INFO:tasks.rados.rados.0.smithi092.stdout:138: expect (ObjNum 74 snap 5 seq_num 74) 2024-04-26T03:39:01.636 INFO:tasks.rados.rados.0.smithi092.stdout:139: read oid 4 snap 4 2024-04-26T03:39:01.636 INFO:tasks.rados.rados.0.smithi092.stdout:139: expect (ObjNum 56 snap 0 seq_num 56) 2024-04-26T03:39:01.636 INFO:tasks.rados.rados.0.smithi092.stdout:140: rollback oid 50 current snap is 6 2024-04-26T03:39:01.637 INFO:tasks.rados.rados.0.smithi092.stdout:rollback oid 50 to 5 2024-04-26T03:39:01.637 INFO:tasks.rados.rados.0.smithi092.stdout:134: finishing write tid 2 to smithi09235267-11 2024-04-26T03:39:01.637 INFO:tasks.rados.rados.0.smithi092.stdout:134: finishing write tid 3 to smithi09235267-11 2024-04-26T03:39:01.637 INFO:tasks.rados.rados.0.smithi092.stdout:134: finishing write tid 4 to smithi09235267-11 2024-04-26T03:39:01.637 INFO:tasks.rados.rados.0.smithi092.stdout:134: finishing write tid 5 to smithi09235267-11 2024-04-26T03:39:01.637 INFO:tasks.rados.rados.0.smithi092.stdout:134: oid 11 updating version 0 to 22 2024-04-26T03:39:01.637 INFO:tasks.rados.rados.0.smithi092.stdout:134: oid 11 version 22 is already newer than 21 2024-04-26T03:39:01.637 INFO:tasks.rados.rados.0.smithi092.stdout:134: oid 11 updating version 22 to 23 2024-04-26T03:39:01.637 INFO:tasks.rados.rados.0.smithi092.stdout:134: oid 11 updating version 23 to 24 2024-04-26T03:39:01.637 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 11 v 24 (ObjNum 77 snap 6 seq_num 77) dirty exists 2024-04-26T03:39:01.637 INFO:tasks.rados.rados.0.smithi092.stdout:134: left oid 11 (ObjNum 77 snap 6 seq_num 77) 2024-04-26T03:39:01.637 INFO:tasks.rados.rados.0.smithi092.stdout:136: expect (ObjNum 28 snap 0 seq_num 28) 2024-04-26T03:39:01.639 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-26T03:39:01.651 INFO:tasks.rados.rados.0.smithi092.stdout:140: finishing rollback tid 1 to smithi09235267-50 2024-04-26T03:39:01.652 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 50 v 26 (ObjNum 2513 snap 0 seq_num 3583212928) dirty dne 2024-04-26T03:39:01.652 INFO:tasks.rados.rados.0.smithi092.stdout:137: finishing write tid 1 to smithi09235267-24 2024-04-26T03:39:01.652 INFO:tasks.rados.rados.0.smithi092.stdout:139: expect (ObjNum 56 snap 0 seq_num 56) 2024-04-26T03:39:01.691 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:39:01.689+0000 7fba0d2ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31961) UID: 0 2024-04-26T03:39:01.707 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-26T03:39:01.708 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-04-26T03:39:01.708 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-04-26T03:39:01.708 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-26T03:39:01.711 INFO:tasks.rados.rados.0.smithi092.stdout:137: finishing write tid 2 to smithi09235267-24 2024-04-26T03:39:01.711 INFO:tasks.rados.rados.0.smithi092.stdout:137: finishing write tid 3 to smithi09235267-24 2024-04-26T03:39:01.711 INFO:tasks.rados.rados.0.smithi092.stdout:137: finishing write tid 4 to smithi09235267-24 2024-04-26T03:39:01.711 INFO:tasks.rados.rados.0.smithi092.stdout:137: finishing write tid 5 to smithi09235267-24 2024-04-26T03:39:01.711 INFO:tasks.rados.rados.0.smithi092.stdout:137: oid 24 updating version 0 to 23 2024-04-26T03:39:01.712 INFO:tasks.rados.rados.0.smithi092.stdout:137: oid 24 version 23 is already newer than 22 2024-04-26T03:39:01.712 INFO:tasks.rados.rados.0.smithi092.stdout:137: oid 24 version 23 is already newer than 21 2024-04-26T03:39:01.712 INFO:tasks.rados.rados.0.smithi092.stdout:137: oid 24 updating version 23 to 24 2024-04-26T03:39:01.712 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 24 v 24 (ObjNum 78 snap 6 seq_num 78) dirty exists 2024-04-26T03:39:01.712 INFO:tasks.rados.rados.0.smithi092.stdout:137: left oid 24 (ObjNum 78 snap 6 seq_num 78) 2024-04-26T03:39:01.712 INFO:tasks.rados.rados.0.smithi092.stdout:134: done (5 left) 2024-04-26T03:39:01.712 INFO:tasks.rados.rados.0.smithi092.stdout:136: done (4 left) 2024-04-26T03:39:01.712 INFO:tasks.rados.rados.0.smithi092.stdout:137: done (3 left) 2024-04-26T03:39:01.712 INFO:tasks.rados.rados.0.smithi092.stdout:139: done (2 left) 2024-04-26T03:39:01.712 INFO:tasks.rados.rados.0.smithi092.stdout:140: done (1 left) 2024-04-26T03:39:01.712 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 43 current snap is 6 2024-04-26T03:39:01.712 INFO:tasks.rados.rados.0.smithi092.stdout:141: seq_num 79 ranges {0=573440,573440=425984,999424=344064} 2024-04-26T03:39:01.714 INFO:tasks.rados.rados.0.smithi092.stdout:141: writing smithi09235267-43 from 0 to 573440 tid 1 2024-04-26T03:39:01.716 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-26T03:39:01.717 INFO:tasks.rados.rados.0.smithi092.stdout:141: writing smithi09235267-43 from 573440 to 999424 tid 2 2024-04-26T03:39:01.718 INFO:tasks.rados.rados.0.smithi092.stdout:141: writing smithi09235267-43 from 999424 to 1343488 tid 3 2024-04-26T03:39:01.718 INFO:tasks.rados.rados.0.smithi092.stdout:142: snap_create 2024-04-26T03:39:01.718 INFO:tasks.rados.rados.0.smithi092.stdout:138: expect (ObjNum 74 snap 5 seq_num 74) 2024-04-26T03:39:01.728 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0: {} 2024-04-26T03:39:01.728 INFO:teuthology.orchestra.run.smithi028.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T03:39:01.749 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1: {} 2024-04-26T03:39:01.749 INFO:teuthology.orchestra.run.smithi028.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T03:39:01.770 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2: {} 2024-04-26T03:39:01.770 INFO:teuthology.orchestra.run.smithi028.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T03:39:01.787 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3: {} 2024-04-26T03:39:01.788 INFO:teuthology.orchestra.run.smithi028.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T03:39:01.792 INFO:tasks.ceph.osd.10.smithi066.stderr:2024-04-26T03:39:01.791+0000 7f130fcc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 31763) UID: 0 2024-04-26T03:39:01.804 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4: {} 2024-04-26T03:39:01.805 INFO:teuthology.orchestra.run.smithi028.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T03:39:01.821 INFO:tasks.rados.rados.0.smithi092.stdout:141: finishing write tid 1 to smithi09235267-43 2024-04-26T03:39:01.821 INFO:tasks.rados.rados.0.smithi092.stdout:141: finishing write tid 2 to smithi09235267-43 2024-04-26T03:39:01.821 INFO:tasks.rados.rados.0.smithi092.stdout:141: finishing write tid 3 to smithi09235267-43 2024-04-26T03:39:01.821 INFO:tasks.rados.rados.0.smithi092.stdout:141: finishing write tid 4 to smithi09235267-43 2024-04-26T03:39:01.821 INFO:tasks.rados.rados.0.smithi092.stdout:141: finishing write tid 5 to smithi09235267-43 2024-04-26T03:39:01.822 INFO:tasks.rados.rados.0.smithi092.stdout:141: oid 43 updating version 0 to 31 2024-04-26T03:39:01.822 INFO:tasks.rados.rados.0.smithi092.stdout:141: oid 43 updating version 31 to 33 2024-04-26T03:39:01.822 INFO:tasks.rados.rados.0.smithi092.stdout:141: oid 43 updating version 33 to 34 2024-04-26T03:39:01.822 INFO:tasks.rados.rados.0.smithi092.stdout:141: oid 43 version 34 is already newer than 32 2024-04-26T03:39:01.822 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 43 v 34 (ObjNum 79 snap 6 seq_num 79) dirty exists 2024-04-26T03:39:01.822 INFO:tasks.rados.rados.0.smithi092.stdout:141: left oid 43 (ObjNum 79 snap 6 seq_num 79) 2024-04-26T03:39:01.824 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5: {} 2024-04-26T03:39:01.824 INFO:teuthology.orchestra.run.smithi028.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T03:39:01.826 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:01.826 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:01.836 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-26T03:39:01.841 INFO:teuthology.orchestra.run.smithi028.stdout:osd.6: {} 2024-04-26T03:39:01.841 INFO:teuthology.orchestra.run.smithi028.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T03:39:01.858 INFO:teuthology.orchestra.run.smithi028.stdout:osd.7: {} 2024-04-26T03:39:01.859 INFO:teuthology.orchestra.run.smithi028.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T03:39:01.876 INFO:teuthology.orchestra.run.smithi028.stdout:osd.8: {} 2024-04-26T03:39:01.876 INFO:teuthology.orchestra.run.smithi028.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T03:39:01.892 INFO:tasks.ceph.osd.9.smithi042.stderr:2024-04-26T03:39:01.891+0000 7fdac63b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 31960) UID: 0 2024-04-26T03:39:01.893 INFO:teuthology.orchestra.run.smithi028.stdout:osd.9: {} 2024-04-26T03:39:01.893 INFO:teuthology.orchestra.run.smithi028.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T03:39:01.911 INFO:teuthology.orchestra.run.smithi028.stdout:osd.10: {} 2024-04-26T03:39:01.912 INFO:teuthology.orchestra.run.smithi028.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T03:39:01.928 INFO:teuthology.orchestra.run.smithi028.stdout:osd.11: {} 2024-04-26T03:39:01.928 INFO:teuthology.orchestra.run.smithi028.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T03:39:01.942 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:01.943 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:01.946 INFO:teuthology.orchestra.run.smithi028.stdout:osd.12: {} 2024-04-26T03:39:01.946 INFO:teuthology.orchestra.run.smithi028.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T03:39:01.951 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-04-26T03:39:01.963 INFO:teuthology.orchestra.run.smithi028.stdout:osd.13: {} 2024-04-26T03:39:01.963 INFO:teuthology.orchestra.run.smithi028.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T03:39:01.990 INFO:teuthology.orchestra.run.smithi028.stdout:osd.14: {} 2024-04-26T03:39:01.990 INFO:teuthology.orchestra.run.smithi028.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T03:39:01.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:39:01.989+0000 7fc72ed9d640 0 -- 172.21.15.28:0/4246214285 >> [v2:172.21.15.92:6826/3982634323,v1:172.21.15.92:6827/3982634323] conn(0x7fc7302229d0 msgr2=0x7fc730224dc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-26T03:39:01.993 INFO:tasks.ceph.osd.4.smithi028.stderr:2024-04-26T03:39:01.991+0000 7fd9511b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31963) UID: 0 2024-04-26T03:39:02.009 INFO:teuthology.orchestra.run.smithi028.stdout:osd.15: {} 2024-04-26T03:39:02.009 INFO:teuthology.orchestra.run.smithi028.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T03:39:02.093 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:39:02.092+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:39:02.099 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:02.099 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:02.108 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-04-26T03:39:02.194 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-04-26T03:39:02.193+0000 7fdc2d21c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31819) UID: 0 2024-04-26T03:39:02.255 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:02.255 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:02.264 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-04-26T03:39:02.307 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:39:02.306+0000 7fb78c544640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31956) UID: 0 2024-04-26T03:39:02.371 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:02.371 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:02.380 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-04-26T03:39:02.421 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:39:02.419+0000 7fb78c544640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31956) UID: 0 2024-04-26T03:39:02.498 INFO:tasks.ceph.osd.9.smithi042.stderr:2024-04-26T03:39:02.496+0000 7fdac63b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 31960) UID: 0 2024-04-26T03:39:02.526 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:02.526 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:02.535 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-04-26T03:39:02.537 INFO:teuthology.orchestra.run.smithi028.stderr:nodeep-scrub is set 2024-04-26T03:39:02.540 INFO:tasks.rados.rados.0.smithi092.stdout:138: done (2 left) 2024-04-26T03:39:02.540 INFO:tasks.rados.rados.0.smithi092.stdout:141: done (1 left) 2024-04-26T03:39:02.540 INFO:tasks.rados.rados.0.smithi092.stdout:142: done (0 left) 2024-04-26T03:39:02.540 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 6 current snap is 7 2024-04-26T03:39:02.541 INFO:tasks.rados.rados.0.smithi092.stdout:143: seq_num 80 ranges {98304=589824,688128=573440,1261568=196608} 2024-04-26T03:39:02.544 INFO:tasks.rados.rados.0.smithi092.stdout:143: writing smithi09235267-6 from 98304 to 688128 tid 1 2024-04-26T03:39:02.547 INFO:tasks.rados.rados.0.smithi092.stdout:143: writing smithi09235267-6 from 688128 to 1261568 tid 2 2024-04-26T03:39:02.548 INFO:tasks.rados.rados.0.smithi092.stdout:143: writing smithi09235267-6 from 1261568 to 1458176 tid 3 2024-04-26T03:39:02.548 INFO:tasks.rados.rados.0.smithi092.stdout:144: setattr oid 31 current snap is 7 2024-04-26T03:39:02.549 INFO:tasks.rados.rados.0.smithi092.stdout:145: rollback oid 10 current snap is 7 2024-04-26T03:39:02.549 INFO:tasks.rados.rados.0.smithi092.stdout:rollback oid 10 to 5 2024-04-26T03:39:02.550 INFO:tasks.rados.rados.0.smithi092.stdout:146: read oid 39 snap -1 2024-04-26T03:39:02.550 INFO:tasks.rados.rados.0.smithi092.stdout:146: expect (ObjNum 38 snap 0 seq_num 38) 2024-04-26T03:39:02.550 INFO:tasks.rados.rados.0.smithi092.stdout:147: rollback oid 9 current snap is 7 2024-04-26T03:39:02.550 INFO:tasks.rados.rados.0.smithi092.stdout:rollback oid 9 to 5 2024-04-26T03:39:02.550 INFO:tasks.rados.rados.0.smithi092.stdout:148: copy_from oid 27 from oid 19 current snap is 7 2024-04-26T03:39:02.550 INFO:tasks.rados.rados.0.smithi092.stdout:149: delete oid 20 current snap is 7 2024-04-26T03:39:02.555 INFO:tasks.rados.rados.0.smithi092.stdout:145: finishing rollback tid 0 to smithi09235267-10 2024-04-26T03:39:02.557 INFO:tasks.rados.rados.0.smithi092.stdout:145: finishing rollback tid 1 to smithi09235267-10 2024-04-26T03:39:02.557 INFO:tasks.rados.rados.0.smithi092.stdout:145: finishing rollback tid 2 to smithi09235267-10 2024-04-26T03:39:02.557 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 10 v 30 (ObjNum 54 snap 0 seq_num 54) dirty exists 2024-04-26T03:39:02.557 INFO:tasks.rados.rados.0.smithi092.stdout:146: expect (ObjNum 38 snap 0 seq_num 38) 2024-04-26T03:39:02.583 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 31 v 18 (ObjNum 30 snap 0 seq_num 30) dirty exists 2024-04-26T03:39:02.583 INFO:tasks.rados.rados.0.smithi092.stdout:147: finishing rollback tid 0 to smithi09235267-9 2024-04-26T03:39:02.583 INFO:tasks.rados.rados.0.smithi092.stdout:147: finishing rollback tid 1 to smithi09235267-9 2024-04-26T03:39:02.584 INFO:tasks.rados.rados.0.smithi092.stdout:147: finishing rollback tid 2 to smithi09235267-9 2024-04-26T03:39:02.584 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 9 v 24 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-04-26T03:39:02.584 INFO:tasks.rados.rados.0.smithi092.stdout:143: finishing write tid 1 to smithi09235267-6 2024-04-26T03:39:02.584 INFO:tasks.rados.rados.0.smithi092.stdout:143: finishing write tid 2 to smithi09235267-6 2024-04-26T03:39:02.584 INFO:tasks.rados.rados.0.smithi092.stdout:143: finishing write tid 3 to smithi09235267-6 2024-04-26T03:39:02.584 INFO:tasks.rados.rados.0.smithi092.stdout:143: finishing write tid 4 to smithi09235267-6 2024-04-26T03:39:02.584 INFO:tasks.rados.rados.0.smithi092.stdout:143: finishing write tid 5 to smithi09235267-6 2024-04-26T03:39:02.584 INFO:tasks.rados.rados.0.smithi092.stdout:143: oid 6 updating version 0 to 22 2024-04-26T03:39:02.584 INFO:tasks.rados.rados.0.smithi092.stdout:143: oid 6 updating version 22 to 24 2024-04-26T03:39:02.584 INFO:tasks.rados.rados.0.smithi092.stdout:143: oid 6 version 24 is already newer than 21 2024-04-26T03:39:02.584 INFO:tasks.rados.rados.0.smithi092.stdout:143: oid 6 version 24 is already newer than 23 2024-04-26T03:39:02.584 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 6 v 24 (ObjNum 80 snap 7 seq_num 80) dirty exists 2024-04-26T03:39:02.584 INFO:tasks.rados.rados.0.smithi092.stdout:143: left oid 6 (ObjNum 80 snap 7 seq_num 80) 2024-04-26T03:39:02.584 INFO:tasks.rados.rados.0.smithi092.stdout:148: finishing copy_from racing read to smithi09235267-27 2024-04-26T03:39:02.585 INFO:tasks.rados.rados.0.smithi092.stdout:148: finishing copy_from to smithi09235267-27 2024-04-26T03:39:02.585 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 27 v 16 (ObjNum 66 snap 3 seq_num 66) dirty exists 2024-04-26T03:39:02.585 INFO:tasks.rados.rados.0.smithi092.stdout:143: done (6 left) 2024-04-26T03:39:02.585 INFO:tasks.rados.rados.0.smithi092.stdout:144: done (5 left) 2024-04-26T03:39:02.585 INFO:tasks.rados.rados.0.smithi092.stdout:145: done (4 left) 2024-04-26T03:39:02.585 INFO:tasks.rados.rados.0.smithi092.stdout:146: done (3 left) 2024-04-26T03:39:02.585 INFO:tasks.rados.rados.0.smithi092.stdout:147: done (2 left) 2024-04-26T03:39:02.585 INFO:tasks.rados.rados.0.smithi092.stdout:148: done (1 left) 2024-04-26T03:39:02.585 INFO:tasks.rados.rados.0.smithi092.stdout:149: done (0 left) 2024-04-26T03:39:02.585 INFO:tasks.rados.rados.0.smithi092.stdout:150: read oid 23 snap -1 2024-04-26T03:39:02.585 INFO:tasks.rados.rados.0.smithi092.stdout:150: expect deleted 2024-04-26T03:39:02.585 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 11 current snap is 7 2024-04-26T03:39:02.585 INFO:tasks.rados.rados.0.smithi092.stdout:151: seq_num 82 ranges {2473984=622592,3096576=475136,3571712=294912} 2024-04-26T03:39:02.599 INFO:tasks.rados.rados.0.smithi092.stdout:151: writing smithi09235267-11 from 2473984 to 3096576 tid 1 2024-04-26T03:39:02.603 INFO:tasks.rados.rados.0.smithi092.stdout:151: writing smithi09235267-11 from 3096576 to 3571712 tid 2 2024-04-26T03:39:02.604 INFO:tasks.rados.rados.0.smithi092.stdout:151: writing smithi09235267-11 from 3571712 to 3866624 tid 3 2024-04-26T03:39:02.604 INFO:tasks.rados.rados.0.smithi092.stdout:152: read oid 18 snap -1 2024-04-26T03:39:02.604 INFO:tasks.rados.rados.0.smithi092.stdout:152: expect deleted 2024-04-26T03:39:02.604 INFO:tasks.rados.rados.0.smithi092.stdout:153: setattr oid 50 current snap is 7 2024-04-26T03:39:02.605 INFO:tasks.rados.rados.0.smithi092.stdout:150: done (3 left) 2024-04-26T03:39:02.605 INFO:tasks.rados.rados.0.smithi092.stdout:152: done (2 left) 2024-04-26T03:39:02.605 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 17 current snap is 7 2024-04-26T03:39:02.605 INFO:tasks.rados.rados.0.smithi092.stdout:154: seq_num 83 ranges {0=638976,638976=622592,1261568=147456} 2024-04-26T03:39:02.609 INFO:tasks.rados.rados.0.smithi092.stdout:154: writing smithi09235267-17 from 0 to 638976 tid 1 2024-04-26T03:39:02.612 INFO:tasks.rados.rados.0.smithi092.stdout:154: writing smithi09235267-17 from 638976 to 1261568 tid 2 2024-04-26T03:39:02.615 INFO:tasks.rados.rados.0.smithi092.stdout:154: writing smithi09235267-17 from 1261568 to 1409024 tid 3 2024-04-26T03:39:02.615 INFO:tasks.rados.rados.0.smithi092.stdout:155: rollback oid 19 current snap is 7 2024-04-26T03:39:02.615 INFO:tasks.rados.rados.0.smithi092.stdout:rollback oid 19 to 4 2024-04-26T03:39:02.615 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 50 v 30 (ObjNum 33 snap 0 seq_num 3583233136) dirty exists 2024-04-26T03:39:02.615 INFO:tasks.rados.rados.0.smithi092.stdout:153: done (3 left) 2024-04-26T03:39:02.615 INFO:tasks.rados.rados.0.smithi092.stdout:156: read oid 37 snap 3 2024-04-26T03:39:02.615 INFO:tasks.rados.rados.0.smithi092.stdout:156: expect (ObjNum 36 snap 0 seq_num 36) 2024-04-26T03:39:02.615 INFO:tasks.rados.rados.0.smithi092.stdout:157: rollback oid 23 current snap is 7 2024-04-26T03:39:02.615 INFO:tasks.rados.rados.0.smithi092.stdout:rollback oid 23 to 4 2024-04-26T03:39:02.615 INFO:tasks.rados.rados.0.smithi092.stdout:158: read oid 14 snap -1 2024-04-26T03:39:02.616 INFO:tasks.rados.rados.0.smithi092.stdout:158: expect (ObjNum 52 snap 0 seq_num 52) 2024-04-26T03:39:02.616 INFO:tasks.rados.rados.0.smithi092.stdout:159: read oid 38 snap -1 2024-04-26T03:39:02.616 INFO:tasks.rados.rados.0.smithi092.stdout:159: expect (ObjNum 37 snap 0 seq_num 37) 2024-04-26T03:39:02.616 INFO:tasks.rados.rados.0.smithi092.stdout:160: copy_from oid 1 from oid 44 current snap is 7 2024-04-26T03:39:02.616 INFO:tasks.rados.rados.0.smithi092.stdout:161: rollback oid 15 current snap is 7 2024-04-26T03:39:02.616 INFO:tasks.rados.rados.0.smithi092.stdout:rollback oid 15 to 4 2024-04-26T03:39:02.616 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 48 current snap is 7 2024-04-26T03:39:02.616 INFO:tasks.rados.rados.0.smithi092.stdout:162: seq_num 84 ranges {802816=655360,1458176=638976,2097152=131072} 2024-04-26T03:39:02.621 INFO:tasks.ceph.osd.9.smithi042.stderr:2024-04-26T03:39:02.619+0000 7fdac63b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 31960) UID: 0 2024-04-26T03:39:02.623 INFO:tasks.rados.rados.0.smithi092.stdout:162: writing smithi09235267-48 from 802816 to 1458176 tid 1 2024-04-26T03:39:02.627 INFO:tasks.rados.rados.0.smithi092.stdout:162: writing smithi09235267-48 from 1458176 to 2097152 tid 2 2024-04-26T03:39:02.629 INFO:tasks.rados.rados.0.smithi092.stdout:162: writing smithi09235267-48 from 2097152 to 2228224 tid 3 2024-04-26T03:39:02.629 INFO:tasks.rados.rados.0.smithi092.stdout:163: snap_remove snap 5 2024-04-26T03:39:02.694 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:02.694 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:02.699 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-04-26T03:39:02.698+0000 7f745118d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 31821) UID: 0 2024-04-26T03:39:02.702 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-04-26T03:39:02.800 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-04-26T03:39:02.798+0000 7fe9223c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31954) UID: 0 2024-04-26T03:39:02.849 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:02.850 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:02.858 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-26T03:39:02.900 INFO:tasks.ceph.osd.8.smithi028.stderr:2024-04-26T03:39:02.899+0000 7f5bb75d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 31968) UID: 0 2024-04-26T03:39:02.975 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:02.975 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:02.984 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-26T03:39:03.002 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:39:03.000+0000 7fb78c544640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31956) UID: 0 2024-04-26T03:39:03.091 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:03.091 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:03.099 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-26T03:39:03.103 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-04-26T03:39:03.101+0000 7fdc2d21c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31819) UID: 0 2024-04-26T03:39:03.203 INFO:tasks.ceph.osd.4.smithi028.stderr:2024-04-26T03:39:03.202+0000 7fd9511b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31963) UID: 0 2024-04-26T03:39:03.206 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:03.206 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:03.215 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-26T03:39:03.305 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:39:03.303+0000 7fba0d2ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31961) UID: 0 2024-04-26T03:39:03.323 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:03.323 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:03.332 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-26T03:39:03.406 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:39:03.405+0000 7f60d3c4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31757) UID: 0 2024-04-26T03:39:03.439 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:03.440 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:03.448 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-26T03:39:03.506 INFO:tasks.ceph.osd.12.smithi028.stderr:2024-04-26T03:39:03.504+0000 7f114fe14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 31967) UID: 0 2024-04-26T03:39:03.539 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-04-26T03:39:03.539 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-26T03:39:03.539 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_0 2024-04-26T03:39:03.539 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T03:39:03.544 INFO:tasks.rados.rados.0.smithi092.stdout:163: done (10 left) 2024-04-26T03:39:03.544 INFO:tasks.rados.rados.0.smithi092.stdout:164: setattr oid 21 current snap is 7 2024-04-26T03:39:03.545 INFO:tasks.rados.rados.0.smithi092.stdout:151: finishing write tid 1 to smithi09235267-11 2024-04-26T03:39:03.545 INFO:tasks.rados.rados.0.smithi092.stdout:151: finishing write tid 2 to smithi09235267-11 2024-04-26T03:39:03.545 INFO:tasks.rados.rados.0.smithi092.stdout:159: expect (ObjNum 37 snap 0 seq_num 37) 2024-04-26T03:39:03.569 INFO:tasks.rados.rados.0.smithi092.stdout:161: finishing rollback tid 0 to smithi09235267-15 2024-04-26T03:39:03.570 INFO:tasks.rados.rados.0.smithi092.stdout:157: finishing rollback tid 1 to smithi09235267-23 2024-04-26T03:39:03.570 INFO:tasks.rados.rados.0.smithi092.stdout:161: finishing rollback tid 1 to smithi09235267-15 2024-04-26T03:39:03.570 INFO:tasks.rados.rados.0.smithi092.stdout:156: expect (ObjNum 36 snap 0 seq_num 36) 2024-04-26T03:39:03.585 INFO:tasks.rados.rados.0.smithi092.stdout:157: finishing rollback tid 2 to smithi09235267-23 2024-04-26T03:39:03.585 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 23 v 32 (ObjNum 22 snap 0 seq_num 22) dirty exists 2024-04-26T03:39:03.585 INFO:tasks.rados.rados.0.smithi092.stdout:155: finishing rollback tid 0 to smithi09235267-19 2024-04-26T03:39:03.586 INFO:tasks.rados.rados.0.smithi092.stdout:161: finishing rollback tid 2 to smithi09235267-15 2024-04-26T03:39:03.586 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 15 v 13 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-04-26T03:39:03.586 INFO:tasks.rados.rados.0.smithi092.stdout:155: finishing rollback tid 1 to smithi09235267-19 2024-04-26T03:39:03.586 INFO:tasks.rados.rados.0.smithi092.stdout:155: finishing rollback tid 2 to smithi09235267-19 2024-04-26T03:39:03.586 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 19 v 22 (ObjNum 66 snap 3 seq_num 66) dirty exists 2024-04-26T03:39:03.586 INFO:tasks.rados.rados.0.smithi092.stdout:158: expect (ObjNum 52 snap 0 seq_num 52) 2024-04-26T03:39:03.603 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:03.604 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:03.607 INFO:tasks.ceph.osd.12.smithi028.stderr:2024-04-26T03:39:03.605+0000 7f114fe14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 31967) UID: 0 2024-04-26T03:39:03.613 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-04-26T03:39:03.615 INFO:tasks.rados.rados.0.smithi092.stdout:151: finishing write tid 3 to smithi09235267-11 2024-04-26T03:39:03.615 INFO:tasks.rados.rados.0.smithi092.stdout:151: finishing write tid 4 to smithi09235267-11 2024-04-26T03:39:03.615 INFO:tasks.rados.rados.0.smithi092.stdout:154: finishing write tid 1 to smithi09235267-17 2024-04-26T03:39:03.615 INFO:tasks.rados.rados.0.smithi092.stdout:154: finishing write tid 2 to smithi09235267-17 2024-04-26T03:39:03.615 INFO:tasks.rados.rados.0.smithi092.stdout:154: finishing write tid 3 to smithi09235267-17 2024-04-26T03:39:03.615 INFO:tasks.rados.rados.0.smithi092.stdout:154: finishing write tid 4 to smithi09235267-17 2024-04-26T03:39:03.616 INFO:tasks.rados.rados.0.smithi092.stdout:151: finishing write tid 5 to smithi09235267-11 2024-04-26T03:39:03.616 INFO:tasks.rados.rados.0.smithi092.stdout:151: oid 11 updating version 0 to 26 2024-04-26T03:39:03.616 INFO:tasks.rados.rados.0.smithi092.stdout:151: oid 11 updating version 26 to 28 2024-04-26T03:39:03.616 INFO:tasks.rados.rados.0.smithi092.stdout:151: oid 11 updating version 28 to 29 2024-04-26T03:39:03.616 INFO:tasks.rados.rados.0.smithi092.stdout:151: oid 11 version 29 is already newer than 27 2024-04-26T03:39:03.616 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 11 v 29 (ObjNum 82 snap 7 seq_num 82) dirty exists 2024-04-26T03:39:03.616 INFO:tasks.rados.rados.0.smithi092.stdout:151: left oid 11 (ObjNum 82 snap 7 seq_num 82) 2024-04-26T03:39:03.617 INFO:tasks.rados.rados.0.smithi092.stdout:154: finishing write tid 5 to smithi09235267-17 2024-04-26T03:39:03.617 INFO:tasks.rados.rados.0.smithi092.stdout:154: oid 17 updating version 0 to 20 2024-04-26T03:39:03.617 INFO:tasks.rados.rados.0.smithi092.stdout:154: oid 17 updating version 20 to 21 2024-04-26T03:39:03.617 INFO:tasks.rados.rados.0.smithi092.stdout:154: oid 17 version 21 is already newer than 19 2024-04-26T03:39:03.617 INFO:tasks.rados.rados.0.smithi092.stdout:154: oid 17 updating version 21 to 22 2024-04-26T03:39:03.618 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 17 v 22 (ObjNum 83 snap 7 seq_num 83) dirty exists 2024-04-26T03:39:03.618 INFO:tasks.rados.rados.0.smithi092.stdout:154: left oid 17 (ObjNum 83 snap 7 seq_num 83) 2024-04-26T03:39:03.618 INFO:tasks.rados.rados.0.smithi092.stdout:160: finishing copy_from racing read to smithi09235267-1 2024-04-26T03:39:03.618 INFO:tasks.rados.rados.0.smithi092.stdout:160: finishing copy_from to smithi09235267-1 2024-04-26T03:39:03.618 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 1 v 12 (ObjNum 43 snap 0 seq_num 43) dirty exists 2024-04-26T03:39:03.619 INFO:tasks.rados.rados.0.smithi092.stdout:162: finishing write tid 1 to smithi09235267-48 2024-04-26T03:39:03.619 INFO:tasks.rados.rados.0.smithi092.stdout:151: done (10 left) 2024-04-26T03:39:03.619 INFO:tasks.rados.rados.0.smithi092.stdout:154: done (9 left) 2024-04-26T03:39:03.619 INFO:tasks.rados.rados.0.smithi092.stdout:155: done (8 left) 2024-04-26T03:39:03.619 INFO:tasks.rados.rados.0.smithi092.stdout:156: done (7 left) 2024-04-26T03:39:03.619 INFO:tasks.rados.rados.0.smithi092.stdout:157: done (6 left) 2024-04-26T03:39:03.619 INFO:tasks.rados.rados.0.smithi092.stdout:158: done (5 left) 2024-04-26T03:39:03.621 INFO:tasks.rados.rados.0.smithi092.stdout:159: done (4 left) 2024-04-26T03:39:03.621 INFO:tasks.rados.rados.0.smithi092.stdout:160: done (3 left) 2024-04-26T03:39:03.621 INFO:tasks.rados.rados.0.smithi092.stdout:161: done (2 left) 2024-04-26T03:39:03.621 INFO:tasks.rados.rados.0.smithi092.stdout:165: rmattr oid 43 current snap is 7 2024-04-26T03:39:03.622 INFO:tasks.rados.rados.0.smithi092.stdout:165: done (2 left) 2024-04-26T03:39:03.622 INFO:tasks.rados.rados.0.smithi092.stdout:166: snap_remove snap 4 2024-04-26T03:39:03.708 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:39:03.706+0000 7fba0d2ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31961) UID: 0 2024-04-26T03:39:03.729 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:03.729 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:03.738 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-04-26T03:39:03.809 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:39:03.808+0000 7f60d3c4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31757) UID: 0 2024-04-26T03:39:03.845 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:03.845 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:03.853 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-04-26T03:39:03.855 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:39:03.856 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-04-26T03:39:03.870 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":48,"stamp":"2024-04-26T03:39:03.538690+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":40419872,"num_objects":60,"num_object_clones":14,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":2,"num_read":339,"num_read_kb":40109,"num_write":511,"num_write_kb":47524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":527,"ondisk_log_size":527,"up":200,"acting":200,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":163,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":525972,"kb_used_data":76912,"kb_used_omap":24,"kb_used_meta":448743,"kb_avail":1509423468,"statfs":{"total":1546188226560,"available":1545649631232,"internally_reserved":0,"allocated":78757888,"data_stored":88915971,"data_compressed":686498,"data_compressed_allocated":1466368,"data_compressed_original":3825664,"omap_allocated":25522,"internal_metadata":459512910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7929856,"num_objects":5,"num_object_clones":5,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":29,"num_read_kb":19754,"num_write":75,"num_write_kb":9504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.018808"},"pg_stats":[{"pgid":"3.19","version":"45'10","reported_seq":86,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T03:39:01.580162+0000","last_change":"2024-04-26T03:39:01.545345+0000","last_active":"2024-04-26T03:39:01.580162+0000","last_peered":"2024-04-26T03:39:01.580162+0000","last_clean":"2024-04-26T03:39:01.580162+0000","last_became_active":"2024-04-26T03:39:01.544381+0000","last_became_peered":"2024-04-26T03:39:01.544381+0000","last_unstale":"2024-04-26T03:39:01.580162+0000","last_undegraded":"2024-04-26T03:39:01.580162+0000","last_fullsized":"2024-04-26T03:39:01.580162+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:07.282240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":614400,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":626,"num_write":5,"num_write_kb":600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2,10,9,15,8,4],"acting":[13,2,10,9,15,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"3.18","version":"45'16","reported_seq":99,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T03:39:01.568225+0000","last_change":"2024-04-26T03:39:01.554794+0000","last_active":"2024-04-26T03:39:01.568225+0000","last_peered":"2024-04-26T03:39:01.568225+0000","last_clean":"2024-04-26T03:39:01.568225+0000","last_became_active":"2024-04-26T03:39:01.553702+0000","last_became_peered":"2024-04-26T03:39:01.553702+0000","last_unstale":"2024-04-26T03:39:01.568225+0000","last_undegraded":"2024-04-26T03:39:01.568225+0000","last_fullsized":"2024-04-26T03:39:01.568225+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:07.073488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":10,"num_read_kb":964,"num_write":7,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,10,11,15,5],"acting":[6,0,1,10,11,15,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"3.17","version":"43'6","reported_seq":80,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T03:39:02.539005+0000","last_change":"2024-04-26T03:39:01.553030+0000","last_active":"2024-04-26T03:39:02.539005+0000","last_peered":"2024-04-26T03:39:02.539005+0000","last_clean":"2024-04-26T03:39:02.539005+0000","last_became_active":"2024-04-26T03:39:01.550697+0000","last_became_peered":"2024-04-26T03:39:01.550697+0000","last_unstale":"2024-04-26T03:39:02.539005+0000","last_undegraded":"2024-04-26T03:39:02.539005+0000","last_fullsized":"2024-04-26T03:39:02.539005+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65536,"num_objects":1,"num_object_clones":1,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,13,6,0,14,10,11],"acting":[4,13,6,0,14,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"3.16","version":"35'3","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T03:39:02.554395+0000","last_change":"2024-04-26T03:39:01.547476+0000","last_active":"2024-04-26T03:39:02.554395+0000","last_peered":"2024-04-26T03:39:02.554395+0000","last_clean":"2024-04-26T03:39:02.554395+0000","last_became_active":"2024-04-26T03:39:01.546594+0000","last_became_peered":"2024-04-26T03:39:01.546594+0000","last_unstale":"2024-04-26T03:39:02.554395+0000","last_undegraded":"2024-04-26T03:39:02.554395+0000","last_fullsized":"2024-04-26T03:39:02.554395+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:08.256795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":409600,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,14,6,2,12,7,1],"acting":[0,14,6,2,12,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"3.15","version":"42'24","reported_seq":97,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T03:39:01.545139+0000","last_change":"2024-04-26T03:39:01.543974+0000","last_active":"2024-04-26T03:39:01.545139+0000","last_peered":"2024-04-26T03:39:01.545139+0000","last_clean":"2024-04-26T03:39:01.545139+0000","last_became_active":"2024-04-26T03:39:01.543191+0000","last_became_peered":"2024-04-26T03:39:01.543191+0000","last_unstale":"2024-04-26T03:39:01.545139+0000","last_undegraded":"2024-04-26T03:39:01.545139+0000","last_fullsized":"2024-04-26T03:39:01.545139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:07.208096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":729088,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":555,"num_write":16,"num_write_kb":712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,8,4,2,7,0,5],"acting":[15,8,4,2,7,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"3.14","version":"42'18","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T03:39:01.552360+0000","last_change":"2024-04-26T03:39:01.551560+0000","last_active":"2024-04-26T03:39:01.552360+0000","last_peered":"2024-04-26T03:39:01.552360+0000","last_clean":"2024-04-26T03:39:01.552360+0000","last_became_active":"2024-04-26T03:39:01.550254+0000","last_became_peered":"2024-04-26T03:39:01.550254+0000","last_unstale":"2024-04-26T03:39:01.552360+0000","last_undegraded":"2024-04-26T03:39:01.552360+0000","last_fullsized":"2024-04-26T03:39:01.552360+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1376256,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":8,"num_write_kb":1768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,13,4,2,6,10,5],"acting":[8,13,4,2,6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"2.7","version":"0'0","reported_seq":46,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T03:39:01.531210+0000","last_change":"2024-04-26T03:38:37.359252+0000","last_active":"2024-04-26T03:39:01.531210+0000","last_peered":"2024-04-26T03:39:01.531210+0000","last_clean":"2024-04-26T03:39:01.531210+0000","last_became_active":"2024-04-26T03:38:37.358910+0000","last_became_peered":"2024-04-26T03:38:37.358910+0000","last_unstale":"2024-04-26T03:39:01.531210+0000","last_undegraded":"2024-04-26T03:39:01.531210+0000","last_fullsized":"2024-04-26T03:39:01.531210+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:04.186001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"3.6","version":"35'3","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T03:39:02.554424+0000","last_change":"2024-04-26T03:39:01.544471+0000","last_active":"2024-04-26T03:39:02.554424+0000","last_peered":"2024-04-26T03:39:02.554424+0000","last_clean":"2024-04-26T03:39:02.554424+0000","last_became_active":"2024-04-26T03:39:01.543735+0000","last_became_peered":"2024-04-26T03:39:01.543735+0000","last_unstale":"2024-04-26T03:39:02.554424+0000","last_undegraded":"2024-04-26T03:39:02.554424+0000","last_fullsized":"2024-04-26T03:39:02.554424+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053142800000000002,"stat_sum":{"num_bytes":409600,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":1,"num_write_kb":400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,14,6,2,12,7,1],"acting":[0,14,6,2,12,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"2.6","version":"0'0","reported_seq":46,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T03:39:01.529398+0000","last_change":"2024-04-26T03:38:36.484782+0000","last_active":"2024-04-26T03:39:01.529398+0000","last_peered":"2024-04-26T03:39:01.529398+0000","last_clean":"2024-04-26T03:39:01.529398+0000","last_became_active":"2024-04-26T03:38:36.484388+0000","last_became_peered":"2024-04-26T03:38:36.484388+0000","last_unstale":"2024-04-26T03:39:01.529398+0000","last_undegraded":"2024-04-26T03:39:01.529398+0000","last_fullsized":"2024-04-26T03:39:01.529398+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:36.318287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,13],"acting":[8,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.7","version":"43'6","reported_seq":81,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T03:39:02.539308+0000","last_change":"2024-04-26T03:39:01.553901+0000","last_active":"2024-04-26T03:39:02.539308+0000","last_peered":"2024-04-26T03:39:02.539308+0000","last_clean":"2024-04-26T03:39:02.539308+0000","last_became_active":"2024-04-26T03:39:01.549431+0000","last_became_peered":"2024-04-26T03:39:01.549431+0000","last_unstale":"2024-04-26T03:39:02.539308+0000","last_undegraded":"2024-04-26T03:39:02.539308+0000","last_fullsized":"2024-04-26T03:39:02.539308+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055862700000000002,"stat_sum":{"num_bytes":65536,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,13,6,0,14,10,11],"acting":[4,13,6,0,14,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"2.5","version":"0'0","reported_seq":45,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T03:39:01.531593+0000","last_change":"2024-04-26T03:38:38.187714+0000","last_active":"2024-04-26T03:39:01.531593+0000","last_peered":"2024-04-26T03:39:01.531593+0000","last_clean":"2024-04-26T03:39:01.531593+0000","last_became_active":"2024-04-26T03:38:38.187383+0000","last_became_peered":"2024-04-26T03:38:38.187383+0000","last_unstale":"2024-04-26T03:39:01.531593+0000","last_undegraded":"2024-04-26T03:39:01.531593+0000","last_fullsized":"2024-04-26T03:39:01.531593+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:59.062623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4],"acting":[9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.4","version":"42'18","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T03:39:01.545168+0000","last_change":"2024-04-26T03:39:01.545168+0000","last_active":"2024-04-26T03:39:01.545168+0000","last_peered":"2024-04-26T03:39:01.545168+0000","last_clean":"2024-04-26T03:39:01.545168+0000","last_became_active":"2024-04-26T03:39:01.543050+0000","last_became_peered":"2024-04-26T03:39:01.543050+0000","last_unstale":"2024-04-26T03:39:01.545168+0000","last_undegraded":"2024-04-26T03:39:01.545168+0000","last_fullsized":"2024-04-26T03:39:01.545168+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0077667109999999999,"stat_sum":{"num_bytes":1376256,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":7,"num_write_kb":1768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,13,4,2,6,10,5],"acting":[8,13,4,2,6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T03:39:02.539171+0000","last_change":"2024-04-26T03:38:37.255389+0000","last_active":"2024-04-26T03:39:02.539171+0000","last_peered":"2024-04-26T03:39:02.539171+0000","last_clean":"2024-04-26T03:39:02.539171+0000","last_became_active":"2024-04-26T03:38:37.255066+0000","last_became_peered":"2024-04-26T03:38:37.255066+0000","last_unstale":"2024-04-26T03:39:02.539171+0000","last_undegraded":"2024-04-26T03:39:02.539171+0000","last_fullsized":"2024-04-26T03:39:02.539171+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:43.860480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.5","version":"45'26","reported_seq":99,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T03:39:01.544665+0000","last_change":"2024-04-26T03:39:01.543021+0000","last_active":"2024-04-26T03:39:01.544665+0000","last_peered":"2024-04-26T03:39:01.544665+0000","last_clean":"2024-04-26T03:39:01.544665+0000","last_became_active":"2024-04-26T03:39:01.540394+0000","last_became_peered":"2024-04-26T03:39:01.540394+0000","last_unstale":"2024-04-26T03:39:01.544665+0000","last_undegraded":"2024-04-26T03:39:01.544665+0000","last_fullsized":"2024-04-26T03:39:01.544665+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0089281789999999996,"stat_sum":{"num_bytes":729088,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":5,"num_read_kb":554,"num_write":16,"num_write_kb":712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,8,4,2,7,0,5],"acting":[15,8,4,2,7,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"3.3","version":"43'19","reported_seq":102,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T03:39:01.560613+0000","last_change":"2024-04-26T03:39:01.552134+0000","last_active":"2024-04-26T03:39:01.560613+0000","last_peered":"2024-04-26T03:39:01.560613+0000","last_clean":"2024-04-26T03:39:01.560613+0000","last_became_active":"2024-04-26T03:39:01.550287+0000","last_became_peered":"2024-04-26T03:39:01.550287+0000","last_unstale":"2024-04-26T03:39:01.560613+0000","last_undegraded":"2024-04-26T03:39:01.560613+0000","last_fullsized":"2024-04-26T03:39:01.560613+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099631799999999994,"stat_sum":{"num_bytes":2400256,"num_objects":3,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":3532,"num_write":9,"num_write_kb":2344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,7,4,15,1,2,3],"acting":[10,7,4,15,1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"2.2","version":"0'0","reported_seq":34,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T03:39:00.533591+0000","last_change":"2024-04-26T03:38:48.475569+0000","last_active":"2024-04-26T03:39:00.533591+0000","last_peered":"2024-04-26T03:39:00.533591+0000","last_clean":"2024-04-26T03:39:00.533591+0000","last_became_active":"2024-04-26T03:38:48.475155+0000","last_became_peered":"2024-04-26T03:38:48.475155+0000","last_unstale":"2024-04-26T03:39:00.533591+0000","last_undegraded":"2024-04-26T03:39:00.533591+0000","last_fullsized":"2024-04-26T03:39:00.533591+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:38.214686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,12],"acting":[11,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.0","version":"35'9","reported_seq":86,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T03:39:02.550328+0000","last_change":"2024-04-26T03:39:01.541490+0000","last_active":"2024-04-26T03:39:02.550328+0000","last_peered":"2024-04-26T03:39:02.550328+0000","last_clean":"2024-04-26T03:39:02.550328+0000","last_became_active":"2024-04-26T03:39:01.540712+0000","last_became_peered":"2024-04-26T03:39:01.540712+0000","last_unstale":"2024-04-26T03:39:02.550328+0000","last_undegraded":"2024-04-26T03:39:02.550328+0000","last_fullsized":"2024-04-26T03:39:02.550328+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091056600000000002,"stat_sum":{"num_bytes":507904,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,9,15,2,6,14,4],"acting":[1,9,15,2,6,14,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"2.1","version":"0'0","reported_seq":46,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T03:39:01.530909+0000","last_change":"2024-04-26T03:38:37.255450+0000","last_active":"2024-04-26T03:39:01.530909+0000","last_peered":"2024-04-26T03:39:01.530909+0000","last_clean":"2024-04-26T03:39:01.530909+0000","last_became_active":"2024-04-26T03:38:37.255181+0000","last_became_peered":"2024-04-26T03:38:37.255181+0000","last_unstale":"2024-04-26T03:39:01.530909+0000","last_undegraded":"2024-04-26T03:39:01.530909+0000","last_fullsized":"2024-04-26T03:39:01.530909+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:00.449388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0],"acting":[14,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.1","version":"42'8","reported_seq":82,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T03:39:02.539055+0000","last_change":"2024-04-26T03:39:01.548176+0000","last_active":"2024-04-26T03:39:02.539055+0000","last_peered":"2024-04-26T03:39:02.539055+0000","last_clean":"2024-04-26T03:39:02.539055+0000","last_became_active":"2024-04-26T03:39:01.547451+0000","last_became_peered":"2024-04-26T03:39:01.547451+0000","last_unstale":"2024-04-26T03:39:02.539055+0000","last_undegraded":"2024-04-26T03:39:02.539055+0000","last_fullsized":"2024-04-26T03:39:02.539055+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0079599159999999992,"stat_sum":{"num_bytes":1310720,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":1,"num_write_kb":1608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,14,10,9,1,2],"acting":[4,11,14,10,9,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"2.0","version":"0'0","reported_seq":39,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-26T03:38:58.523565+0000","last_change":"2024-04-26T03:38:36.431325+0000","last_active":"2024-04-26T03:38:58.523565+0000","last_peered":"2024-04-26T03:38:58.523565+0000","last_clean":"2024-04-26T03:38:58.523565+0000","last_became_active":"2024-04-26T03:38:36.431121+0000","last_became_peered":"2024-04-26T03:38:36.431121+0000","last_unstale":"2024-04-26T03:38:58.523565+0000","last_undegraded":"2024-04-26T03:38:58.523565+0000","last_fullsized":"2024-04-26T03:38:58.523565+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:53.294619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"45'12","reported_seq":89,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T03:39:01.571537+0000","last_change":"2024-04-26T03:39:01.546681+0000","last_active":"2024-04-26T03:39:01.571537+0000","last_peered":"2024-04-26T03:39:01.571537+0000","last_clean":"2024-04-26T03:39:01.571537+0000","last_became_active":"2024-04-26T03:39:01.543967+0000","last_became_peered":"2024-04-26T03:39:01.543967+0000","last_unstale":"2024-04-26T03:39:01.571537+0000","last_undegraded":"2024-04-26T03:39:01.571537+0000","last_fullsized":"2024-04-26T03:39:01.571537+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041848699999999999,"stat_sum":{"num_bytes":1089536,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":6,"num_write_kb":1664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,5,3,9,15,10,11],"acting":[14,5,3,9,15,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"2.3","version":"0'0","reported_seq":46,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T03:39:01.531094+0000","last_change":"2024-04-26T03:38:36.431667+0000","last_active":"2024-04-26T03:39:01.531094+0000","last_peered":"2024-04-26T03:39:01.531094+0000","last_clean":"2024-04-26T03:39:01.531094+0000","last_became_active":"2024-04-26T03:38:36.431449+0000","last_became_peered":"2024-04-26T03:38:36.431449+0000","last_unstale":"2024-04-26T03:39:01.531094+0000","last_undegraded":"2024-04-26T03:39:01.531094+0000","last_fullsized":"2024-04-26T03:39:01.531094+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:47.562648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"27'157","reported_seq":247,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T03:39:01.534174+0000","last_change":"2024-04-26T03:38:34.390206+0000","last_active":"2024-04-26T03:39:01.534174+0000","last_peered":"2024-04-26T03:39:01.534174+0000","last_clean":"2024-04-26T03:39:01.534174+0000","last_became_active":"2024-04-26T03:38:34.389099+0000","last_became_peered":"2024-04-26T03:38:34.389099+0000","last_unstale":"2024-04-26T03:39:01.534174+0000","last_undegraded":"2024-04-26T03:39:01.534174+0000","last_fullsized":"2024-04-26T03:39:01.534174+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:32.306810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:32.306810+0000","last_clean_scrub_stamp":"2024-04-26T03:38:32.306810+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:40.216975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.8","version":"42'12","reported_seq":95,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T03:39:01.548129+0000","last_change":"2024-04-26T03:39:01.548129+0000","last_active":"2024-04-26T03:39:01.548129+0000","last_peered":"2024-04-26T03:39:01.548129+0000","last_clean":"2024-04-26T03:39:01.548129+0000","last_became_active":"2024-04-26T03:39:01.546346+0000","last_became_peered":"2024-04-26T03:39:01.546346+0000","last_unstale":"2024-04-26T03:39:01.548129+0000","last_undegraded":"2024-04-26T03:39:01.548129+0000","last_fullsized":"2024-04-26T03:39:01.548129+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.007687518,"stat_sum":{"num_bytes":983040,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":963,"num_write":4,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,10,11,15,5],"acting":[6,0,1,10,11,15,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"3.9","version":"35'6","reported_seq":84,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T03:39:01.543379+0000","last_change":"2024-04-26T03:39:01.542376+0000","last_active":"2024-04-26T03:39:01.543379+0000","last_peered":"2024-04-26T03:39:01.543379+0000","last_clean":"2024-04-26T03:39:01.543379+0000","last_became_active":"2024-04-26T03:39:01.541605+0000","last_became_peered":"2024-04-26T03:39:01.541605+0000","last_unstale":"2024-04-26T03:39:01.543379+0000","last_undegraded":"2024-04-26T03:39:01.543379+0000","last_fullsized":"2024-04-26T03:39:01.543379+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001319764,"stat_sum":{"num_bytes":614400,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":626,"num_write":3,"num_write_kb":600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2,10,9,15,8,4],"acting":[13,2,10,9,15,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"3.a","version":"42'26","reported_seq":95,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-26T03:38:58.539559+0000","last_change":"2024-04-26T03:38:58.530868+0000","last_active":"2024-04-26T03:38:58.539559+0000","last_peered":"2024-04-26T03:38:58.539559+0000","last_clean":"2024-04-26T03:38:58.539559+0000","last_became_active":"2024-04-26T03:38:49.486365+0000","last_became_peered":"2024-04-26T03:38:49.486365+0000","last_unstale":"2024-04-26T03:38:58.539559+0000","last_undegraded":"2024-04-26T03:38:58.539559+0000","last_fullsized":"2024-04-26T03:38:58.539559+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:15.024963+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0070227960000000004,"stat_sum":{"num_bytes":4423680,"num_objects":7,"num_object_clones":1,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":31,"num_read_kb":6156,"num_write":20,"num_write_kb":4880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,10,13,1,11],"acting":[3,7,6,10,13,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"3.b","version":"45'34","reported_seq":110,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T03:39:02.539161+0000","last_change":"2024-04-26T03:38:58.531405+0000","last_active":"2024-04-26T03:39:02.539161+0000","last_peered":"2024-04-26T03:39:02.539161+0000","last_clean":"2024-04-26T03:39:02.539161+0000","last_became_active":"2024-04-26T03:38:49.940561+0000","last_became_peered":"2024-04-26T03:38:49.940561+0000","last_unstale":"2024-04-26T03:39:02.539161+0000","last_undegraded":"2024-04-26T03:39:02.539161+0000","last_fullsized":"2024-04-26T03:39:02.539161+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:17.641819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0079553890000000002,"stat_sum":{"num_bytes":4374528,"num_objects":7,"num_object_clones":3,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":23,"num_read_kb":4331,"num_write":66,"num_write_kb":5584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,10,12,1,7,14],"acting":[4,0,10,12,1,7,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"3.c","version":"42'19","reported_seq":78,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T03:39:00.531241+0000","last_change":"2024-04-26T03:38:58.524651+0000","last_active":"2024-04-26T03:39:00.531241+0000","last_peered":"2024-04-26T03:39:00.531241+0000","last_clean":"2024-04-26T03:39:00.531241+0000","last_became_active":"2024-04-26T03:38:49.941017+0000","last_became_peered":"2024-04-26T03:38:49.941017+0000","last_unstale":"2024-04-26T03:39:00.531241+0000","last_undegraded":"2024-04-26T03:39:00.531241+0000","last_fullsized":"2024-04-26T03:39:00.531241+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:51.044689+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0080080389999999998,"stat_sum":{"num_bytes":3145728,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":29,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,0,1,9,12,14],"acting":[11,8,0,1,9,12,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"3.d","version":"42'9","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T03:39:01.535059+0000","last_change":"2024-04-26T03:38:58.517330+0000","last_active":"2024-04-26T03:39:01.535059+0000","last_peered":"2024-04-26T03:39:01.535059+0000","last_clean":"2024-04-26T03:39:01.535059+0000","last_became_active":"2024-04-26T03:38:49.511783+0000","last_became_peered":"2024-04-26T03:38:49.511783+0000","last_unstale":"2024-04-26T03:39:01.535059+0000","last_undegraded":"2024-04-26T03:39:01.535059+0000","last_fullsized":"2024-04-26T03:39:01.535059+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:10.459823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050621300000000004,"stat_sum":{"num_bytes":606208,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":7,"num_read_kb":3,"num_write":27,"num_write_kb":880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,9,1,5,8,4],"acting":[7,3,9,1,5,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"3.e","version":"45'24","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T03:39:01.648420+0000","last_change":"2024-04-26T03:38:58.522954+0000","last_active":"2024-04-26T03:39:01.648420+0000","last_peered":"2024-04-26T03:39:01.648420+0000","last_clean":"2024-04-26T03:39:01.648420+0000","last_became_active":"2024-04-26T03:38:49.941260+0000","last_became_peered":"2024-04-26T03:38:49.941260+0000","last_unstale":"2024-04-26T03:39:01.648420+0000","last_undegraded":"2024-04-26T03:39:01.648420+0000","last_fullsized":"2024-04-26T03:39:01.648420+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:50.541006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064987400000000005,"stat_sum":{"num_bytes":6356992,"num_objects":6,"num_object_clones":2,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":4859,"num_write":22,"num_write_kb":6208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,14,6,11,8,9,12],"acting":[13,14,6,11,8,9,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"3.f","version":"42'15","reported_seq":81,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T03:39:00.533293+0000","last_change":"2024-04-26T03:38:58.524808+0000","last_active":"2024-04-26T03:39:00.533293+0000","last_peered":"2024-04-26T03:39:00.533293+0000","last_clean":"2024-04-26T03:39:00.533293+0000","last_became_active":"2024-04-26T03:38:49.486850+0000","last_became_peered":"2024-04-26T03:38:49.486850+0000","last_unstale":"2024-04-26T03:39:00.533293+0000","last_undegraded":"2024-04-26T03:39:00.533293+0000","last_fullsized":"2024-04-26T03:39:00.533293+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:15.834358+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0079217420000000007,"stat_sum":{"num_bytes":2490368,"num_objects":3,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":1829,"num_write":29,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,13,5,14,2,1,0],"acting":[11,13,5,14,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"3.10","version":"35'9","reported_seq":90,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T03:39:02.621317+0000","last_change":"2024-04-26T03:39:01.551412+0000","last_active":"2024-04-26T03:39:02.621317+0000","last_peered":"2024-04-26T03:39:02.621317+0000","last_clean":"2024-04-26T03:39:02.621317+0000","last_became_active":"2024-04-26T03:39:01.550673+0000","last_became_peered":"2024-04-26T03:39:01.550673+0000","last_unstale":"2024-04-26T03:39:02.621317+0000","last_undegraded":"2024-04-26T03:39:02.621317+0000","last_fullsized":"2024-04-26T03:39:02.621317+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:08.145863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":507904,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":2979,"num_write":5,"num_write_kb":648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,9,15,2,6,14,4],"acting":[1,9,15,2,6,14,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"3.11","version":"42'8","reported_seq":83,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T03:39:02.538925+0000","last_change":"2024-04-26T03:39:01.558614+0000","last_active":"2024-04-26T03:39:02.538925+0000","last_peered":"2024-04-26T03:39:02.538925+0000","last_clean":"2024-04-26T03:39:02.538925+0000","last_became_active":"2024-04-26T03:39:01.557340+0000","last_became_peered":"2024-04-26T03:39:01.557340+0000","last_unstale":"2024-04-26T03:39:02.538925+0000","last_undegraded":"2024-04-26T03:39:02.538925+0000","last_fullsized":"2024-04-26T03:39:02.538925+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1310720,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":2,"num_write_kb":1608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,14,10,9,1,2],"acting":[4,11,14,10,9,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"3.12","version":"35'11","reported_seq":86,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T03:39:01.549730+0000","last_change":"2024-04-26T03:39:01.547252+0000","last_active":"2024-04-26T03:39:01.549730+0000","last_peered":"2024-04-26T03:39:01.549730+0000","last_clean":"2024-04-26T03:39:01.549730+0000","last_became_active":"2024-04-26T03:39:01.545694+0000","last_became_peered":"2024-04-26T03:39:01.545694+0000","last_unstale":"2024-04-26T03:39:01.549730+0000","last_undegraded":"2024-04-26T03:39:01.549730+0000","last_fullsized":"2024-04-26T03:39:01.549730+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":548864,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":6,"num_write_kb":1136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,5,3,9,15,10,11],"acting":[14,5,3,9,15,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"3.13","version":"43'19","reported_seq":103,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T03:39:01.654109+0000","last_change":"2024-04-26T03:39:01.562676+0000","last_active":"2024-04-26T03:39:01.654109+0000","last_peered":"2024-04-26T03:39:01.654109+0000","last_clean":"2024-04-26T03:39:01.654109+0000","last_became_active":"2024-04-26T03:39:01.561931+0000","last_became_peered":"2024-04-26T03:39:01.561931+0000","last_unstale":"2024-04-26T03:39:01.654109+0000","last_undegraded":"2024-04-26T03:39:01.654109+0000","last_fullsized":"2024-04-26T03:39:01.654109+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2400256,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":11997,"num_write":9,"num_write_kb":2344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,7,4,15,1,2,3],"acting":[10,7,4,15,1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"2","length":"3"}]}],"pool_stats":[{"poolid":3,"num_pg":26,"stat_sum":{"num_bytes":39829504,"num_objects":58,"num_object_clones":14,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":2,"num_read":203,"num_read_kb":39991,"num_write":308,"num_write_kb":44064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":76910592,"data_stored":93188096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":370,"ondisk_log_size":370,"up":182,"acting":182,"num_store_stats":16},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":13852,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":24,"seq":103079215114,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34200,"kb_used_data":3012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337640,"statfs":{"total":96636764160,"available":96601743360,"internally_reserved":0,"allocated":3084288,"data_stored":3472392,"data_compressed":44187,"data_compressed_allocated":110592,"data_compressed_original":667648,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[4,6,7,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215114,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29812,"kb_used_data":2792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342028,"statfs":{"total":96636764160,"available":96606236672,"internally_reserved":0,"allocated":2859008,"data_stored":2771432,"data_compressed":37261,"data_compressed_allocated":77824,"data_compressed_original":155648,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":23,"seq":98784247820,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36284,"kb_used_data":5100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335556,"statfs":{"total":96636764160,"available":96599609344,"internally_reserved":0,"allocated":5222400,"data_stored":5841203,"data_compressed":43300,"data_compressed_allocated":90112,"data_compressed_original":180224,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280521,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":4980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":5099520,"data_stored":6280019,"data_compressed":50226,"data_compressed_allocated":122880,"data_compressed_original":692224,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,5,7,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280521,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":4348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":4452352,"data_stored":4934188,"data_compressed":41281,"data_compressed_allocated":86016,"data_compressed_original":172032,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,3,4,5,6,8,9,10,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":22,"seq":94489280522,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30952,"kb_used_data":3924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340888,"statfs":{"total":96636764160,"available":96605069312,"internally_reserved":0,"allocated":4018176,"data_stored":4024876,"data_compressed":41281,"data_compressed_allocated":86016,"data_compressed_original":172032,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[2,4,5,6,7,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280520,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31080,"kb_used_data":4052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340760,"statfs":{"total":96636764160,"available":96604938240,"internally_reserved":0,"allocated":4149248,"data_stored":4155948,"data_compressed":41281,"data_compressed_allocated":86016,"data_compressed_original":172032,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,2,4,7,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280521,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33064,"kb_used_data":6036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338776,"statfs":{"total":96636764160,"available":96602906624,"internally_reserved":0,"allocated":6180864,"data_stored":6605356,"data_compressed":41281,"data_compressed_allocated":86016,"data_compressed_original":172032,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[1,2,3,4,7,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313226,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32784,"kb_used_data":5756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339056,"statfs":{"total":96636764160,"available":96603193344,"internally_reserved":0,"allocated":5894144,"data_stored":5843500,"data_compressed":41281,"data_compressed_allocated":86016,"data_compressed_original":172032,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,5,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":12,"up_from":22,"seq":94489280520,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30416,"kb_used_data":3388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341424,"statfs":{"total":96636764160,"available":96605618176,"internally_reserved":0,"allocated":3469312,"data_stored":3578412,"data_compressed":41281,"data_compressed_allocated":86016,"data_compressed_original":172032,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,7,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280520,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32240,"kb_used_data":5220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339600,"statfs":{"total":96636764160,"available":96603750400,"internally_reserved":0,"allocated":5345280,"data_stored":5837107,"data_compressed":43300,"data_compressed_allocated":90112,"data_compressed_original":180224,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[2,4,6,7,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247818,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36144,"kb_used_data":4956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335696,"statfs":{"total":96636764160,"available":96599752704,"internally_reserved":0,"allocated":5074944,"data_stored":7305553,"data_compressed":45319,"data_compressed_allocated":94208,"data_compressed_original":188416,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,6,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280520,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32212,"kb_used_data":5184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339628,"statfs":{"total":96636764160,"available":96603779072,"internally_reserved":0,"allocated":5308416,"data_stored":5450284,"data_compressed":41281,"data_compressed_allocated":86016,"data_compressed_original":172032,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,3,5,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":23,"seq":98784247820,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39924,"kb_used_data":8736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331916,"statfs":{"total":96636764160,"available":96595881984,"internally_reserved":0,"allocated":8945664,"data_stored":11536721,"data_compressed":45319,"data_compressed_allocated":94208,"data_compressed_original":188416,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,4,6,7,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280520,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30440,"kb_used_data":3420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341400,"statfs":{"total":96636764160,"available":96605593600,"internally_reserved":0,"allocated":3502080,"data_stored":4161843,"data_compressed":43300,"data_compressed_allocated":90112,"data_compressed_original":180224,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,4,5,7,8,9,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247818,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33036,"kb_used_data":6008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338804,"statfs":{"total":96636764160,"available":96602935296,"internally_reserved":0,"allocated":6152192,"data_stored":7117137,"data_compressed":45319,"data_compressed_allocated":94208,"data_compressed_original":188416,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4587520,"data_stored":7004160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8458240,"data_stored":11235328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5664768,"data_stored":6815744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2420736,"data_stored":2510848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5795840,"data_stored":7249920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2592768,"data_stored":2682880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4571136,"data_stored":5459968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4325376,"data_stored":4988928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4210688,"data_stored":5201920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4476928,"data_stored":5468160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":6189056,"data_stored":7258112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":5431296,"data_stored":5562368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":3997696,"data_stored":4984832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":4870144,"data_stored":5545984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":7348224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":3026944,"data_stored":3870720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T03:39:03.875 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set unique_pool_0 pgp_num 26 2024-04-26T03:39:03.910 INFO:tasks.ceph.osd.15.smithi092.stderr:2024-04-26T03:39:03.909+0000 7f6d12dec640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 31824) UID: 0 2024-04-26T03:39:04.001 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:04.002 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:04.010 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-04-26T03:39:04.016 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:39:04.014+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:39:04.021 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-26T03:39:04.116 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:39:04.115+0000 7f60d3c4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31757) UID: 0 2024-04-26T03:39:04.122 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:04.122 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:04.131 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-04-26T03:39:04.147 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0: {} 2024-04-26T03:39:04.147 INFO:teuthology.orchestra.run.smithi028.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T03:39:04.166 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1: {} 2024-04-26T03:39:04.167 INFO:teuthology.orchestra.run.smithi028.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T03:39:04.184 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2: {} 2024-04-26T03:39:04.184 INFO:teuthology.orchestra.run.smithi028.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T03:39:04.202 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3: {} 2024-04-26T03:39:04.203 INFO:teuthology.orchestra.run.smithi028.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T03:39:04.217 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-26T03:39:04.216+0000 7f610b3a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31816) UID: 0 2024-04-26T03:39:04.219 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4: {} 2024-04-26T03:39:04.219 INFO:teuthology.orchestra.run.smithi028.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T03:39:04.237 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5: {} 2024-04-26T03:39:04.237 INFO:teuthology.orchestra.run.smithi028.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T03:39:04.254 INFO:teuthology.orchestra.run.smithi028.stdout:osd.6: {} 2024-04-26T03:39:04.255 INFO:teuthology.orchestra.run.smithi028.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T03:39:04.273 INFO:teuthology.orchestra.run.smithi028.stdout:osd.7: {} 2024-04-26T03:39:04.274 INFO:teuthology.orchestra.run.smithi028.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T03:39:04.278 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:04.278 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:04.287 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-04-26T03:39:04.289 INFO:teuthology.orchestra.run.smithi028.stdout:osd.8: {} 2024-04-26T03:39:04.289 INFO:teuthology.orchestra.run.smithi028.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T03:39:04.307 INFO:teuthology.orchestra.run.smithi028.stdout:osd.9: {} 2024-04-26T03:39:04.308 INFO:teuthology.orchestra.run.smithi028.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T03:39:04.318 INFO:tasks.ceph.osd.4.smithi028.stderr:2024-04-26T03:39:04.316+0000 7fd9511b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31963) UID: 0 2024-04-26T03:39:04.325 INFO:teuthology.orchestra.run.smithi028.stdout:osd.10: {} 2024-04-26T03:39:04.325 INFO:teuthology.orchestra.run.smithi028.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T03:39:04.343 INFO:teuthology.orchestra.run.smithi028.stdout:osd.11: {} 2024-04-26T03:39:04.343 INFO:teuthology.orchestra.run.smithi028.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T03:39:04.359 INFO:teuthology.orchestra.run.smithi028.stdout:osd.12: {} 2024-04-26T03:39:04.360 INFO:teuthology.orchestra.run.smithi028.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T03:39:04.377 INFO:teuthology.orchestra.run.smithi028.stdout:osd.13: {} 2024-04-26T03:39:04.378 INFO:teuthology.orchestra.run.smithi028.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T03:39:04.393 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:04.394 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:04.395 INFO:teuthology.orchestra.run.smithi028.stdout:osd.14: {} 2024-04-26T03:39:04.395 INFO:teuthology.orchestra.run.smithi028.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T03:39:04.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:39:04.393+0000 7fdeaca29640 0 -- 172.21.15.28:0/3421652162 >> [v2:172.21.15.92:6826/3982634323,v1:172.21.15.92:6827/3982634323] conn(0x7fdea8222b20 msgr2=0x7fdea8224f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-26T03:39:04.403 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-26T03:39:04.419 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:39:04.418+0000 7fb78c544640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31956) UID: 0 2024-04-26T03:39:04.515 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:04.515 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-26T03:39:04.515 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-26T03:39:04.515 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:04.519 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:39:04.518+0000 7f60d3c4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31757) UID: 0 2024-04-26T03:39:04.524 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-26T03:39:04.549 INFO:teuthology.orchestra.run.smithi028.stderr:set pool 3 pgp_num to 26 2024-04-26T03:39:04.553 INFO:tasks.rados.rados.0.smithi092.stdout:166: done (2 left) 2024-04-26T03:39:04.553 INFO:tasks.rados.rados.0.smithi092.stdout:167: snap_remove snap 3 2024-04-26T03:39:04.555 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-26T03:39:04.613 INFO:teuthology.orchestra.run.smithi028.stdout:osd.15: {} 2024-04-26T03:39:04.613 INFO:teuthology.orchestra.run.smithi028.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T03:39:04.621 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-04-26T03:39:04.620+0000 7fee562fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31759) UID: 0 2024-04-26T03:39:04.636 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:04.636 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-26T03:39:04.637 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-26T03:39:04.637 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-26T03:39:04.637 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:04.646 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-26T03:39:04.685 INFO:tasks.rados.rados.0.smithi092.stdout:167: done (2 left) 2024-04-26T03:39:04.686 INFO:tasks.rados.rados.0.smithi092.stdout:168: rollback oid 49 current snap is 7 2024-04-26T03:39:04.686 INFO:tasks.rados.rados.0.smithi092.stdout:rollback oid 49 to 6 2024-04-26T03:39:04.686 INFO:tasks.rados.rados.0.smithi092.stdout:162: finishing write tid 2 to smithi09235267-48 2024-04-26T03:39:04.686 INFO:tasks.rados.rados.0.smithi092.stdout:162: finishing write tid 3 to smithi09235267-48 2024-04-26T03:39:04.686 INFO:tasks.rados.rados.0.smithi092.stdout:162: finishing write tid 4 to smithi09235267-48 2024-04-26T03:39:04.686 INFO:tasks.rados.rados.0.smithi092.stdout:162: finishing write tid 5 to smithi09235267-48 2024-04-26T03:39:04.686 INFO:tasks.rados.rados.0.smithi092.stdout:162: oid 48 updating version 0 to 23 2024-04-26T03:39:04.686 INFO:tasks.rados.rados.0.smithi092.stdout:162: oid 48 updating version 23 to 24 2024-04-26T03:39:04.686 INFO:tasks.rados.rados.0.smithi092.stdout:162: oid 48 version 24 is already newer than 22 2024-04-26T03:39:04.686 INFO:tasks.rados.rados.0.smithi092.stdout:162: oid 48 version 24 is already newer than 21 2024-04-26T03:39:04.687 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 48 v 24 (ObjNum 84 snap 7 seq_num 84) dirty exists 2024-04-26T03:39:04.687 INFO:tasks.rados.rados.0.smithi092.stdout:162: left oid 48 (ObjNum 84 snap 7 seq_num 84) 2024-04-26T03:39:04.687 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 21 v 36 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-04-26T03:39:04.687 INFO:tasks.rados.rados.0.smithi092.stdout:162: done (2 left) 2024-04-26T03:39:04.687 INFO:tasks.rados.rados.0.smithi092.stdout:164: done (1 left) 2024-04-26T03:39:04.687 INFO:tasks.rados.rados.0.smithi092.stdout:169: rmattr oid 45 current snap is 7 2024-04-26T03:39:04.687 INFO:tasks.rados.rados.0.smithi092.stdout:169: done (1 left) 2024-04-26T03:39:04.687 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 1 current snap is 7 2024-04-26T03:39:04.687 INFO:tasks.rados.rados.0.smithi092.stdout:170: seq_num 85 ranges {737280=671744,1409024=770048} 2024-04-26T03:39:04.693 INFO:tasks.rados.rados.0.smithi092.stdout:170: writing smithi09235267-1 from 737280 to 1409024 tid 1 2024-04-26T03:39:04.698 INFO:tasks.rados.rados.0.smithi092.stdout:170: writing smithi09235267-1 from 1409024 to 2179072 tid 2 2024-04-26T03:39:04.698 INFO:tasks.rados.rados.0.smithi092.stdout:171: delete oid 16 current snap is 7 2024-04-26T03:39:04.701 INFO:tasks.rados.rados.0.smithi092.stdout:168: finishing rollback tid 0 to smithi09235267-49 2024-04-26T03:39:04.702 INFO:tasks.rados.rados.0.smithi092.stdout:168: finishing rollback tid 1 to smithi09235267-49 2024-04-26T03:39:04.702 INFO:tasks.rados.rados.0.smithi092.stdout:168: finishing rollback tid 2 to smithi09235267-49 2024-04-26T03:39:04.702 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 49 v 7 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-04-26T03:39:04.711 INFO:tasks.rados.rados.0.smithi092.stdout:168: done (2 left) 2024-04-26T03:39:04.711 INFO:tasks.rados.rados.0.smithi092.stdout:171: done (1 left) 2024-04-26T03:39:04.711 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 28 current snap is 7 2024-04-26T03:39:04.712 INFO:tasks.rados.rados.0.smithi092.stdout:172: seq_num 87 ranges {1556480=704512,2260992=688128,2949120=81920} 2024-04-26T03:39:04.722 INFO:tasks.ceph.osd.4.smithi028.stderr:2024-04-26T03:39:04.720+0000 7fd9511b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31963) UID: 0 2024-04-26T03:39:04.724 INFO:tasks.rados.rados.0.smithi092.stdout:172: writing smithi09235267-28 from 1556480 to 2260992 tid 1 2024-04-26T03:39:04.728 INFO:tasks.rados.rados.0.smithi092.stdout:172: writing smithi09235267-28 from 2260992 to 2949120 tid 2 2024-04-26T03:39:04.729 INFO:tasks.rados.rados.0.smithi092.stdout:172: writing smithi09235267-28 from 2949120 to 3031040 tid 3 2024-04-26T03:39:04.730 INFO:tasks.rados.rados.0.smithi092.stdout:173: copy_from oid 2 from oid 25 current snap is 7 2024-04-26T03:39:04.730 INFO:tasks.rados.rados.0.smithi092.stdout:170: finishing write tid 1 to smithi09235267-1 2024-04-26T03:39:04.730 INFO:tasks.rados.rados.0.smithi092.stdout:174: snap_remove snap 6 2024-04-26T03:39:04.804 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:04.804 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-26T03:39:04.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-26T03:39:04.805 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-26T03:39:04.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.805 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:20 3.as0 3:5f4e6bd0:::smithi09235267-7:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:04.805 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.485002+0000", 2024-04-26T03:39:04.805 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.316677557, 2024-04-26T03:39:04.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017259429, 2024-04-26T03:39:04.805 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:04.805 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:04.805 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:04.805 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:04.805 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:04.805 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:04.805 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 20 2024-04-26T03:39:04.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.806 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:04.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:04.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485002+0000", 2024-04-26T03:39:04.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:04.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:04.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485002+0000", 2024-04-26T03:39:04.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:04.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:04.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485003+0000", 2024-04-26T03:39:04.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6400000000000002e-07 2024-04-26T03:39:04.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:04.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485007+0000", 2024-04-26T03:39:04.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4390000000000002e-06 2024-04-26T03:39:04.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:04.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485015+0000", 2024-04-26T03:39:04.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9389999999999997e-06 2024-04-26T03:39:04.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:04.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485243+0000", 2024-04-26T03:39:04.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000227345 2024-04-26T03:39:04.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:04.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.487060+0000", 2024-04-26T03:39:04.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001817117 2024-04-26T03:39:04.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:04.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.487148+0000", 2024-04-26T03:39:04.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8712e-05 2024-04-26T03:39:04.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:04.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.487611+0000", 2024-04-26T03:39:04.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000462223 2024-04-26T03:39:04.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:04.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.491670+0000", 2024-04-26T03:39:04.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004058842 2024-04-26T03:39:04.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:04.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.502181+0000", 2024-04-26T03:39:04.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010511360000000001 2024-04-26T03:39:04.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:04.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.502262+0000", 2024-04-26T03:39:04.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0888000000000002e-05 2024-04-26T03:39:04.812 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:04.812 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:04.812 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:04.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.813 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:21 3.as0 3:5f4e6bd0:::smithi09235267-7:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:04.813 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.485256+0000", 2024-04-26T03:39:04.813 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.31642392, 2024-04-26T03:39:04.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026302736, 2024-04-26T03:39:04.813 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:04.813 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:04.813 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:04.813 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:04.813 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:04.814 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:04.814 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 21 2024-04-26T03:39:04.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.814 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:04.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:04.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485256+0000", 2024-04-26T03:39:04.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:04.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:04.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485256+0000", 2024-04-26T03:39:04.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:04.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:04.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485257+0000", 2024-04-26T03:39:04.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4000000000000001e-07 2024-04-26T03:39:04.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:04.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485261+0000", 2024-04-26T03:39:04.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2910000000000001e-06 2024-04-26T03:39:04.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:04.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485269+0000", 2024-04-26T03:39:04.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1780000000000003e-06 2024-04-26T03:39:04.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:04.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485275+0000", 2024-04-26T03:39:04.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2419999999999997e-06 2024-04-26T03:39:04.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:04.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.487806+0000", 2024-04-26T03:39:04.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025310369999999999 2024-04-26T03:39:04.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:04.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.488019+0000", 2024-04-26T03:39:04.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021297800000000001 2024-04-26T03:39:04.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:04.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.502612+0000", 2024-04-26T03:39:04.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014592327 2024-04-26T03:39:04.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:04.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.502711+0000", 2024-04-26T03:39:04.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9432999999999995e-05 2024-04-26T03:39:04.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:04.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.511559+0000", 2024-04-26T03:39:04.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0088476100000000005 2024-04-26T03:39:04.823 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:04.823 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:04.824 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:04.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.824 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:75 3.as0 3:50cd80b4:::smithi09235267-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:04.824 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.524874+0000", 2024-04-26T03:39:04.824 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.276806078, 2024-04-26T03:39:04.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017219323000000002, 2024-04-26T03:39:04.825 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:04.825 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:04.825 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:04.825 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:04.825 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:04.826 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:04.826 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 75 2024-04-26T03:39:04.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.826 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:04.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:04.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.524874+0000", 2024-04-26T03:39:04.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:04.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:04.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.524874+0000", 2024-04-26T03:39:04.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:04.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:04.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.524874+0000", 2024-04-26T03:39:04.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9200000000000001e-07 2024-04-26T03:39:04.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:04.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.524877+0000", 2024-04-26T03:39:04.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6079999999999998e-06 2024-04-26T03:39:04.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:04.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.524883+0000", 2024-04-26T03:39:04.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0039999999999996e-06 2024-04-26T03:39:04.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:04.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.524888+0000", 2024-04-26T03:39:04.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3070000000000002e-06 2024-04-26T03:39:04.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:04.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.530308+0000", 2024-04-26T03:39:04.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054201850000000001 2024-04-26T03:39:04.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:04.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.530412+0000", 2024-04-26T03:39:04.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010306699999999999 2024-04-26T03:39:04.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:04.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.534668+0000", 2024-04-26T03:39:04.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042560949999999997 2024-04-26T03:39:04.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:04.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.534740+0000", 2024-04-26T03:39:04.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2082000000000005e-05 2024-04-26T03:39:04.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:04.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.542093+0000", 2024-04-26T03:39:04.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0073534830000000001 2024-04-26T03:39:04.836 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:04.836 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:04.836 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:04.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.837 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:103 3.as0 3:527f9dfd:::smithi09235267-35:head [append 0~573440,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:04.837 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.562322+0000", 2024-04-26T03:39:04.837 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.239357904, 2024-04-26T03:39:04.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012574926, 2024-04-26T03:39:04.837 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:04.837 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:04.837 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:04.838 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:04.838 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:04.838 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:04.838 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 103 2024-04-26T03:39:04.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.839 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:04.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:04.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.562322+0000", 2024-04-26T03:39:04.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:04.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:04.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.562322+0000", 2024-04-26T03:39:04.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:04.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:04.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.562323+0000", 2024-04-26T03:39:04.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4410000000000001e-06 2024-04-26T03:39:04.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:04.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.562565+0000", 2024-04-26T03:39:04.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024172899999999999 2024-04-26T03:39:04.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:04.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.562580+0000", 2024-04-26T03:39:04.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4844000000000001e-05 2024-04-26T03:39:04.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:04.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.562586+0000", 2024-04-26T03:39:04.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9109999999999998e-06 2024-04-26T03:39:04.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:04.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.562639+0000", 2024-04-26T03:39:04.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2698999999999998e-05 2024-04-26T03:39:04.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:04.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.562748+0000", 2024-04-26T03:39:04.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000109712 2024-04-26T03:39:04.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:04.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.563347+0000", 2024-04-26T03:39:04.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00059887500000000001 2024-04-26T03:39:04.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:04.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.568019+0000", 2024-04-26T03:39:04.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046716589999999999 2024-04-26T03:39:04.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:04.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.574819+0000", 2024-04-26T03:39:04.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0068006059999999998 2024-04-26T03:39:04.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:04.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.574897+0000", 2024-04-26T03:39:04.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7449999999999999e-05 2024-04-26T03:39:04.844 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:04.844 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:04.844 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:04.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.844 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:104 3.as0 3:527f9dfd:::smithi09235267-35:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:04.845 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.562602+0000", 2024-04-26T03:39:04.845 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.239078063999999, 2024-04-26T03:39:04.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012485850999999999, 2024-04-26T03:39:04.845 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:04.845 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:04.845 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:04.845 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:04.845 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:04.845 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:04.845 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104 2024-04-26T03:39:04.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.845 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:04.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:04.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.562602+0000", 2024-04-26T03:39:04.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:04.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:04.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.562602+0000", 2024-04-26T03:39:04.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:04.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:04.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.562602+0000", 2024-04-26T03:39:04.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5400000000000001e-07 2024-04-26T03:39:04.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:04.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.562607+0000", 2024-04-26T03:39:04.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1740000000000002e-06 2024-04-26T03:39:04.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:04.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.562613+0000", 2024-04-26T03:39:04.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7240000000000003e-06 2024-04-26T03:39:04.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:04.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.562618+0000", 2024-04-26T03:39:04.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0590000000000002e-06 2024-04-26T03:39:04.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:04.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.567217+0000", 2024-04-26T03:39:04.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045985720000000004 2024-04-26T03:39:04.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:04.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.567333+0000", 2024-04-26T03:39:04.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011642999999999999 2024-04-26T03:39:04.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:04.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.567811+0000", 2024-04-26T03:39:04.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00047731999999999999 2024-04-26T03:39:04.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:04.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.568257+0000", 2024-04-26T03:39:04.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00044601100000000001 2024-04-26T03:39:04.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:04.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.575009+0000", 2024-04-26T03:39:04.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006752381 2024-04-26T03:39:04.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:04.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.575088+0000", 2024-04-26T03:39:04.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8626000000000002e-05 2024-04-26T03:39:04.850 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:04.850 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:04.850 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:04.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.850 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:105 3.as0 3:527f9dfd:::smithi09235267-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:04.851 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.562626+0000", 2024-04-26T03:39:04.851 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.239054239, 2024-04-26T03:39:04.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.023769061000000001, 2024-04-26T03:39:04.851 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:04.851 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:04.851 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:04.851 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:04.851 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:04.851 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:04.851 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 105 2024-04-26T03:39:04.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.851 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:04.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:04.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.562626+0000", 2024-04-26T03:39:04.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:04.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:04.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.562626+0000", 2024-04-26T03:39:04.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:04.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:04.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.562626+0000", 2024-04-26T03:39:04.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7800000000000002e-07 2024-04-26T03:39:04.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:04.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.562629+0000", 2024-04-26T03:39:04.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7779999999999999e-06 2024-04-26T03:39:04.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:04.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.562636+0000", 2024-04-26T03:39:04.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3520000000000001e-06 2024-04-26T03:39:04.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:04.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.562641+0000", 2024-04-26T03:39:04.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4789999999999998e-06 2024-04-26T03:39:04.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:04.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.568071+0000", 2024-04-26T03:39:04.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054305719999999998 2024-04-26T03:39:04.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:04.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.568158+0000", 2024-04-26T03:39:04.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6726999999999998e-05 2024-04-26T03:39:04.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:04.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.575494+0000", 2024-04-26T03:39:04.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0073362180000000003 2024-04-26T03:39:04.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:04.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.575571+0000", 2024-04-26T03:39:04.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7134999999999999e-05 2024-04-26T03:39:04.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:04.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.586395+0000", 2024-04-26T03:39:04.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010823322 2024-04-26T03:39:04.856 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:04.856 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:04.856 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:04.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.856 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:117 3.as0 3:58923ed4:::smithi09235267-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:04.856 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.578829+0000", 2024-04-26T03:39:04.856 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.222851121, 2024-04-26T03:39:04.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017648347000000002, 2024-04-26T03:39:04.856 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:04.856 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:04.856 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:04.856 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:04.857 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:04.857 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:04.857 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 117 2024-04-26T03:39:04.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.857 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:04.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:04.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.578829+0000", 2024-04-26T03:39:04.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:04.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:04.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.578829+0000", 2024-04-26T03:39:04.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:04.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:04.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.578829+0000", 2024-04-26T03:39:04.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4900000000000001e-07 2024-04-26T03:39:04.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:04.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.578831+0000", 2024-04-26T03:39:04.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0700000000000001e-06 2024-04-26T03:39:04.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:04.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.578838+0000", 2024-04-26T03:39:04.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2689999999999996e-06 2024-04-26T03:39:04.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:04.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.578843+0000", 2024-04-26T03:39:04.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5300000000000004e-06 2024-04-26T03:39:04.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:04.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.586870+0000", 2024-04-26T03:39:04.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0080264919999999997 2024-04-26T03:39:04.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:04.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.586955+0000", 2024-04-26T03:39:04.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5753000000000004e-05 2024-04-26T03:39:04.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:04.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.589337+0000", 2024-04-26T03:39:04.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023813010000000002 2024-04-26T03:39:04.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:04.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.589419+0000", 2024-04-26T03:39:04.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2078999999999998e-05 2024-04-26T03:39:04.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:04.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.596477+0000", 2024-04-26T03:39:04.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0070584039999999999 2024-04-26T03:39:04.861 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:04.861 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:04.861 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:04.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.862 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.2s2 35/32 ECSubWrite(tid=8, reqid=client.4556.0:139, at_version=35'5, trim_to=0'0, roll_forward_to=35'4))", 2024-04-26T03:39:04.862 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.620795+0000", 2024-04-26T03:39:04.862 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.180885341, 2024-04-26T03:39:04.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013838492000000001, 2024-04-26T03:39:04.862 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:04.862 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:04.862 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:04.862 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:04.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:04.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.620795+0000", 2024-04-26T03:39:04.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:04.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:04.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.620795+0000", 2024-04-26T03:39:04.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:04.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:04.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.620796+0000", 2024-04-26T03:39:04.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.192e-06 2024-04-26T03:39:04.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:04.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.621134+0000", 2024-04-26T03:39:04.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00033806999999999999 2024-04-26T03:39:04.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:04.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.621152+0000", 2024-04-26T03:39:04.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8539e-05 2024-04-26T03:39:04.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:04.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.621164+0000", 2024-04-26T03:39:04.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1987999999999999e-05 2024-04-26T03:39:04.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:04.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.621244+0000", 2024-04-26T03:39:04.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0065e-05 2024-04-26T03:39:04.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:04.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.621255+0000", 2024-04-26T03:39:04.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0196e-05 2024-04-26T03:39:04.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:04.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.634596+0000", 2024-04-26T03:39:04.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013341362000000001 2024-04-26T03:39:04.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:04.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.634633+0000", 2024-04-26T03:39:04.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7079999999999997e-05 2024-04-26T03:39:04.867 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:04.867 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:04.867 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:04.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.868 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:150 3.as0 3:5930de0f:::smithi09235267-50:head [append 0~458752,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:04.868 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.634461+0000", 2024-04-26T03:39:04.868 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.167218632000001, 2024-04-26T03:39:04.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017493549000000001, 2024-04-26T03:39:04.869 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:04.869 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:04.869 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:04.869 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:04.869 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:04.869 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:04.870 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 150 2024-04-26T03:39:04.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.870 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:04.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:04.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.634461+0000", 2024-04-26T03:39:04.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:04.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:04.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.634461+0000", 2024-04-26T03:39:04.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:04.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:04.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.634463+0000", 2024-04-26T03:39:04.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.618e-06 2024-04-26T03:39:04.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:04.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.634611+0000", 2024-04-26T03:39:04.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000148214 2024-04-26T03:39:04.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:04.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.634628+0000", 2024-04-26T03:39:04.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7105999999999999e-05 2024-04-26T03:39:04.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:04.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.634636+0000", 2024-04-26T03:39:04.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7829999999999999e-06 2024-04-26T03:39:04.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:04.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.634700+0000", 2024-04-26T03:39:04.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4286000000000001e-05 2024-04-26T03:39:04.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:04.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.634839+0000", 2024-04-26T03:39:04.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013907200000000001 2024-04-26T03:39:04.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:04.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.635607+0000", 2024-04-26T03:39:04.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00076726300000000002 2024-04-26T03:39:04.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:04.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.647963+0000", 2024-04-26T03:39:04.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012356525 2024-04-26T03:39:04.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:04.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.651885+0000", 2024-04-26T03:39:04.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039220180000000002 2024-04-26T03:39:04.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:04.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.651955+0000", 2024-04-26T03:39:04.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9664000000000003e-05 2024-04-26T03:39:04.879 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:04.879 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:04.879 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:04.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.880 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:151 3.as0 3:5930de0f:::smithi09235267-50:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:04.880 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.637279+0000", 2024-04-26T03:39:04.880 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.16440079, 2024-04-26T03:39:04.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016253165999999999, 2024-04-26T03:39:04.880 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:04.881 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:04.881 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:04.881 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:04.881 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:04.881 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:04.881 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 151 2024-04-26T03:39:04.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.881 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:04.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:04.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637279+0000", 2024-04-26T03:39:04.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:04.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:04.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637279+0000", 2024-04-26T03:39:04.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:04.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:04.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637281+0000", 2024-04-26T03:39:04.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5319999999999999e-06 2024-04-26T03:39:04.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:04.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637373+0000", 2024-04-26T03:39:04.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2198000000000001e-05 2024-04-26T03:39:04.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:04.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637384+0000", 2024-04-26T03:39:04.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1198e-05 2024-04-26T03:39:04.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:04.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637393+0000", 2024-04-26T03:39:04.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0280000000000006e-06 2024-04-26T03:39:04.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:04.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.639806+0000", 2024-04-26T03:39:04.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024132929999999999 2024-04-26T03:39:04.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:04.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.639879+0000", 2024-04-26T03:39:04.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2470000000000002e-05 2024-04-26T03:39:04.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:04.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.640572+0000", 2024-04-26T03:39:04.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00069311299999999998 2024-04-26T03:39:04.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:04.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.647964+0000", 2024-04-26T03:39:04.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0073922019999999996 2024-04-26T03:39:04.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:04.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.653490+0000", 2024-04-26T03:39:04.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055261850000000003 2024-04-26T03:39:04.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:04.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.653532+0000", 2024-04-26T03:39:04.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1946999999999997e-05 2024-04-26T03:39:04.885 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:04.885 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:04.885 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:04.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.886 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:152 3.as0 3:5930de0f:::smithi09235267-50:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:04.886 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.637409+0000", 2024-04-26T03:39:04.886 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.164271346, 2024-04-26T03:39:04.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016318039999999999, 2024-04-26T03:39:04.886 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:04.886 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:04.886 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:04.886 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:04.886 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:04.886 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:04.886 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 152 2024-04-26T03:39:04.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.886 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:04.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:04.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637409+0000", 2024-04-26T03:39:04.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:04.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:04.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637409+0000", 2024-04-26T03:39:04.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:04.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:04.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637409+0000", 2024-04-26T03:39:04.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5499999999999998e-07 2024-04-26T03:39:04.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:04.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637412+0000", 2024-04-26T03:39:04.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2959999999999999e-06 2024-04-26T03:39:04.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:04.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637419+0000", 2024-04-26T03:39:04.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4670000000000001e-06 2024-04-26T03:39:04.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:04.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637424+0000", 2024-04-26T03:39:04.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8720000000000001e-06 2024-04-26T03:39:04.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:04.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.643035+0000", 2024-04-26T03:39:04.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056115899999999996 2024-04-26T03:39:04.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:04.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.643126+0000", 2024-04-26T03:39:04.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0525999999999994e-05 2024-04-26T03:39:04.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:04.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.643545+0000", 2024-04-26T03:39:04.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00041870200000000002 2024-04-26T03:39:04.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:04.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.647965+0000", 2024-04-26T03:39:04.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0044201730000000003 2024-04-26T03:39:04.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:04.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.653647+0000", 2024-04-26T03:39:04.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056822670000000004 2024-04-26T03:39:04.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:04.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.653727+0000", 2024-04-26T03:39:04.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9592000000000006e-05 2024-04-26T03:39:04.891 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:04.891 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:04.891 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:04.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.891 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:153 3.as0 3:5930de0f:::smithi09235267-50:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:04.891 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.637431+0000", 2024-04-26T03:39:04.891 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.164249071, 2024-04-26T03:39:04.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019190048000000001, 2024-04-26T03:39:04.891 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:04.891 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:04.891 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:04.892 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:04.892 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:04.892 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:04.892 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 153 2024-04-26T03:39:04.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.892 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:04.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:04.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637431+0000", 2024-04-26T03:39:04.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:04.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:04.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637431+0000", 2024-04-26T03:39:04.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:04.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:04.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637431+0000", 2024-04-26T03:39:04.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2024-04-26T03:39:04.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:04.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637434+0000", 2024-04-26T03:39:04.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7690000000000001e-06 2024-04-26T03:39:04.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:04.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637439+0000", 2024-04-26T03:39:04.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4190000000000001e-06 2024-04-26T03:39:04.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:04.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637444+0000", 2024-04-26T03:39:04.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6800000000000001e-06 2024-04-26T03:39:04.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:04.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.643745+0000", 2024-04-26T03:39:04.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0063006570000000003 2024-04-26T03:39:04.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:04.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.643818+0000", 2024-04-26T03:39:04.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3558000000000003e-05 2024-04-26T03:39:04.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:04.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.654106+0000", 2024-04-26T03:39:04.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010287436 2024-04-26T03:39:04.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:04.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.654189+0000", 2024-04-26T03:39:04.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2719999999999994e-05 2024-04-26T03:39:04.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:04.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.656621+0000", 2024-04-26T03:39:04.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024323690000000002 2024-04-26T03:39:04.896 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:04.896 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:04.896 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:04.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.896 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:159 3.as0 3:50cd80b4:::smithi09235267-25:head [read 0~409600,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:04.896 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.637451+0000", 2024-04-26T03:39:04.896 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.164228723000001, 2024-04-26T03:39:04.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015016775, 2024-04-26T03:39:04.896 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:04.896 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:04.896 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:04.896 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:04.896 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:04.897 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:04.897 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 159 2024-04-26T03:39:04.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.897 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:04.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:04.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637451+0000", 2024-04-26T03:39:04.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:04.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:04.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637451+0000", 2024-04-26T03:39:04.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:04.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:04.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637452+0000", 2024-04-26T03:39:04.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0499999999999999e-07 2024-04-26T03:39:04.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:04.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637454+0000", 2024-04-26T03:39:04.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.853e-06 2024-04-26T03:39:04.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:04.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637460+0000", 2024-04-26T03:39:04.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5960000000000003e-06 2024-04-26T03:39:04.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:04.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637464+0000", 2024-04-26T03:39:04.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.155e-06 2024-04-26T03:39:04.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:04.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.643851+0000", 2024-04-26T03:39:04.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0063864560000000004 2024-04-26T03:39:04.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:04.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.643919+0000", 2024-04-26T03:39:04.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7834000000000001e-05 2024-04-26T03:39:04.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:04.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.652468+0000", 2024-04-26T03:39:04.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0085494760000000003 2024-04-26T03:39:04.900 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:04.900 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:04.900 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:04.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.900 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:160 3.as0 3:50cd80b4:::smithi09235267-25:head [read 0~409600,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:04.900 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.637472+0000", 2024-04-26T03:39:04.901 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.164208216, 2024-04-26T03:39:04.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015503981, 2024-04-26T03:39:04.901 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:04.901 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:04.901 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:04.901 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:04.901 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:04.901 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:04.901 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 160 2024-04-26T03:39:04.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.901 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:04.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:04.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637472+0000", 2024-04-26T03:39:04.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:04.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:04.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637472+0000", 2024-04-26T03:39:04.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:04.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:04.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637472+0000", 2024-04-26T03:39:04.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2800000000000002e-07 2024-04-26T03:39:04.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:04.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637475+0000", 2024-04-26T03:39:04.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.65e-06 2024-04-26T03:39:04.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:04.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637481+0000", 2024-04-26T03:39:04.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9309999999999996e-06 2024-04-26T03:39:04.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:04.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637487+0000", 2024-04-26T03:39:04.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6830000000000002e-06 2024-04-26T03:39:04.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:04.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.644230+0000", 2024-04-26T03:39:04.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067428519999999997 2024-04-26T03:39:04.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:04.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.644300+0000", 2024-04-26T03:39:04.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9443999999999994e-05 2024-04-26T03:39:04.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:04.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.652976+0000", 2024-04-26T03:39:04.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0086759929999999999 2024-04-26T03:39:04.904 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:04.905 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:04.905 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:04.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.905 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:161 3.as0 3:50cd80b4:::smithi09235267-25:head [read 0~409600,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:04.905 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.637494+0000", 2024-04-26T03:39:04.905 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.164185823, 2024-04-26T03:39:04.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015912572999999999, 2024-04-26T03:39:04.905 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:04.905 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:04.905 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:04.905 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:04.905 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:04.905 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:04.905 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 161 2024-04-26T03:39:04.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.906 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:04.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:04.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637494+0000", 2024-04-26T03:39:04.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:04.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:04.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637494+0000", 2024-04-26T03:39:04.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:04.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:04.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637494+0000", 2024-04-26T03:39:04.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0400000000000002e-07 2024-04-26T03:39:04.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:04.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637496+0000", 2024-04-26T03:39:04.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8729999999999999e-06 2024-04-26T03:39:04.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:04.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637500+0000", 2024-04-26T03:39:04.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4769999999999999e-06 2024-04-26T03:39:04.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:04.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637502+0000", 2024-04-26T03:39:04.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5100000000000001e-06 2024-04-26T03:39:04.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:04.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.648250+0000", 2024-04-26T03:39:04.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010747283999999999 2024-04-26T03:39:04.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:04.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.648336+0000", 2024-04-26T03:39:04.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6910999999999995e-05 2024-04-26T03:39:04.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:04.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.653407+0000", 2024-04-26T03:39:04.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005070214 2024-04-26T03:39:04.909 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:04.909 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:04.909 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:04.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.909 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:170 3.as0 3:58923ed4:::smithi09235267-39:head [read 0~638976,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:04.909 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.644344+0000", 2024-04-26T03:39:04.909 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.157335765999999, 2024-04-26T03:39:04.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011871138999999999, 2024-04-26T03:39:04.910 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:04.910 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:04.910 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:04.910 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:04.910 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:04.910 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:04.910 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 170 2024-04-26T03:39:04.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.910 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:04.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:04.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.644344+0000", 2024-04-26T03:39:04.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:04.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:04.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.644344+0000", 2024-04-26T03:39:04.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:04.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:04.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.644345+0000", 2024-04-26T03:39:04.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5999999999999999e-07 2024-04-26T03:39:04.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:04.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.644348+0000", 2024-04-26T03:39:04.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.534e-06 2024-04-26T03:39:04.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:04.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.644355+0000", 2024-04-26T03:39:04.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7229999999999998e-06 2024-04-26T03:39:04.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:04.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.644360+0000", 2024-04-26T03:39:04.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2789999999999998e-06 2024-04-26T03:39:04.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:04.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.650772+0000", 2024-04-26T03:39:04.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006411791 2024-04-26T03:39:04.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:04.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.650874+0000", 2024-04-26T03:39:04.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101847 2024-04-26T03:39:04.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:04.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.656215+0000", 2024-04-26T03:39:04.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053415049999999999 2024-04-26T03:39:04.913 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:04.913 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:04.913 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:04.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.914 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:171 3.as0 3:58923ed4:::smithi09235267-39:head [read 0~638976,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:04.914 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.644368+0000", 2024-04-26T03:39:04.914 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.157311992, 2024-04-26T03:39:04.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013770015, 2024-04-26T03:39:04.914 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:04.914 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:04.914 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:04.914 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:04.914 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:04.914 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:04.914 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 171 2024-04-26T03:39:04.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.915 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:04.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:04.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.644368+0000", 2024-04-26T03:39:04.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:04.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:04.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.644368+0000", 2024-04-26T03:39:04.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:04.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:04.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.644368+0000", 2024-04-26T03:39:04.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9599999999999999e-07 2024-04-26T03:39:04.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:04.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.644372+0000", 2024-04-26T03:39:04.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6040000000000001e-06 2024-04-26T03:39:04.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:04.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.644378+0000", 2024-04-26T03:39:04.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.241e-06 2024-04-26T03:39:04.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:04.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.644383+0000", 2024-04-26T03:39:04.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8629999999999999e-06 2024-04-26T03:39:04.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:04.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.656754+0000", 2024-04-26T03:39:04.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012370622 2024-04-26T03:39:04.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:04.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.656844+0000", 2024-04-26T03:39:04.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9987000000000004e-05 2024-04-26T03:39:04.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:04.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.658138+0000", 2024-04-26T03:39:04.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0012942019999999999 2024-04-26T03:39:04.918 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:04.918 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:04.918 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:04.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.918 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:310 3.as0 3:50cd80b4:::smithi09235267-25:head [copy-from ver 3] snapc 6=[6,5] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-26T03:39:04.918 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:59.536617+0000", 2024-04-26T03:39:04.918 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.265062479, 2024-04-26T03:39:04.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022665805000000001, 2024-04-26T03:39:04.918 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:04.918 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:04.919 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:04.919 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:04.919 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:04.919 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:04.919 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 310 2024-04-26T03:39:04.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.919 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:04.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.919 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:04.919 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536617+0000", 2024-04-26T03:39:04.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:04.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:04.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536617+0000", 2024-04-26T03:39:04.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:04.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:04.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536620+0000", 2024-04-26T03:39:04.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0540000000000002e-06 2024-04-26T03:39:04.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:04.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536636+0000", 2024-04-26T03:39:04.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6963e-05 2024-04-26T03:39:04.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:04.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536710+0000", 2024-04-26T03:39:04.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3711e-05 2024-04-26T03:39:04.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:04.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536719+0000", 2024-04-26T03:39:04.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9889999999999998e-06 2024-04-26T03:39:04.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:04.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536789+0000", 2024-04-26T03:39:04.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9884999999999999e-05 2024-04-26T03:39:04.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-04-26T03:39:04.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536797+0000", 2024-04-26T03:39:04.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5909999999999998e-06 2024-04-26T03:39:04.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:04.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.538587+0000", 2024-04-26T03:39:04.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017899249999999999 2024-04-26T03:39:04.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:04.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.538669+0000", 2024-04-26T03:39:04.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2045000000000005e-05 2024-04-26T03:39:04.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:04.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.549606+0000", 2024-04-26T03:39:04.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010937448000000001 2024-04-26T03:39:04.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:04.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.558484+0000", 2024-04-26T03:39:04.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00887808 2024-04-26T03:39:04.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:04.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.559195+0000", 2024-04-26T03:39:04.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00071067799999999996 2024-04-26T03:39:04.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:04.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.559283+0000", 2024-04-26T03:39:04.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8436000000000005e-05 2024-04-26T03:39:04.924 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:04.924 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:04.924 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:04.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.925 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:311 3.as0 3:50cd80b4:::smithi09235267-25:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e43)", 2024-04-26T03:39:04.925 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:59.536742+0000", 2024-04-26T03:39:04.925 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.2649375870000004, 2024-04-26T03:39:04.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021335742000000001, 2024-04-26T03:39:04.925 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:04.925 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:04.925 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:04.925 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:04.925 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:04.925 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:04.925 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 311 2024-04-26T03:39:04.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.926 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:04.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:04.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536742+0000", 2024-04-26T03:39:04.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:04.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:04.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536742+0000", 2024-04-26T03:39:04.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:04.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:04.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536743+0000", 2024-04-26T03:39:04.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5000000000000003e-07 2024-04-26T03:39:04.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:04.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536746+0000", 2024-04-26T03:39:04.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8720000000000002e-06 2024-04-26T03:39:04.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:04.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536771+0000", 2024-04-26T03:39:04.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4949000000000001e-05 2024-04-26T03:39:04.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:04.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536776+0000", 2024-04-26T03:39:04.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1089999999999997e-06 2024-04-26T03:39:04.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:04.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536853+0000", 2024-04-26T03:39:04.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7348000000000006e-05 2024-04-26T03:39:04.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-04-26T03:39:04.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536859+0000", 2024-04-26T03:39:04.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.784e-06 2024-04-26T03:39:04.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:04.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.538806+0000", 2024-04-26T03:39:04.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019471029999999999 2024-04-26T03:39:04.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for blocked object", 2024-04-26T03:39:04.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.538849+0000", 2024-04-26T03:39:04.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2901000000000001e-05 2024-04-26T03:39:04.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:04.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.557853+0000", 2024-04-26T03:39:04.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019003955999999999 2024-04-26T03:39:04.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:04.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.557954+0000", 2024-04-26T03:39:04.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010153399999999999 2024-04-26T03:39:04.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:04.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.558078+0000", 2024-04-26T03:39:04.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012363599999999999 2024-04-26T03:39:04.931 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:04.931 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:04.931 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:04.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.932 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.2s2 49/44 ECSubWrite(tid=34, reqid=client.4556.0:434, at_version=49'23, trim_to=0'0, roll_forward_to=48'19))", 2024-04-26T03:39:04.932 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:04.712444+0000", 2024-04-26T03:39:04.932 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.089235664000000006, 2024-04-26T03:39:04.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016297222, 2024-04-26T03:39:04.932 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:04.932 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:04.932 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:04.932 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:04.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:04.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.712444+0000", 2024-04-26T03:39:04.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:04.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:04.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.712444+0000", 2024-04-26T03:39:04.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:04.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:04.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.712445+0000", 2024-04-26T03:39:04.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4399999999999999e-07 2024-04-26T03:39:04.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:04.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.712821+0000", 2024-04-26T03:39:04.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000376194 2024-04-26T03:39:04.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:04.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.712838+0000", 2024-04-26T03:39:04.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7076999999999999e-05 2024-04-26T03:39:04.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:04.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.712854+0000", 2024-04-26T03:39:04.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5852999999999999e-05 2024-04-26T03:39:04.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:04.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.712940+0000", 2024-04-26T03:39:04.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6061000000000002e-05 2024-04-26T03:39:04.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:04.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.712952+0000", 2024-04-26T03:39:04.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1810999999999999e-05 2024-04-26T03:39:04.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:04.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.728691+0000", 2024-04-26T03:39:04.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015739306000000002 2024-04-26T03:39:04.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:04.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:04.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:04.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.728741+0000", 2024-04-26T03:39:04.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0076e-05 2024-04-26T03:39:04.936 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:04.936 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:04.936 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:04.936 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:04.936 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:04.936 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:04.938 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-26T03:39:04.938 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:39:04.936+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:39:05.038 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-04-26T03:39:05.037+0000 7fdc2d21c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31819) UID: 0 2024-04-26T03:39:05.082 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:05.082 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-26T03:39:05.082 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-26T03:39:05.083 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:05.091 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-26T03:39:05.140 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:39:05.138+0000 7fba0d2ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31961) UID: 0 2024-04-26T03:39:05.235 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:05.235 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-26T03:39:05.235 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-26T03:39:05.235 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-26T03:39:05.235 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:05.240 INFO:tasks.ceph.osd.10.smithi066.stderr:2024-04-26T03:39:05.239+0000 7f130fcc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 31763) UID: 0 2024-04-26T03:39:05.243 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-26T03:39:05.341 INFO:tasks.ceph.osd.4.smithi028.stderr:2024-04-26T03:39:05.340+0000 7fd9511b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31963) UID: 0 2024-04-26T03:39:05.348 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:05.348 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-26T03:39:05.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-26T03:39:05.348 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-26T03:39:05.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.348 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:43 3.ds0 3:b4337c88:::smithi09235267-15:head [append 0~245760,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:05.348 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.498664+0000", 2024-04-26T03:39:05.348 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.846857476, 2024-04-26T03:39:05.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013831098, 2024-04-26T03:39:05.348 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:05.348 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:05.349 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:05.349 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:05.349 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:05.349 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:05.349 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 43 2024-04-26T03:39:05.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.349 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:05.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:05.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498664+0000", 2024-04-26T03:39:05.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:05.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498664+0000", 2024-04-26T03:39:05.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:05.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498665+0000", 2024-04-26T03:39:05.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6470000000000001e-06 2024-04-26T03:39:05.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:05.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498872+0000", 2024-04-26T03:39:05.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000206744 2024-04-26T03:39:05.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:05.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498886+0000", 2024-04-26T03:39:05.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3651e-05 2024-04-26T03:39:05.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:05.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498894+0000", 2024-04-26T03:39:05.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2900000000000002e-06 2024-04-26T03:39:05.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498969+0000", 2024-04-26T03:39:05.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5511000000000003e-05 2024-04-26T03:39:05.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:05.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499103+0000", 2024-04-26T03:39:05.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000133585 2024-04-26T03:39:05.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:05.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500082+0000", 2024-04-26T03:39:05.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00097891300000000009 2024-04-26T03:39:05.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:05.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.509562+0000", 2024-04-26T03:39:05.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0094798539999999994 2024-04-26T03:39:05.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:05.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.512412+0000", 2024-04-26T03:39:05.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028504160000000001 2024-04-26T03:39:05.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:05.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.512495+0000", 2024-04-26T03:39:05.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2486999999999998e-05 2024-04-26T03:39:05.355 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.355 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:05.355 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.355 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:44 3.ds0 3:b4337c88:::smithi09235267-15:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:05.355 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.498924+0000", 2024-04-26T03:39:05.355 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.846597395, 2024-04-26T03:39:05.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013970621000000001, 2024-04-26T03:39:05.355 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:05.356 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:05.356 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:05.356 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:05.356 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:05.356 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:05.356 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 44 2024-04-26T03:39:05.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.356 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:05.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:05.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498924+0000", 2024-04-26T03:39:05.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:05.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498924+0000", 2024-04-26T03:39:05.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:05.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498924+0000", 2024-04-26T03:39:05.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8700000000000005e-07 2024-04-26T03:39:05.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:05.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498928+0000", 2024-04-26T03:39:05.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0450000000000002e-06 2024-04-26T03:39:05.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:05.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498942+0000", 2024-04-26T03:39:05.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3867e-05 2024-04-26T03:39:05.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:05.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498949+0000", 2024-04-26T03:39:05.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5729999999999996e-06 2024-04-26T03:39:05.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.503325+0000", 2024-04-26T03:39:05.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043766289999999999 2024-04-26T03:39:05.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:05.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.503423+0000", 2024-04-26T03:39:05.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7079999999999999e-05 2024-04-26T03:39:05.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:05.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.503854+0000", 2024-04-26T03:39:05.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00043101599999999999 2024-04-26T03:39:05.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:05.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.509563+0000", 2024-04-26T03:39:05.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057094010000000002 2024-04-26T03:39:05.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:05.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.512822+0000", 2024-04-26T03:39:05.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0032585930000000002 2024-04-26T03:39:05.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:05.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.512894+0000", 2024-04-26T03:39:05.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2730000000000003e-05 2024-04-26T03:39:05.364 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.364 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:05.364 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.365 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:45 3.ds0 3:b4337c88:::smithi09235267-15:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:05.365 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.498959+0000", 2024-04-26T03:39:05.365 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.846562176999999, 2024-04-26T03:39:05.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.028116775, 2024-04-26T03:39:05.365 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:05.365 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:05.366 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:05.366 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:05.366 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:05.366 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:05.366 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45 2024-04-26T03:39:05.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.366 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:05.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:05.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498959+0000", 2024-04-26T03:39:05.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:05.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498959+0000", 2024-04-26T03:39:05.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:05.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498959+0000", 2024-04-26T03:39:05.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.58e-07 2024-04-26T03:39:05.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:05.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498963+0000", 2024-04-26T03:39:05.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2449999999999998e-06 2024-04-26T03:39:05.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:05.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498970+0000", 2024-04-26T03:39:05.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5780000000000001e-06 2024-04-26T03:39:05.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:05.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498976+0000", 2024-04-26T03:39:05.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6400000000000002e-06 2024-04-26T03:39:05.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.504199+0000", 2024-04-26T03:39:05.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052229190000000003 2024-04-26T03:39:05.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:05.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.504274+0000", 2024-04-26T03:39:05.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5187999999999999e-05 2024-04-26T03:39:05.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.512941+0000", 2024-04-26T03:39:05.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0086671879999999993 2024-04-26T03:39:05.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:05.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.513001+0000", 2024-04-26T03:39:05.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0204000000000002e-05 2024-04-26T03:39:05.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:05.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.527076+0000", 2024-04-26T03:39:05.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014074355 2024-04-26T03:39:05.373 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.373 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:05.373 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.373 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.5s4 34/32 ECSubWrite(tid=2, reqid=client.4556.0:29, at_version=0'0, trim_to=0'0, roll_forward_to=34'2))", 2024-04-26T03:39:05.373 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.499859+0000", 2024-04-26T03:39:05.373 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.845662011, 2024-04-26T03:39:05.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097334050000000005, 2024-04-26T03:39:05.374 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:05.374 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:05.374 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:05.374 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:05.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:05.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499859+0000", 2024-04-26T03:39:05.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:05.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499859+0000", 2024-04-26T03:39:05.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:05.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499860+0000", 2024-04-26T03:39:05.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4900000000000005e-07 2024-04-26T03:39:05.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:05.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499870+0000", 2024-04-26T03:39:05.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0685e-05 2024-04-26T03:39:05.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:05.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499883+0000", 2024-04-26T03:39:05.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.224e-05 2024-04-26T03:39:05.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:05.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499891+0000", 2024-04-26T03:39:05.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8960000000000003e-06 2024-04-26T03:39:05.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.504052+0000", 2024-04-26T03:39:05.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041611030000000002 2024-04-26T03:39:05.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:05.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.504061+0000", 2024-04-26T03:39:05.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3160000000000003e-06 2024-04-26T03:39:05.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:05.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.509564+0000", 2024-04-26T03:39:05.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055027640000000003 2024-04-26T03:39:05.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:05.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.509592+0000", 2024-04-26T03:39:05.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8651999999999999e-05 2024-04-26T03:39:05.381 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.381 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:05.381 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.381 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:52 3.ds0 3:b895fe71:::smithi09235267-18:head [append 0~294912,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:05.382 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.505560+0000", 2024-04-26T03:39:05.382 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.839961177999999, 2024-04-26T03:39:05.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016117464000000001, 2024-04-26T03:39:05.382 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:05.382 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:05.382 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:05.382 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:05.382 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:05.382 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:05.382 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 52 2024-04-26T03:39:05.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.382 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:05.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:05.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505560+0000", 2024-04-26T03:39:05.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:05.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505560+0000", 2024-04-26T03:39:05.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:05.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505562+0000", 2024-04-26T03:39:05.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0729999999999999e-06 2024-04-26T03:39:05.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:05.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505753+0000", 2024-04-26T03:39:05.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019100500000000001 2024-04-26T03:39:05.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:05.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505768+0000", 2024-04-26T03:39:05.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5213e-05 2024-04-26T03:39:05.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:05.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505776+0000", 2024-04-26T03:39:05.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7279999999999995e-06 2024-04-26T03:39:05.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505830+0000", 2024-04-26T03:39:05.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3749999999999999e-05 2024-04-26T03:39:05.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:05.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505998+0000", 2024-04-26T03:39:05.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000168259 2024-04-26T03:39:05.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:05.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.506585+0000", 2024-04-26T03:39:05.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00058713299999999999 2024-04-26T03:39:05.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:05.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.511525+0000", 2024-04-26T03:39:05.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004939789 2024-04-26T03:39:05.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:05.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.521601+0000", 2024-04-26T03:39:05.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010075766 2024-04-26T03:39:05.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:05.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.521677+0000", 2024-04-26T03:39:05.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6748000000000004e-05 2024-04-26T03:39:05.388 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.388 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:05.388 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.388 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:53 3.ds0 3:b895fe71:::smithi09235267-18:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:05.389 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.505792+0000", 2024-04-26T03:39:05.389 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.839728739, 2024-04-26T03:39:05.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016119725000000001, 2024-04-26T03:39:05.389 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:05.389 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:05.389 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:05.389 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:05.389 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:05.389 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:05.389 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 53 2024-04-26T03:39:05.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.389 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:05.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:05.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505792+0000", 2024-04-26T03:39:05.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:05.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505792+0000", 2024-04-26T03:39:05.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:05.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505793+0000", 2024-04-26T03:39:05.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5400000000000001e-07 2024-04-26T03:39:05.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:05.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505806+0000", 2024-04-26T03:39:05.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2733e-05 2024-04-26T03:39:05.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:05.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505814+0000", 2024-04-26T03:39:05.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0260000000000007e-06 2024-04-26T03:39:05.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:05.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505820+0000", 2024-04-26T03:39:05.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8769999999999998e-06 2024-04-26T03:39:05.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.509659+0000", 2024-04-26T03:39:05.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0038395149999999999 2024-04-26T03:39:05.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:05.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.509764+0000", 2024-04-26T03:39:05.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105159 2024-04-26T03:39:05.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:05.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.510189+0000", 2024-04-26T03:39:05.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00042430899999999999 2024-04-26T03:39:05.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:05.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.511526+0000", 2024-04-26T03:39:05.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001337934 2024-04-26T03:39:05.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:05.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.521837+0000", 2024-04-26T03:39:05.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010310503 2024-04-26T03:39:05.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:05.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.521912+0000", 2024-04-26T03:39:05.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5014999999999999e-05 2024-04-26T03:39:05.395 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.396 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:05.396 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.396 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:54 3.ds0 3:b895fe71:::smithi09235267-18:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:05.396 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.505829+0000", 2024-04-26T03:39:05.396 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.839691898, 2024-04-26T03:39:05.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021406122, 2024-04-26T03:39:05.396 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:05.396 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:05.396 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:05.396 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:05.396 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:05.396 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:05.397 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 54 2024-04-26T03:39:05.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.397 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:05.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:05.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505829+0000", 2024-04-26T03:39:05.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:05.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505829+0000", 2024-04-26T03:39:05.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:05.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505830+0000", 2024-04-26T03:39:05.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2e-07 2024-04-26T03:39:05.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:05.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505832+0000", 2024-04-26T03:39:05.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.509e-06 2024-04-26T03:39:05.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:05.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505839+0000", 2024-04-26T03:39:05.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9110000000000004e-06 2024-04-26T03:39:05.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:05.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505844+0000", 2024-04-26T03:39:05.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2109999999999997e-06 2024-04-26T03:39:05.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.510372+0000", 2024-04-26T03:39:05.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045275990000000002 2024-04-26T03:39:05.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:05.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.510419+0000", 2024-04-26T03:39:05.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7524000000000001e-05 2024-04-26T03:39:05.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.522258+0000", 2024-04-26T03:39:05.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011838448999999999 2024-04-26T03:39:05.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:05.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.522334+0000", 2024-04-26T03:39:05.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6134e-05 2024-04-26T03:39:05.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:05.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.527235+0000", 2024-04-26T03:39:05.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049012650000000001 2024-04-26T03:39:05.401 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.401 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:05.401 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.401 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.ds0 35/32 ECSubReadReply(tid=4, attrs_read=0))", 2024-04-26T03:39:05.401 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.513706+0000", 2024-04-26T03:39:05.402 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.831815157999999, 2024-04-26T03:39:05.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010589298, 2024-04-26T03:39:05.402 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:05.402 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:05.402 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:05.402 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:05.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:05.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.513706+0000", 2024-04-26T03:39:05.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:05.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.513706+0000", 2024-04-26T03:39:05.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:05.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.513707+0000", 2024-04-26T03:39:05.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2024-04-26T03:39:05.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:05.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.513719+0000", 2024-04-26T03:39:05.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2993000000000001e-05 2024-04-26T03:39:05.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:05.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.513735+0000", 2024-04-26T03:39:05.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.558e-05 2024-04-26T03:39:05.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:05.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.513742+0000", 2024-04-26T03:39:05.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9720000000000003e-06 2024-04-26T03:39:05.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.524263+0000", 2024-04-26T03:39:05.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010520730000000001 2024-04-26T03:39:05.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:05.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.524295+0000", 2024-04-26T03:39:05.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2387e-05 2024-04-26T03:39:05.406 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.406 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:05.406 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.406 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.ds0 35/32 ECSubReadReply(tid=4, attrs_read=0))", 2024-04-26T03:39:05.406 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.513708+0000", 2024-04-26T03:39:05.406 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.831813435000001, 2024-04-26T03:39:05.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0091089410000000006, 2024-04-26T03:39:05.406 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:05.406 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:05.406 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:05.407 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:05.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:05.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.513708+0000", 2024-04-26T03:39:05.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:05.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.513708+0000", 2024-04-26T03:39:05.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:05.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.513708+0000", 2024-04-26T03:39:05.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5999999999999999e-07 2024-04-26T03:39:05.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:05.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.513721+0000", 2024-04-26T03:39:05.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2713e-05 2024-04-26T03:39:05.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:05.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.513732+0000", 2024-04-26T03:39:05.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1464999999999999e-05 2024-04-26T03:39:05.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:05.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.513743+0000", 2024-04-26T03:39:05.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0900999999999999e-05 2024-04-26T03:39:05.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.522773+0000", 2024-04-26T03:39:05.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0090302049999999995 2024-04-26T03:39:05.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:05.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.522817+0000", 2024-04-26T03:39:05.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3297000000000002e-05 2024-04-26T03:39:05.410 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.411 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:05.411 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.411 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:64 3.ds0 3:b3f958c3:::smithi09235267-22:head [append 0~360448,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:05.411 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.515665+0000", 2024-04-26T03:39:05.411 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.829855703, 2024-04-26T03:39:05.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.023538667999999999, 2024-04-26T03:39:05.411 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:05.411 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:05.411 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:05.411 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:05.411 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:05.412 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:05.412 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 64 2024-04-26T03:39:05.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.412 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:05.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:05.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515665+0000", 2024-04-26T03:39:05.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:05.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515665+0000", 2024-04-26T03:39:05.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:05.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515667+0000", 2024-04-26T03:39:05.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.652e-06 2024-04-26T03:39:05.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:05.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515857+0000", 2024-04-26T03:39:05.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000190447 2024-04-26T03:39:05.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:05.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515869+0000", 2024-04-26T03:39:05.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1450999999999999e-05 2024-04-26T03:39:05.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:05.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515876+0000", 2024-04-26T03:39:05.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2024-04-26T03:39:05.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.527263+0000", 2024-04-26T03:39:05.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011387061 2024-04-26T03:39:05.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:05.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.527389+0000", 2024-04-26T03:39:05.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000126172 2024-04-26T03:39:05.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:05.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.528593+0000", 2024-04-26T03:39:05.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00120386 2024-04-26T03:39:05.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:05.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.533773+0000", 2024-04-26T03:39:05.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005179973 2024-04-26T03:39:05.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:05.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.539131+0000", 2024-04-26T03:39:05.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053582279999999996 2024-04-26T03:39:05.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:05.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.539204+0000", 2024-04-26T03:39:05.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2809e-05 2024-04-26T03:39:05.417 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.417 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:05.417 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.417 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:65 3.ds0 3:b3f958c3:::smithi09235267-22:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:05.417 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.515889+0000", 2024-04-26T03:39:05.417 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.829632063, 2024-04-26T03:39:05.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.023486671000000001, 2024-04-26T03:39:05.417 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:05.418 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:05.418 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:05.418 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:05.418 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:05.418 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:05.418 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 65 2024-04-26T03:39:05.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.418 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:05.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:05.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515889+0000", 2024-04-26T03:39:05.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:05.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515889+0000", 2024-04-26T03:39:05.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:05.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515890+0000", 2024-04-26T03:39:05.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.13e-07 2024-04-26T03:39:05.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:05.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515894+0000", 2024-04-26T03:39:05.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.386e-06 2024-04-26T03:39:05.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:05.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515903+0000", 2024-04-26T03:39:05.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.166e-06 2024-04-26T03:39:05.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:05.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515909+0000", 2024-04-26T03:39:05.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.412e-06 2024-04-26T03:39:05.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.532799+0000", 2024-04-26T03:39:05.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016889990000000001 2024-04-26T03:39:05.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:05.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.532885+0000", 2024-04-26T03:39:05.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6336999999999997e-05 2024-04-26T03:39:05.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:05.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.533334+0000", 2024-04-26T03:39:05.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00044950799999999999 2024-04-26T03:39:05.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:05.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.538492+0000", 2024-04-26T03:39:05.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051571389999999998 2024-04-26T03:39:05.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:05.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.539303+0000", 2024-04-26T03:39:05.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000811325 2024-04-26T03:39:05.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:05.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.539376+0000", 2024-04-26T03:39:05.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2794999999999997e-05 2024-04-26T03:39:05.423 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.423 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:05.423 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.423 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:66 3.ds0 3:b3f958c3:::smithi09235267-22:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:05.423 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.515917+0000", 2024-04-26T03:39:05.423 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.829604487999999, 2024-04-26T03:39:05.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.024757049, 2024-04-26T03:39:05.423 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:05.424 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:05.424 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:05.424 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:05.424 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:05.424 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:05.424 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 66 2024-04-26T03:39:05.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.424 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:05.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:05.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515917+0000", 2024-04-26T03:39:05.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:05.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515917+0000", 2024-04-26T03:39:05.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:05.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515917+0000", 2024-04-26T03:39:05.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8699999999999995e-07 2024-04-26T03:39:05.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:05.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515920+0000", 2024-04-26T03:39:05.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.055e-06 2024-04-26T03:39:05.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:05.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515927+0000", 2024-04-26T03:39:05.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5150000000000003e-06 2024-04-26T03:39:05.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:05.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515936+0000", 2024-04-26T03:39:05.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7920000000000004e-06 2024-04-26T03:39:05.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.533517+0000", 2024-04-26T03:39:05.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017580535000000001 2024-04-26T03:39:05.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:05.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.533598+0000", 2024-04-26T03:39:05.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1354999999999996e-05 2024-04-26T03:39:05.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.539706+0000", 2024-04-26T03:39:05.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061075569999999996 2024-04-26T03:39:05.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:05.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.539758+0000", 2024-04-26T03:39:05.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2500000000000002e-05 2024-04-26T03:39:05.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:05.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.540674+0000", 2024-04-26T03:39:05.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00091525299999999999 2024-04-26T03:39:05.428 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.428 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:05.428 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.428 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.5s4 35/32 ECSubWrite(tid=4, reqid=client.4556.0:67, at_version=35'3, trim_to=0'0, roll_forward_to=34'2))", 2024-04-26T03:39:05.428 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.520065+0000", 2024-04-26T03:39:05.428 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.82545575, 2024-04-26T03:39:05.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01368393, 2024-04-26T03:39:05.428 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:05.428 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:05.428 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:05.429 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:05.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:05.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.520065+0000", 2024-04-26T03:39:05.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:05.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.520065+0000", 2024-04-26T03:39:05.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:05.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.520066+0000", 2024-04-26T03:39:05.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2799999999999996e-07 2024-04-26T03:39:05.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:05.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.520102+0000", 2024-04-26T03:39:05.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6399000000000002e-05 2024-04-26T03:39:05.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:05.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.520112+0000", 2024-04-26T03:39:05.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0193e-05 2024-04-26T03:39:05.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:05.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.520119+0000", 2024-04-26T03:39:05.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9999999999999999e-06 2024-04-26T03:39:05.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.523201+0000", 2024-04-26T03:39:05.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030814610000000002 2024-04-26T03:39:05.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:05.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.523209+0000", 2024-04-26T03:39:05.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9799999999999998e-06 2024-04-26T03:39:05.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:05.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.533719+0000", 2024-04-26T03:39:05.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010510146 2024-04-26T03:39:05.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:05.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.533749+0000", 2024-04-26T03:39:05.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0222999999999998e-05 2024-04-26T03:39:05.432 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.432 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:05.432 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.432 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.5s4 35/32 ECSubWrite(tid=5, reqid=client.4556.0:68, at_version=35'4, trim_to=0'0, roll_forward_to=34'2))", 2024-04-26T03:39:05.432 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.524456+0000", 2024-04-26T03:39:05.432 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.821064956000001, 2024-04-26T03:39:05.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0093148220000000004, 2024-04-26T03:39:05.432 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:05.433 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:05.433 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:05.433 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:05.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:05.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.524456+0000", 2024-04-26T03:39:05.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:05.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.524456+0000", 2024-04-26T03:39:05.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:05.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.524457+0000", 2024-04-26T03:39:05.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7100000000000002e-07 2024-04-26T03:39:05.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:05.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.524475+0000", 2024-04-26T03:39:05.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7963000000000001e-05 2024-04-26T03:39:05.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:05.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.524484+0000", 2024-04-26T03:39:05.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8779999999999993e-06 2024-04-26T03:39:05.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:05.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.524492+0000", 2024-04-26T03:39:05.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1509999999999998e-06 2024-04-26T03:39:05.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.527465+0000", 2024-04-26T03:39:05.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029733049999999999 2024-04-26T03:39:05.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:05.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.527474+0000", 2024-04-26T03:39:05.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6819999999999992e-06 2024-04-26T03:39:05.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:05.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.533753+0000", 2024-04-26T03:39:05.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0062798319999999999 2024-04-26T03:39:05.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:05.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.533771+0000", 2024-04-26T03:39:05.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7439999999999999e-05 2024-04-26T03:39:05.436 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.437 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:05.437 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.437 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.6s5 35/32 ECSubWrite(tid=0, reqid=client.4556.0:146, at_version=35'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:39:05.437 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.630448+0000", 2024-04-26T03:39:05.437 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.71507349, 2024-04-26T03:39:05.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0087667289999999991, 2024-04-26T03:39:05.437 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:05.437 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:05.437 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:05.437 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:05.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:05.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.630448+0000", 2024-04-26T03:39:05.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:05.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.630448+0000", 2024-04-26T03:39:05.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:05.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.630448+0000", 2024-04-26T03:39:05.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3500000000000005e-07 2024-04-26T03:39:05.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:05.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.630870+0000", 2024-04-26T03:39:05.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00042182199999999997 2024-04-26T03:39:05.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:05.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.630881+0000", 2024-04-26T03:39:05.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1056999999999999e-05 2024-04-26T03:39:05.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:05.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.630887+0000", 2024-04-26T03:39:05.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8139999999999997e-06 2024-04-26T03:39:05.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.630949+0000", 2024-04-26T03:39:05.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2179000000000002e-05 2024-04-26T03:39:05.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:05.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.630958+0000", 2024-04-26T03:39:05.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9879999999999993e-06 2024-04-26T03:39:05.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:05.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.638549+0000", 2024-04-26T03:39:05.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0075908520000000004 2024-04-26T03:39:05.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:05.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.639214+0000", 2024-04-26T03:39:05.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00066518199999999995 2024-04-26T03:39:05.441 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.441 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:05.441 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.441 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.as1 35/32 ECSubWrite(tid=24, reqid=client.4556.0:150, at_version=35'13, trim_to=0'0, roll_forward_to=35'12))", 2024-04-26T03:39:05.441 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.635662+0000", 2024-04-26T03:39:05.441 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.709859427, 2024-04-26T03:39:05.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0088353879999999996, 2024-04-26T03:39:05.441 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:05.441 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:05.441 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:05.442 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:05.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:05.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.635662+0000", 2024-04-26T03:39:05.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:05.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.635662+0000", 2024-04-26T03:39:05.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:05.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.635662+0000", 2024-04-26T03:39:05.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0299999999999998e-07 2024-04-26T03:39:05.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:05.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.635706+0000", 2024-04-26T03:39:05.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3254999999999998e-05 2024-04-26T03:39:05.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:05.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.635720+0000", 2024-04-26T03:39:05.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3912e-05 2024-04-26T03:39:05.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:05.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.635727+0000", 2024-04-26T03:39:05.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7519999999999996e-06 2024-04-26T03:39:05.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.635807+0000", 2024-04-26T03:39:05.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9933999999999997e-05 2024-04-26T03:39:05.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:05.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.635817+0000", 2024-04-26T03:39:05.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6579999999999997e-06 2024-04-26T03:39:05.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:05.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.644454+0000", 2024-04-26T03:39:05.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0086367119999999995 2024-04-26T03:39:05.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:05.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.644497+0000", 2024-04-26T03:39:05.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3362000000000003e-05 2024-04-26T03:39:05.445 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.445 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:05.445 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.445 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.as1 35/32 ECSubWrite(tid=25, reqid=client.4556.0:151, at_version=35'14, trim_to=0'0, roll_forward_to=35'12))", 2024-04-26T03:39:05.445 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.640547+0000", 2024-04-26T03:39:05.445 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.704974353000001, 2024-04-26T03:39:05.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010410780999999999, 2024-04-26T03:39:05.446 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:05.446 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:05.446 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:05.446 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:05.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:05.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.640547+0000", 2024-04-26T03:39:05.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:05.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.640547+0000", 2024-04-26T03:39:05.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:05.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.640547+0000", 2024-04-26T03:39:05.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0299999999999998e-07 2024-04-26T03:39:05.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:05.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.640588+0000", 2024-04-26T03:39:05.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0782999999999999e-05 2024-04-26T03:39:05.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:05.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.640599+0000", 2024-04-26T03:39:05.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0946e-05 2024-04-26T03:39:05.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:05.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.640607+0000", 2024-04-26T03:39:05.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4619999999999999e-06 2024-04-26T03:39:05.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.641377+0000", 2024-04-26T03:39:05.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00077009699999999997 2024-04-26T03:39:05.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:05.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.641387+0000", 2024-04-26T03:39:05.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0370999999999999e-05 2024-04-26T03:39:05.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:05.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.650923+0000", 2024-04-26T03:39:05.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0095354670000000006 2024-04-26T03:39:05.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:05.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.650957+0000", 2024-04-26T03:39:05.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4952e-05 2024-04-26T03:39:05.450 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.450 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:05.450 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.450 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.as1 43/32 ECSubWrite(tid=54, reqid=client.4556.0:310, at_version=43'28, trim_to=0'0, roll_forward_to=42'26))", 2024-04-26T03:39:05.450 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:59.550115+0000", 2024-04-26T03:39:05.450 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.7954060790000002, 2024-04-26T03:39:05.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0088632399999999997, 2024-04-26T03:39:05.451 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:05.451 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:05.451 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:05.451 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:05.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:05.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.550115+0000", 2024-04-26T03:39:05.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:05.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.550115+0000", 2024-04-26T03:39:05.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:05.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.550115+0000", 2024-04-26T03:39:05.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.41e-07 2024-04-26T03:39:05.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:05.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.550729+0000", 2024-04-26T03:39:05.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00061396300000000003 2024-04-26T03:39:05.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:05.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.550743+0000", 2024-04-26T03:39:05.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3380999999999999e-05 2024-04-26T03:39:05.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:05.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.550751+0000", 2024-04-26T03:39:05.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2239999999999995e-06 2024-04-26T03:39:05.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.550845+0000", 2024-04-26T03:39:05.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3852999999999997e-05 2024-04-26T03:39:05.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:05.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.550854+0000", 2024-04-26T03:39:05.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4460000000000007e-06 2024-04-26T03:39:05.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:05.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.558898+0000", 2024-04-26T03:39:05.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0080437479999999999 2024-04-26T03:39:05.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:05.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.558978+0000", 2024-04-26T03:39:05.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0383999999999995e-05 2024-04-26T03:39:05.454 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.454 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:05.454 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.455 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.6s5 49/44 ECSubWrite(tid=6, reqid=client.4556.0:430, at_version=49'5, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:39:05.455 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:04.689657+0000", 2024-04-26T03:39:05.455 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.65586359100000002, 2024-04-26T03:39:05.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011270558, 2024-04-26T03:39:05.455 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:05.455 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:05.455 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:05.455 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:05.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:05.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.689657+0000", 2024-04-26T03:39:05.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:05.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.689657+0000", 2024-04-26T03:39:05.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:05.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.689658+0000", 2024-04-26T03:39:05.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0800000000000001e-07 2024-04-26T03:39:05.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:05.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.689673+0000", 2024-04-26T03:39:05.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5082999999999999e-05 2024-04-26T03:39:05.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:05.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.689680+0000", 2024-04-26T03:39:05.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3100000000000003e-06 2024-04-26T03:39:05.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:05.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.689687+0000", 2024-04-26T03:39:05.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2919999999999998e-06 2024-04-26T03:39:05.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.689783+0000", 2024-04-26T03:39:05.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5600999999999995e-05 2024-04-26T03:39:05.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.458 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:05.458 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.689803+0000", 2024-04-26T03:39:05.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9743999999999999e-05 2024-04-26T03:39:05.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.458 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:05.458 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.700892+0000", 2024-04-26T03:39:05.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011089447000000001 2024-04-26T03:39:05.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.458 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:05.458 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.700928+0000", 2024-04-26T03:39:05.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5772999999999998e-05 2024-04-26T03:39:05.459 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.459 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:05.459 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.459 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.6s5 49/44 ECSubWrite(tid=7, reqid=client.4556.0:431, at_version=49'6, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:39:05.459 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:04.691058+0000", 2024-04-26T03:39:05.459 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.65446272500000002, 2024-04-26T03:39:05.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010298882000000001, 2024-04-26T03:39:05.459 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:05.459 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:05.459 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:05.459 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:05.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:05.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.691058+0000", 2024-04-26T03:39:05.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:05.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.691058+0000", 2024-04-26T03:39:05.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:05.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.691059+0000", 2024-04-26T03:39:05.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3399999999999999e-07 2024-04-26T03:39:05.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:05.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.691075+0000", 2024-04-26T03:39:05.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6331e-05 2024-04-26T03:39:05.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:05.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.691091+0000", 2024-04-26T03:39:05.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5922000000000001e-05 2024-04-26T03:39:05.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:05.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.691099+0000", 2024-04-26T03:39:05.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7619999999999995e-06 2024-04-26T03:39:05.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.697163+0000", 2024-04-26T03:39:05.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060636580000000004 2024-04-26T03:39:05.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:05.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.697172+0000", 2024-04-26T03:39:05.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2110000000000004e-06 2024-04-26T03:39:05.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:05.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.701330+0000", 2024-04-26T03:39:05.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041578530000000004 2024-04-26T03:39:05.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:05.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.701357+0000", 2024-04-26T03:39:05.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7511000000000001e-05 2024-04-26T03:39:05.463 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.463 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:05.463 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.463 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.463 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:05.463 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:05.464 INFO:tasks.ceph.osd.4.smithi028.stderr:2024-04-26T03:39:05.463+0000 7fd9511b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31963) UID: 0 2024-04-26T03:39:05.465 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-04-26T03:39:05.565 INFO:tasks.ceph.osd.14.smithi066.stderr:2024-04-26T03:39:05.564+0000 7f5395f08640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 31765) UID: 0 2024-04-26T03:39:05.570 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:05.570 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-26T03:39:05.570 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-26T03:39:05.570 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:05.579 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-04-26T03:39:05.666 INFO:tasks.ceph.osd.12.smithi028.stderr:2024-04-26T03:39:05.665+0000 7f114fe14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 31967) UID: 0 2024-04-26T03:39:05.683 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:05.683 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-26T03:39:05.683 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-26T03:39:05.683 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-26T03:39:05.683 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:05.684 INFO:teuthology.orchestra.run.smithi028.stderr:noscrub is unset 2024-04-26T03:39:05.688 INFO:tasks.rados.rados.0.smithi092.stdout:170: finishing write tid 2 to smithi09235267-1 2024-04-26T03:39:05.688 INFO:tasks.rados.rados.0.smithi092.stdout:170: finishing write tid 3 to smithi09235267-1 2024-04-26T03:39:05.688 INFO:tasks.rados.rados.0.smithi092.stdout:170: finishing write tid 4 to smithi09235267-1 2024-04-26T03:39:05.688 INFO:tasks.rados.rados.0.smithi092.stdout:170: oid 1 updating version 0 to 24 2024-04-26T03:39:05.689 INFO:tasks.rados.rados.0.smithi092.stdout:170: oid 1 version 24 is already newer than 23 2024-04-26T03:39:05.689 INFO:tasks.rados.rados.0.smithi092.stdout:170: oid 1 version 24 is already newer than 20 2024-04-26T03:39:05.689 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 1 v 24 (ObjNum 85 snap 7 seq_num 85) dirty exists 2024-04-26T03:39:05.689 INFO:tasks.rados.rados.0.smithi092.stdout:170: left oid 1 (ObjNum 85 snap 7 seq_num 85) 2024-04-26T03:39:05.689 INFO:tasks.rados.rados.0.smithi092.stdout:172: finishing write tid 1 to smithi09235267-28 2024-04-26T03:39:05.689 INFO:tasks.rados.rados.0.smithi092.stdout:173: finishing copy_from racing read to smithi09235267-2 2024-04-26T03:39:05.689 INFO:tasks.rados.rados.0.smithi092.stdout:172: finishing write tid 2 to smithi09235267-28 2024-04-26T03:39:05.689 INFO:tasks.rados.rados.0.smithi092.stdout:172: finishing write tid 3 to smithi09235267-28 2024-04-26T03:39:05.689 INFO:tasks.rados.rados.0.smithi092.stdout:172: finishing write tid 4 to smithi09235267-28 2024-04-26T03:39:05.689 INFO:tasks.rados.rados.0.smithi092.stdout:173: finishing copy_from to smithi09235267-2 2024-04-26T03:39:05.689 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 2 v 3 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-04-26T03:39:05.689 INFO:tasks.rados.rados.0.smithi092.stdout:172: finishing write tid 5 to smithi09235267-28 2024-04-26T03:39:05.690 INFO:tasks.rados.rados.0.smithi092.stdout:172: oid 28 updating version 0 to 25 2024-04-26T03:39:05.690 INFO:tasks.rados.rados.0.smithi092.stdout:172: oid 28 updating version 25 to 26 2024-04-26T03:39:05.690 INFO:tasks.rados.rados.0.smithi092.stdout:172: oid 28 version 26 is already newer than 24 2024-04-26T03:39:05.690 INFO:tasks.rados.rados.0.smithi092.stdout:172: oid 28 updating version 26 to 27 2024-04-26T03:39:05.690 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 28 v 27 (ObjNum 87 snap 7 seq_num 87) dirty exists 2024-04-26T03:39:05.691 INFO:tasks.rados.rados.0.smithi092.stdout:172: left oid 28 (ObjNum 87 snap 7 seq_num 87) 2024-04-26T03:39:05.691 INFO:tasks.rados.rados.0.smithi092.stdout:170: done (3 left) 2024-04-26T03:39:05.691 INFO:tasks.rados.rados.0.smithi092.stdout:172: done (2 left) 2024-04-26T03:39:05.691 INFO:tasks.rados.rados.0.smithi092.stdout:173: done (1 left) 2024-04-26T03:39:05.692 INFO:tasks.rados.rados.0.smithi092.stdout:174: done (0 left) 2024-04-26T03:39:05.692 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 6 current snap is 7 2024-04-26T03:39:05.692 INFO:tasks.rados.rados.0.smithi092.stdout:175: seq_num 88 ranges {1458176=720896,2179072=442368,2621440=327680} 2024-04-26T03:39:05.695 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-04-26T03:39:05.700 INFO:tasks.rados.rados.0.smithi092.stdout:175: writing smithi09235267-6 from 1458176 to 2179072 tid 1 2024-04-26T03:39:05.702 INFO:tasks.rados.rados.0.smithi092.stdout:175: writing smithi09235267-6 from 2179072 to 2621440 tid 2 2024-04-26T03:39:05.705 INFO:tasks.rados.rados.0.smithi092.stdout:175: writing smithi09235267-6 from 2621440 to 2949120 tid 3 2024-04-26T03:39:05.705 INFO:tasks.rados.rados.0.smithi092.stdout:176: read oid 33 snap -1 2024-04-26T03:39:05.705 INFO:tasks.rados.rados.0.smithi092.stdout:176: expect (ObjNum 32 snap 0 seq_num 32) 2024-04-26T03:39:05.705 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 50 current snap is 7 2024-04-26T03:39:05.705 INFO:tasks.rados.rados.0.smithi092.stdout:177: seq_num 89 ranges {0=737280,737280=573440,1310720=196608} 2024-04-26T03:39:05.710 INFO:tasks.rados.rados.0.smithi092.stdout:177: writing smithi09235267-50 from 0 to 737280 tid 1 2024-04-26T03:39:05.714 INFO:tasks.rados.rados.0.smithi092.stdout:177: writing smithi09235267-50 from 737280 to 1310720 tid 2 2024-04-26T03:39:05.716 INFO:tasks.rados.rados.0.smithi092.stdout:177: writing smithi09235267-50 from 1310720 to 1507328 tid 3 2024-04-26T03:39:05.716 INFO:tasks.rados.rados.0.smithi092.stdout:176: expect (ObjNum 32 snap 0 seq_num 32) 2024-04-26T03:39:05.738 INFO:tasks.rados.rados.0.smithi092.stdout:175: finishing write tid 1 to smithi09235267-6 2024-04-26T03:39:05.739 INFO:tasks.rados.rados.0.smithi092.stdout:175: finishing write tid 2 to smithi09235267-6 2024-04-26T03:39:05.739 INFO:tasks.rados.rados.0.smithi092.stdout:175: finishing write tid 3 to smithi09235267-6 2024-04-26T03:39:05.739 INFO:tasks.rados.rados.0.smithi092.stdout:175: finishing write tid 4 to smithi09235267-6 2024-04-26T03:39:05.739 INFO:tasks.rados.rados.0.smithi092.stdout:175: finishing write tid 5 to smithi09235267-6 2024-04-26T03:39:05.739 INFO:tasks.rados.rados.0.smithi092.stdout:175: oid 6 updating version 0 to 40 2024-04-26T03:39:05.739 INFO:tasks.rados.rados.0.smithi092.stdout:175: oid 6 version 40 is already newer than 39 2024-04-26T03:39:05.739 INFO:tasks.rados.rados.0.smithi092.stdout:175: oid 6 version 40 is already newer than 38 2024-04-26T03:39:05.739 INFO:tasks.rados.rados.0.smithi092.stdout:175: oid 6 version 40 is already newer than 37 2024-04-26T03:39:05.739 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 6 v 40 (ObjNum 88 snap 7 seq_num 88) dirty exists 2024-04-26T03:39:05.739 INFO:tasks.rados.rados.0.smithi092.stdout:175: left oid 6 (ObjNum 88 snap 7 seq_num 88) 2024-04-26T03:39:05.739 INFO:tasks.rados.rados.0.smithi092.stdout:177: finishing write tid 1 to smithi09235267-50 2024-04-26T03:39:05.739 INFO:tasks.rados.rados.0.smithi092.stdout:177: finishing write tid 2 to smithi09235267-50 2024-04-26T03:39:05.740 INFO:tasks.rados.rados.0.smithi092.stdout:177: finishing write tid 3 to smithi09235267-50 2024-04-26T03:39:05.740 INFO:tasks.rados.rados.0.smithi092.stdout:177: finishing write tid 4 to smithi09235267-50 2024-04-26T03:39:05.740 INFO:tasks.rados.rados.0.smithi092.stdout:175: done (2 left) 2024-04-26T03:39:05.740 INFO:tasks.rados.rados.0.smithi092.stdout:176: done (1 left) 2024-04-26T03:39:05.740 INFO:tasks.rados.rados.0.smithi092.stdout:178: read oid 12 snap -1 2024-04-26T03:39:05.740 INFO:tasks.rados.rados.0.smithi092.stdout:178: expect (ObjNum 11 snap 0 seq_num 11) 2024-04-26T03:39:05.740 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 10 current snap is 7 2024-04-26T03:39:05.740 INFO:tasks.rados.rados.0.smithi092.stdout:177: finishing write tid 5 to smithi09235267-50 2024-04-26T03:39:05.740 INFO:tasks.rados.rados.0.smithi092.stdout:177: oid 50 updating version 0 to 40 2024-04-26T03:39:05.740 INFO:tasks.rados.rados.0.smithi092.stdout:177: oid 50 version 40 is already newer than 38 2024-04-26T03:39:05.740 INFO:tasks.rados.rados.0.smithi092.stdout:177: oid 50 version 40 is already newer than 39 2024-04-26T03:39:05.740 INFO:tasks.rados.rados.0.smithi092.stdout:177: oid 50 version 40 is already newer than 37 2024-04-26T03:39:05.740 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 50 v 40 (ObjNum 89 snap 7 seq_num 89) dirty exists 2024-04-26T03:39:05.741 INFO:tasks.rados.rados.0.smithi092.stdout:177: left oid 50 (ObjNum 89 snap 7 seq_num 89) 2024-04-26T03:39:05.741 INFO:tasks.rados.rados.0.smithi092.stdout:179: seq_num 90 ranges {1081344=753664,1835008=720896,2555904=49152} 2024-04-26T03:39:05.750 INFO:tasks.rados.rados.0.smithi092.stdout:179: writing smithi09235267-10 from 1081344 to 1835008 tid 1 2024-04-26T03:39:05.755 INFO:tasks.rados.rados.0.smithi092.stdout:179: writing smithi09235267-10 from 1835008 to 2555904 tid 2 2024-04-26T03:39:05.756 INFO:tasks.rados.rados.0.smithi092.stdout:179: writing smithi09235267-10 from 2555904 to 2605056 tid 3 2024-04-26T03:39:05.756 INFO:tasks.rados.rados.0.smithi092.stdout:177: done (2 left) 2024-04-26T03:39:05.756 INFO:tasks.rados.rados.0.smithi092.stdout:180: delete oid 23 current snap is 7 2024-04-26T03:39:05.756 INFO:tasks.rados.rados.0.smithi092.stdout:178: expect (ObjNum 11 snap 0 seq_num 11) 2024-04-26T03:39:05.764 INFO:tasks.rados.rados.0.smithi092.stdout:179: finishing write tid 1 to smithi09235267-10 2024-04-26T03:39:05.766 INFO:tasks.rados.rados.0.smithi092.stdout:179: finishing write tid 2 to smithi09235267-10 2024-04-26T03:39:05.766 INFO:tasks.rados.rados.0.smithi092.stdout:179: finishing write tid 3 to smithi09235267-10 2024-04-26T03:39:05.767 INFO:tasks.rados.rados.0.smithi092.stdout:179: finishing write tid 4 to smithi09235267-10 2024-04-26T03:39:05.767 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-04-26T03:39:05.765+0000 7fe9223c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31954) UID: 0 2024-04-26T03:39:05.777 INFO:tasks.rados.rados.0.smithi092.stdout:178: done (2 left) 2024-04-26T03:39:05.778 INFO:tasks.rados.rados.0.smithi092.stdout:180: done (1 left) 2024-04-26T03:39:05.778 INFO:tasks.rados.rados.0.smithi092.stdout:181: rmattr oid 31 current snap is 7 2024-04-26T03:39:05.778 INFO:tasks.rados.rados.0.smithi092.stdout:182: setattr oid 27 current snap is 7 2024-04-26T03:39:05.778 INFO:tasks.rados.rados.0.smithi092.stdout:179: finishing write tid 5 to smithi09235267-10 2024-04-26T03:39:05.778 INFO:tasks.rados.rados.0.smithi092.stdout:179: oid 10 updating version 0 to 49 2024-04-26T03:39:05.778 INFO:tasks.rados.rados.0.smithi092.stdout:179: oid 10 version 49 is already newer than 48 2024-04-26T03:39:05.778 INFO:tasks.rados.rados.0.smithi092.stdout:179: oid 10 updating version 49 to 50 2024-04-26T03:39:05.778 INFO:tasks.rados.rados.0.smithi092.stdout:179: oid 10 version 50 is already newer than 47 2024-04-26T03:39:05.778 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 10 v 50 (ObjNum 90 snap 7 seq_num 90) dirty exists 2024-04-26T03:39:05.778 INFO:tasks.rados.rados.0.smithi092.stdout:179: left oid 10 (ObjNum 90 snap 7 seq_num 90) 2024-04-26T03:39:05.779 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 31 v 43 (ObjNum 30 snap 0 seq_num 30) dirty exists 2024-04-26T03:39:05.780 INFO:tasks.rados.rados.0.smithi092.stdout:179: done (2 left) 2024-04-26T03:39:05.780 INFO:tasks.rados.rados.0.smithi092.stdout:181: done (1 left) 2024-04-26T03:39:05.780 INFO:tasks.rados.rados.0.smithi092.stdout:183: read oid 42 snap -1 2024-04-26T03:39:05.780 INFO:tasks.rados.rados.0.smithi092.stdout:183: expect (ObjNum 60 snap 0 seq_num 60) 2024-04-26T03:39:05.780 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 44 current snap is 7 2024-04-26T03:39:05.780 INFO:tasks.rados.rados.0.smithi092.stdout:184: seq_num 92 ranges {737280=786432,1523712=491520,2015232=278528} 2024-04-26T03:39:05.790 INFO:tasks.rados.rados.0.smithi092.stdout:184: writing smithi09235267-44 from 737280 to 1523712 tid 1 2024-04-26T03:39:05.794 INFO:tasks.rados.rados.0.smithi092.stdout:184: writing smithi09235267-44 from 1523712 to 2015232 tid 2 2024-04-26T03:39:05.796 INFO:tasks.rados.rados.0.smithi092.stdout:184: writing smithi09235267-44 from 2015232 to 2293760 tid 3 2024-04-26T03:39:05.796 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 47 current snap is 7 2024-04-26T03:39:05.796 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 27 v 27 (ObjNum 66 snap 3 seq_num 66) dirty exists 2024-04-26T03:39:05.797 INFO:tasks.rados.rados.0.smithi092.stdout:183: expect (ObjNum 60 snap 0 seq_num 60) 2024-04-26T03:39:05.832 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:05.832 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-26T03:39:05.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-26T03:39:05.832 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-26T03:39:05.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.832 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:11 3.cs0 3:3946a5f1:::smithi09235267-4:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:05.833 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.481691+0000", 2024-04-26T03:39:05.833 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.348180207, 2024-04-26T03:39:05.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016138393000000001, 2024-04-26T03:39:05.833 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:05.833 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:05.833 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:05.833 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:05.833 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:05.833 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:05.833 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 11 2024-04-26T03:39:05.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.833 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:05.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:05.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.481691+0000", 2024-04-26T03:39:05.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:05.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.481691+0000", 2024-04-26T03:39:05.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:05.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.481692+0000", 2024-04-26T03:39:05.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3600000000000004e-07 2024-04-26T03:39:05.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:05.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.481696+0000", 2024-04-26T03:39:05.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.895e-06 2024-04-26T03:39:05.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:05.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.481706+0000", 2024-04-26T03:39:05.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9839999999999996e-06 2024-04-26T03:39:05.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:05.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.481712+0000", 2024-04-26T03:39:05.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9959999999999999e-06 2024-04-26T03:39:05.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.483468+0000", 2024-04-26T03:39:05.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017565429999999999 2024-04-26T03:39:05.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:05.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.483559+0000", 2024-04-26T03:39:05.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0433999999999995e-05 2024-04-26T03:39:05.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:05.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.484002+0000", 2024-04-26T03:39:05.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00044346799999999998 2024-04-26T03:39:05.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:05.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485764+0000", 2024-04-26T03:39:05.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017621049999999999 2024-04-26T03:39:05.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:05.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.497735+0000", 2024-04-26T03:39:05.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011970998 2024-04-26T03:39:05.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:05.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.497830+0000", 2024-04-26T03:39:05.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4433999999999997e-05 2024-04-26T03:39:05.839 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.839 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:05.839 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.840 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:12 3.cs0 3:3946a5f1:::smithi09235267-4:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:05.840 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.481722+0000", 2024-04-26T03:39:05.840 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.348149429999999, 2024-04-26T03:39:05.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039281273999999998, 2024-04-26T03:39:05.840 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:05.840 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:05.840 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:05.840 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:05.840 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:05.840 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:05.840 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 12 2024-04-26T03:39:05.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.841 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:05.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:05.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.481722+0000", 2024-04-26T03:39:05.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:05.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.481722+0000", 2024-04-26T03:39:05.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:05.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.481723+0000", 2024-04-26T03:39:05.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.75e-07 2024-04-26T03:39:05.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:05.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.481725+0000", 2024-04-26T03:39:05.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6979999999999998e-06 2024-04-26T03:39:05.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:05.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.481733+0000", 2024-04-26T03:39:05.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2899999999999997e-06 2024-04-26T03:39:05.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:05.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.481743+0000", 2024-04-26T03:39:05.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0845999999999999e-05 2024-04-26T03:39:05.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.484203+0000", 2024-04-26T03:39:05.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024598889999999998 2024-04-26T03:39:05.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:05.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.484353+0000", 2024-04-26T03:39:05.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015005199999999999 2024-04-26T03:39:05.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.497880+0000", 2024-04-26T03:39:05.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013526589 2024-04-26T03:39:05.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:05.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.497957+0000", 2024-04-26T03:39:05.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7126000000000007e-05 2024-04-26T03:39:05.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:05.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.521003+0000", 2024-04-26T03:39:05.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.023046309000000001 2024-04-26T03:39:05.845 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.845 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:05.845 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.845 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:27 3.fs0 3:ff04ba47:::smithi09235267-9:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:05.845 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.486833+0000", 2024-04-26T03:39:05.846 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.343038578, 2024-04-26T03:39:05.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017143513999999999, 2024-04-26T03:39:05.846 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:05.846 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:05.846 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:05.846 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:05.846 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:05.846 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:05.846 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 27 2024-04-26T03:39:05.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.846 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:05.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:05.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.486833+0000", 2024-04-26T03:39:05.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:05.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.486833+0000", 2024-04-26T03:39:05.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:05.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.486833+0000", 2024-04-26T03:39:05.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8100000000000003e-07 2024-04-26T03:39:05.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:05.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.486836+0000", 2024-04-26T03:39:05.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3539999999999998e-06 2024-04-26T03:39:05.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:05.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.486843+0000", 2024-04-26T03:39:05.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9410000000000001e-06 2024-04-26T03:39:05.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:05.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.486848+0000", 2024-04-26T03:39:05.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.22e-06 2024-04-26T03:39:05.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.491735+0000", 2024-04-26T03:39:05.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0048875580000000002 2024-04-26T03:39:05.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:05.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.491812+0000", 2024-04-26T03:39:05.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6876000000000001e-05 2024-04-26T03:39:05.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501351+0000", 2024-04-26T03:39:05.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0095387330000000006 2024-04-26T03:39:05.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:05.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501430+0000", 2024-04-26T03:39:05.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9080999999999997e-05 2024-04-26T03:39:05.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:05.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.503976+0000", 2024-04-26T03:39:05.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025462699999999998 2024-04-26T03:39:05.850 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.851 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:05.851 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.851 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:31 3.cs0 3:3edefb40:::smithi09235267-11:head [append 0~180224,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:05.851 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.494104+0000", 2024-04-26T03:39:05.851 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.335767126, 2024-04-26T03:39:05.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025578053999999999, 2024-04-26T03:39:05.851 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:05.851 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:05.851 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:05.851 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:05.851 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:05.851 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:05.851 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 31 2024-04-26T03:39:05.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.852 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:05.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:05.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.494104+0000", 2024-04-26T03:39:05.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:05.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.494104+0000", 2024-04-26T03:39:05.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:05.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.494106+0000", 2024-04-26T03:39:05.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2100000000000001e-06 2024-04-26T03:39:05.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:05.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.494218+0000", 2024-04-26T03:39:05.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000112398 2024-04-26T03:39:05.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:05.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.494230+0000", 2024-04-26T03:39:05.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1688999999999999e-05 2024-04-26T03:39:05.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:05.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.494237+0000", 2024-04-26T03:39:05.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2049999999999996e-06 2024-04-26T03:39:05.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.494745+0000", 2024-04-26T03:39:05.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00050836900000000005 2024-04-26T03:39:05.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:05.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.494857+0000", 2024-04-26T03:39:05.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000112045 2024-04-26T03:39:05.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:05.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495764+0000", 2024-04-26T03:39:05.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00090692400000000003 2024-04-26T03:39:05.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:05.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519293+0000", 2024-04-26T03:39:05.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.023528830000000001 2024-04-26T03:39:05.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:05.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519597+0000", 2024-04-26T03:39:05.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030426100000000001 2024-04-26T03:39:05.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:05.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519682+0000", 2024-04-26T03:39:05.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5123000000000005e-05 2024-04-26T03:39:05.857 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.858 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:05.858 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.858 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:32 3.cs0 3:3edefb40:::smithi09235267-11:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:05.858 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.495390+0000", 2024-04-26T03:39:05.858 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.33448121, 2024-04-26T03:39:05.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027633438, 2024-04-26T03:39:05.859 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:05.859 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:05.859 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:05.859 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:05.859 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:05.859 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:05.859 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 32 2024-04-26T03:39:05.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.860 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:05.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:05.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495390+0000", 2024-04-26T03:39:05.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:05.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495390+0000", 2024-04-26T03:39:05.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:05.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495391+0000", 2024-04-26T03:39:05.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8999999999999995e-07 2024-04-26T03:39:05.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:05.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495396+0000", 2024-04-26T03:39:05.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4360000000000001e-06 2024-04-26T03:39:05.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:05.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495410+0000", 2024-04-26T03:39:05.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3482000000000001e-05 2024-04-26T03:39:05.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:05.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495419+0000", 2024-04-26T03:39:05.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9600000000000006e-06 2024-04-26T03:39:05.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518418+0000", 2024-04-26T03:39:05.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022999116999999999 2024-04-26T03:39:05.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:05.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518517+0000", 2024-04-26T03:39:05.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8542999999999995e-05 2024-04-26T03:39:05.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:05.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519020+0000", 2024-04-26T03:39:05.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00050364099999999998 2024-04-26T03:39:05.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:05.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519837+0000", 2024-04-26T03:39:05.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00081666699999999998 2024-04-26T03:39:05.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:05.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.522936+0000", 2024-04-26T03:39:05.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030991170000000002 2024-04-26T03:39:05.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:05.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.523024+0000", 2024-04-26T03:39:05.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7684999999999998e-05 2024-04-26T03:39:05.865 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.865 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:05.865 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.865 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:33 3.cs0 3:3edefb40:::smithi09235267-11:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:05.865 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.495433+0000", 2024-04-26T03:39:05.865 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.334438287999999, 2024-04-26T03:39:05.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032718572000000001, 2024-04-26T03:39:05.865 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:05.865 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:05.866 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:05.866 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:05.866 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:05.866 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:05.866 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 33 2024-04-26T03:39:05.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.866 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:05.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:05.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495433+0000", 2024-04-26T03:39:05.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:05.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495433+0000", 2024-04-26T03:39:05.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:05.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495434+0000", 2024-04-26T03:39:05.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2300000000000001e-07 2024-04-26T03:39:05.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:05.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495437+0000", 2024-04-26T03:39:05.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2399999999999999e-06 2024-04-26T03:39:05.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:05.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495444+0000", 2024-04-26T03:39:05.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3869999999999997e-06 2024-04-26T03:39:05.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:05.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495450+0000", 2024-04-26T03:39:05.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.412e-06 2024-04-26T03:39:05.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.520715+0000", 2024-04-26T03:39:05.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.02526488 2024-04-26T03:39:05.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:05.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.520783+0000", 2024-04-26T03:39:05.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8553000000000006e-05 2024-04-26T03:39:05.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.523315+0000", 2024-04-26T03:39:05.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025314299999999999 2024-04-26T03:39:05.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:05.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.523384+0000", 2024-04-26T03:39:05.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9503000000000002e-05 2024-04-26T03:39:05.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:05.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.528152+0000", 2024-04-26T03:39:05.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047676439999999997 2024-04-26T03:39:05.870 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.870 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:05.870 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.870 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.cs0 35/32 ECSubReadReply(tid=5, attrs_read=0))", 2024-04-26T03:39:05.871 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.498822+0000", 2024-04-26T03:39:05.871 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.331049644, 2024-04-26T03:39:05.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021016639, 2024-04-26T03:39:05.871 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:05.871 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:05.871 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:05.871 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:05.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:05.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498822+0000", 2024-04-26T03:39:05.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:05.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498822+0000", 2024-04-26T03:39:05.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:05.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498822+0000", 2024-04-26T03:39:05.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3099999999999997e-07 2024-04-26T03:39:05.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:05.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498834+0000", 2024-04-26T03:39:05.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1331e-05 2024-04-26T03:39:05.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:05.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498878+0000", 2024-04-26T03:39:05.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4092999999999999e-05 2024-04-26T03:39:05.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:05.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498888+0000", 2024-04-26T03:39:05.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0049e-05 2024-04-26T03:39:05.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519785+0000", 2024-04-26T03:39:05.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020896656 2024-04-26T03:39:05.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:05.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519838+0000", 2024-04-26T03:39:05.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3878999999999997e-05 2024-04-26T03:39:05.874 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.874 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:05.874 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.875 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 35/32 ECSubWriteReply(tid=4, last_complete=35'3, committed=1, applied=1))", 2024-04-26T03:39:05.875 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.499205+0000", 2024-04-26T03:39:05.875 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.330666637, 2024-04-26T03:39:05.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020089540999999999, 2024-04-26T03:39:05.875 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:05.875 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:05.875 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:05.875 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:05.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:05.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499205+0000", 2024-04-26T03:39:05.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:05.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499205+0000", 2024-04-26T03:39:05.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:05.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499205+0000", 2024-04-26T03:39:05.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5199999999999996e-07 2024-04-26T03:39:05.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:05.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499210+0000", 2024-04-26T03:39:05.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0690000000000003e-06 2024-04-26T03:39:05.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:05.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499223+0000", 2024-04-26T03:39:05.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3783e-05 2024-04-26T03:39:05.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:05.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499230+0000", 2024-04-26T03:39:05.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7970000000000003e-06 2024-04-26T03:39:05.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519273+0000", 2024-04-26T03:39:05.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020043068000000001 2024-04-26T03:39:05.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:05.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519294+0000", 2024-04-26T03:39:05.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1172e-05 2024-04-26T03:39:05.879 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.879 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:05.879 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.883 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 35/32 ECSubWriteReply(tid=4, last_complete=35'3, committed=1, applied=1))", 2024-04-26T03:39:05.883 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.499957+0000", 2024-04-26T03:39:05.884 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.329914851, 2024-04-26T03:39:05.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019482105, 2024-04-26T03:39:05.884 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:05.884 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:05.884 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:05.884 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:05.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:05.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499957+0000", 2024-04-26T03:39:05.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:05.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499957+0000", 2024-04-26T03:39:05.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:05.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499957+0000", 2024-04-26T03:39:05.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8700000000000005e-07 2024-04-26T03:39:05.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:05.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499962+0000", 2024-04-26T03:39:05.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8389999999999997e-06 2024-04-26T03:39:05.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:05.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499977+0000", 2024-04-26T03:39:05.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5126999999999999e-05 2024-04-26T03:39:05.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:05.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499986+0000", 2024-04-26T03:39:05.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9439999999999994e-06 2024-04-26T03:39:05.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519421+0000", 2024-04-26T03:39:05.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019434297999999999 2024-04-26T03:39:05.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:05.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519439+0000", 2024-04-26T03:39:05.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8110000000000001e-05 2024-04-26T03:39:05.892 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.893 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:05.893 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.894 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 35/32 ECSubWriteReply(tid=4, last_complete=35'3, committed=1, applied=1))", 2024-04-26T03:39:05.894 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.500086+0000", 2024-04-26T03:39:05.894 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.329785411, 2024-04-26T03:39:05.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019299798, 2024-04-26T03:39:05.895 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:05.895 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:05.896 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:05.896 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:05.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:05.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500086+0000", 2024-04-26T03:39:05.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:05.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500086+0000", 2024-04-26T03:39:05.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:05.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500087+0000", 2024-04-26T03:39:05.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8599999999999998e-07 2024-04-26T03:39:05.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:05.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500092+0000", 2024-04-26T03:39:05.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8060000000000002e-06 2024-04-26T03:39:05.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:05.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500110+0000", 2024-04-26T03:39:05.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8099999999999999e-05 2024-04-26T03:39:05.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:05.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500117+0000", 2024-04-26T03:39:05.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0520000000000004e-06 2024-04-26T03:39:05.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519370+0000", 2024-04-26T03:39:05.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019253605 2024-04-26T03:39:05.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:05.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519386+0000", 2024-04-26T03:39:05.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5549e-05 2024-04-26T03:39:05.901 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.901 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:05.901 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.902 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 35/32 ECSubWriteReply(tid=4, last_complete=35'3, committed=1, applied=1))", 2024-04-26T03:39:05.902 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.500800+0000", 2024-04-26T03:39:05.902 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.329071029, 2024-04-26T03:39:05.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018893541, 2024-04-26T03:39:05.903 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:05.903 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:05.903 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:05.903 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:05.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:05.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500800+0000", 2024-04-26T03:39:05.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:05.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500800+0000", 2024-04-26T03:39:05.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:05.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500801+0000", 2024-04-26T03:39:05.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9900000000000002e-07 2024-04-26T03:39:05.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:05.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500804+0000", 2024-04-26T03:39:05.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1350000000000001e-06 2024-04-26T03:39:05.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:05.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500834+0000", 2024-04-26T03:39:05.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0414999999999999e-05 2024-04-26T03:39:05.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:05.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500840+0000", 2024-04-26T03:39:05.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7620000000000001e-06 2024-04-26T03:39:05.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519523+0000", 2024-04-26T03:39:05.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018682450999999999 2024-04-26T03:39:05.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:05.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519694+0000", 2024-04-26T03:39:05.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017147899999999999 2024-04-26T03:39:05.909 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.909 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:05.909 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.910 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.cs0 35/32 ECSubReadReply(tid=5, attrs_read=0))", 2024-04-26T03:39:05.910 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.500849+0000", 2024-04-26T03:39:05.910 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.329022788, 2024-04-26T03:39:05.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019810352, 2024-04-26T03:39:05.910 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:05.911 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:05.911 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:05.911 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:05.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:05.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500849+0000", 2024-04-26T03:39:05.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:05.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500849+0000", 2024-04-26T03:39:05.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:05.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500849+0000", 2024-04-26T03:39:05.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9999999999999999e-07 2024-04-26T03:39:05.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:05.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500860+0000", 2024-04-26T03:39:05.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0811e-05 2024-04-26T03:39:05.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:05.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500867+0000", 2024-04-26T03:39:05.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9670000000000004e-06 2024-04-26T03:39:05.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:05.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500871+0000", 2024-04-26T03:39:05.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3109999999999999e-06 2024-04-26T03:39:05.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.520619+0000", 2024-04-26T03:39:05.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019747602 2024-04-26T03:39:05.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:05.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.520659+0000", 2024-04-26T03:39:05.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0361000000000002e-05 2024-04-26T03:39:05.916 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.916 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:05.916 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.916 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.cs0 35/32 ECSubReadReply(tid=5, attrs_read=0))", 2024-04-26T03:39:05.916 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.501131+0000", 2024-04-26T03:39:05.916 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.328740309000001, 2024-04-26T03:39:05.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018837000999999999, 2024-04-26T03:39:05.917 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:05.917 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:05.917 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:05.917 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:05.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:05.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501131+0000", 2024-04-26T03:39:05.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:05.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501131+0000", 2024-04-26T03:39:05.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:05.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501132+0000", 2024-04-26T03:39:05.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8199999999999999e-07 2024-04-26T03:39:05.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:05.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501140+0000", 2024-04-26T03:39:05.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5900000000000008e-06 2024-04-26T03:39:05.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:05.919 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501150+0000", 2024-04-26T03:39:05.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5189999999999998e-06 2024-04-26T03:39:05.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.919 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:05.919 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501157+0000", 2024-04-26T03:39:05.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0550000000000002e-06 2024-04-26T03:39:05.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.919 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.919 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519890+0000", 2024-04-26T03:39:05.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018732954999999999 2024-04-26T03:39:05.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:05.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519968+0000", 2024-04-26T03:39:05.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8200000000000003e-05 2024-04-26T03:39:05.920 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.920 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:05.920 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.920 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 35/32 ECSubWriteReply(tid=4, last_complete=35'3, committed=1, applied=1))", 2024-04-26T03:39:05.920 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.502386+0000", 2024-04-26T03:39:05.920 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.327485326, 2024-04-26T03:39:05.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016963842, 2024-04-26T03:39:05.920 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:05.921 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:05.921 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:05.921 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:05.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:05.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.502386+0000", 2024-04-26T03:39:05.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:05.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.502386+0000", 2024-04-26T03:39:05.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:05.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.502387+0000", 2024-04-26T03:39:05.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3900000000000005e-07 2024-04-26T03:39:05.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:05.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.502391+0000", 2024-04-26T03:39:05.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3490000000000002e-06 2024-04-26T03:39:05.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:05.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.502400+0000", 2024-04-26T03:39:05.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3190000000000001e-06 2024-04-26T03:39:05.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:05.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.502407+0000", 2024-04-26T03:39:05.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2659999999999998e-06 2024-04-26T03:39:05.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519331+0000", 2024-04-26T03:39:05.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016924640000000001 2024-04-26T03:39:05.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:05.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519350+0000", 2024-04-26T03:39:05.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8729e-05 2024-04-26T03:39:05.923 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.924 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:05.924 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.924 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 35/32 ECSubWriteReply(tid=4, last_complete=35'3, committed=1, applied=1))", 2024-04-26T03:39:05.924 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.503293+0000", 2024-04-26T03:39:05.924 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.326578591000001, 2024-04-26T03:39:05.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016207038999999999, 2024-04-26T03:39:05.924 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:05.924 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:05.924 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:05.924 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:05.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:05.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.503293+0000", 2024-04-26T03:39:05.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:05.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.503293+0000", 2024-04-26T03:39:05.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:05.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.503293+0000", 2024-04-26T03:39:05.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9099999999999999e-07 2024-04-26T03:39:05.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:05.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.503296+0000", 2024-04-26T03:39:05.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8660000000000002e-06 2024-04-26T03:39:05.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:05.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.503304+0000", 2024-04-26T03:39:05.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1799999999999996e-06 2024-04-26T03:39:05.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:05.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.503310+0000", 2024-04-26T03:39:05.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0410000000000003e-06 2024-04-26T03:39:05.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519482+0000", 2024-04-26T03:39:05.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0161712 2024-04-26T03:39:05.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:05.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519500+0000", 2024-04-26T03:39:05.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8360999999999999e-05 2024-04-26T03:39:05.927 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.927 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:05.927 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.928 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:325 3.fs0 3:ff04ba47:::smithi09235267-9:head [append 0~507904] snapc 7=[7,6,5] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-26T03:39:05.928 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:01.552318+0000", 2024-04-26T03:39:05.928 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.2775535690000002, 2024-04-26T03:39:05.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017795359, 2024-04-26T03:39:05.928 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:05.928 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:05.928 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:05.928 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:05.928 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:05.929 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:05.929 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 325 2024-04-26T03:39:05.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.929 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:05.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:05.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.552318+0000", 2024-04-26T03:39:05.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:05.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.552318+0000", 2024-04-26T03:39:05.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:05.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.552320+0000", 2024-04-26T03:39:05.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.323e-06 2024-04-26T03:39:05.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:05.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.552673+0000", 2024-04-26T03:39:05.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00035284100000000001 2024-04-26T03:39:05.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:05.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.552690+0000", 2024-04-26T03:39:05.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6917000000000001e-05 2024-04-26T03:39:05.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:05.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.552699+0000", 2024-04-26T03:39:05.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4020000000000008e-06 2024-04-26T03:39:05.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.552764+0000", 2024-04-26T03:39:05.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4564000000000001e-05 2024-04-26T03:39:05.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:05.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.552884+0000", 2024-04-26T03:39:05.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000119608 2024-04-26T03:39:05.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:05.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.554098+0000", 2024-04-26T03:39:05.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001214686 2024-04-26T03:39:05.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:05.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.565486+0000", 2024-04-26T03:39:05.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011387670000000001 2024-04-26T03:39:05.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:05.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.569852+0000", 2024-04-26T03:39:05.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043656659999999998 2024-04-26T03:39:05.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:05.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.570113+0000", 2024-04-26T03:39:05.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026168200000000001 2024-04-26T03:39:05.934 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.934 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:05.934 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.934 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:326 3.fs0 3:ff04ba47:::smithi09235267-9:head [append 0~507904] snapc 7=[7,6,5] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-26T03:39:05.934 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:01.555720+0000", 2024-04-26T03:39:05.934 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.2741512479999999, 2024-04-26T03:39:05.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015601677, 2024-04-26T03:39:05.934 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:05.934 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:05.934 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:05.935 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:05.935 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:05.935 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:05.935 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 326 2024-04-26T03:39:05.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.935 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:05.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:05.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.555720+0000", 2024-04-26T03:39:05.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:05.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.555720+0000", 2024-04-26T03:39:05.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:05.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.555722+0000", 2024-04-26T03:39:05.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.12e-06 2024-04-26T03:39:05.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:05.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.556078+0000", 2024-04-26T03:39:05.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00035606800000000002 2024-04-26T03:39:05.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:05.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.556095+0000", 2024-04-26T03:39:05.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6464999999999999e-05 2024-04-26T03:39:05.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:05.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.556104+0000", 2024-04-26T03:39:05.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4059999999999994e-06 2024-04-26T03:39:05.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.559802+0000", 2024-04-26T03:39:05.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0036973349999999999 2024-04-26T03:39:05.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:05.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.559924+0000", 2024-04-26T03:39:05.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000122667 2024-04-26T03:39:05.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:05.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.560967+0000", 2024-04-26T03:39:05.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010431710000000001 2024-04-26T03:39:05.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:05.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.570537+0000", 2024-04-26T03:39:05.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0095695669999999993 2024-04-26T03:39:05.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:05.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.571246+0000", 2024-04-26T03:39:05.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00070930699999999995 2024-04-26T03:39:05.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:05.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.571322+0000", 2024-04-26T03:39:05.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5570999999999999e-05 2024-04-26T03:39:05.939 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.939 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:05.939 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.940 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:328 3.fs0 3:ff04ba47:::smithi09235267-9:head [setxattr _header (55) in=7b] snapc 7=[7,6,5] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-26T03:39:05.940 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:01.558148+0000", 2024-04-26T03:39:05.940 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.2717233190000004, 2024-04-26T03:39:05.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014927532, 2024-04-26T03:39:05.940 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:05.940 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:05.940 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:05.940 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:05.940 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:05.940 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:05.940 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 328 2024-04-26T03:39:05.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.940 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:05.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:05.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.558148+0000", 2024-04-26T03:39:05.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:05.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.558148+0000", 2024-04-26T03:39:05.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:05.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.558149+0000", 2024-04-26T03:39:05.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4399999999999999e-07 2024-04-26T03:39:05.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:05.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.558153+0000", 2024-04-26T03:39:05.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1840000000000001e-06 2024-04-26T03:39:05.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:05.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.558161+0000", 2024-04-26T03:39:05.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5610000000000001e-06 2024-04-26T03:39:05.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:05.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.558166+0000", 2024-04-26T03:39:05.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2089999999999996e-06 2024-04-26T03:39:05.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.568599+0000", 2024-04-26T03:39:05.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010433530999999999 2024-04-26T03:39:05.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:05.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.568699+0000", 2024-04-26T03:39:05.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9147999999999993e-05 2024-04-26T03:39:05.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:05.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.569217+0000", 2024-04-26T03:39:05.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00051868699999999999 2024-04-26T03:39:05.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:05.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.570563+0000", 2024-04-26T03:39:05.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013459909999999999 2024-04-26T03:39:05.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:05.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.573018+0000", 2024-04-26T03:39:05.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024549429999999998 2024-04-26T03:39:05.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:05.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.573076+0000", 2024-04-26T03:39:05.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7433999999999999e-05 2024-04-26T03:39:05.945 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.945 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:05.945 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.946 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:329 3.fs0 3:ff04ba47:::smithi09235267-9:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e45)", 2024-04-26T03:39:05.946 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:01.558174+0000", 2024-04-26T03:39:05.946 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.2716975320000001, 2024-04-26T03:39:05.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016607542999999999, 2024-04-26T03:39:05.946 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:05.946 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:05.946 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:05.946 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:05.946 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:05.946 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:05.946 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 329 2024-04-26T03:39:05.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.947 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:05.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:05.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.558174+0000", 2024-04-26T03:39:05.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:05.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.558174+0000", 2024-04-26T03:39:05.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:05.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.558174+0000", 2024-04-26T03:39:05.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5199999999999998e-07 2024-04-26T03:39:05.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:05.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.558177+0000", 2024-04-26T03:39:05.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.227e-06 2024-04-26T03:39:05.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:05.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.558184+0000", 2024-04-26T03:39:05.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.272e-06 2024-04-26T03:39:05.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:05.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.558189+0000", 2024-04-26T03:39:05.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9099999999999996e-06 2024-04-26T03:39:05.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.570250+0000", 2024-04-26T03:39:05.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012060998999999999 2024-04-26T03:39:05.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:05.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.570338+0000", 2024-04-26T03:39:05.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8126000000000003e-05 2024-04-26T03:39:05.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.573355+0000", 2024-04-26T03:39:05.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003017619 2024-04-26T03:39:05.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:05.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.573452+0000", 2024-04-26T03:39:05.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7004999999999997e-05 2024-04-26T03:39:05.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:05.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.574781+0000", 2024-04-26T03:39:05.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013290330000000001 2024-04-26T03:39:05.950 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.951 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:05.951 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.951 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:342 3.cs0 3:3edefb40:::smithi09235267-11:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e45)", 2024-04-26T03:39:05.951 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:01.574316+0000", 2024-04-26T03:39:05.951 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.2555554630000003, 2024-04-26T03:39:05.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015336551, 2024-04-26T03:39:05.951 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:05.951 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:05.951 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:05.951 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:05.951 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:05.952 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:05.952 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 342 2024-04-26T03:39:05.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.952 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:05.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:05.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.574316+0000", 2024-04-26T03:39:05.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:05.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.574316+0000", 2024-04-26T03:39:05.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:05.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:05.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.574316+0000", 2024-04-26T03:39:05.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5999999999999999e-07 2024-04-26T03:39:05.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:05.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.574319+0000", 2024-04-26T03:39:05.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.553e-06 2024-04-26T03:39:05.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:05.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.574326+0000", 2024-04-26T03:39:05.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5300000000000002e-06 2024-04-26T03:39:05.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:05.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.574331+0000", 2024-04-26T03:39:05.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1000000000000003e-06 2024-04-26T03:39:05.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.586272+0000", 2024-04-26T03:39:05.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011941422 2024-04-26T03:39:05.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:05.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.586338+0000", 2024-04-26T03:39:05.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6236999999999996e-05 2024-04-26T03:39:05.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:05.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.588757+0000", 2024-04-26T03:39:05.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002418262 2024-04-26T03:39:05.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:05.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.588802+0000", 2024-04-26T03:39:05.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5497999999999997e-05 2024-04-26T03:39:05.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:05.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:05.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:05.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.589653+0000", 2024-04-26T03:39:05.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00085048899999999995 2024-04-26T03:39:05.956 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.956 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:05.956 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.956 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:05.956 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:05.956 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:05.958 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-04-26T03:39:05.960 INFO:tasks.rados.rados.0.smithi092.stdout:184: finishing write tid 1 to smithi09235267-44 2024-04-26T03:39:05.960 INFO:tasks.rados.rados.0.smithi092.stdout:184: finishing write tid 2 to smithi09235267-44 2024-04-26T03:39:05.960 INFO:tasks.rados.rados.0.smithi092.stdout:184: finishing write tid 3 to smithi09235267-44 2024-04-26T03:39:05.960 INFO:tasks.rados.rados.0.smithi092.stdout:184: finishing write tid 4 to smithi09235267-44 2024-04-26T03:39:05.960 INFO:tasks.rados.rados.0.smithi092.stdout:184: finishing write tid 5 to smithi09235267-44 2024-04-26T03:39:05.960 INFO:tasks.rados.rados.0.smithi092.stdout:184: oid 44 updating version 0 to 41 2024-04-26T03:39:05.960 INFO:tasks.rados.rados.0.smithi092.stdout:184: oid 44 updating version 41 to 43 2024-04-26T03:39:05.960 INFO:tasks.rados.rados.0.smithi092.stdout:184: oid 44 version 43 is already newer than 42 2024-04-26T03:39:05.960 INFO:tasks.rados.rados.0.smithi092.stdout:184: oid 44 updating version 43 to 44 2024-04-26T03:39:05.960 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 44 v 44 (ObjNum 92 snap 7 seq_num 92) dirty exists 2024-04-26T03:39:05.960 INFO:tasks.rados.rados.0.smithi092.stdout:184: left oid 44 (ObjNum 92 snap 7 seq_num 92) 2024-04-26T03:39:05.961 INFO:tasks.rados.rados.0.smithi092.stdout:185: seq_num 93 ranges {540672=802816,1343488=638976,1982464=131072} 2024-04-26T03:39:05.961 INFO:tasks.rados.rados.0.smithi092.stdout:185: writing smithi09235267-47 from 540672 to 1343488 tid 1 2024-04-26T03:39:05.961 INFO:tasks.rados.rados.0.smithi092.stdout:185: writing smithi09235267-47 from 1343488 to 1982464 tid 2 2024-04-26T03:39:05.961 INFO:tasks.rados.rados.0.smithi092.stdout:185: writing smithi09235267-47 from 1982464 to 2113536 tid 3 2024-04-26T03:39:05.961 INFO:tasks.rados.rados.0.smithi092.stdout:182: done (3 left) 2024-04-26T03:39:05.961 INFO:tasks.rados.rados.0.smithi092.stdout:183: done (2 left) 2024-04-26T03:39:05.961 INFO:tasks.rados.rados.0.smithi092.stdout:184: done (1 left) 2024-04-26T03:39:05.961 INFO:tasks.rados.rados.0.smithi092.stdout:186: snap_create 2024-04-26T03:39:05.961 INFO:tasks.rados.rados.0.smithi092.stdout:185: finishing write tid 1 to smithi09235267-47 2024-04-26T03:39:05.961 INFO:tasks.rados.rados.0.smithi092.stdout:185: finishing write tid 2 to smithi09235267-47 2024-04-26T03:39:05.961 INFO:tasks.rados.rados.0.smithi092.stdout:185: finishing write tid 3 to smithi09235267-47 2024-04-26T03:39:05.961 INFO:tasks.rados.rados.0.smithi092.stdout:185: finishing write tid 4 to smithi09235267-47 2024-04-26T03:39:05.961 INFO:tasks.rados.rados.0.smithi092.stdout:185: finishing write tid 5 to smithi09235267-47 2024-04-26T03:39:05.962 INFO:tasks.rados.rados.0.smithi092.stdout:185: oid 47 updating version 0 to 30 2024-04-26T03:39:05.962 INFO:tasks.rados.rados.0.smithi092.stdout:185: oid 47 version 30 is already newer than 29 2024-04-26T03:39:05.962 INFO:tasks.rados.rados.0.smithi092.stdout:185: oid 47 version 30 is already newer than 28 2024-04-26T03:39:05.962 INFO:tasks.rados.rados.0.smithi092.stdout:185: oid 47 updating version 30 to 31 2024-04-26T03:39:05.962 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 47 v 31 (ObjNum 93 snap 7 seq_num 93) dirty exists 2024-04-26T03:39:05.962 INFO:tasks.rados.rados.0.smithi092.stdout:185: left oid 47 (ObjNum 93 snap 7 seq_num 93) 2024-04-26T03:39:05.962 INFO:tasks.ceph.osd.14.smithi066.stderr:2024-04-26T03:39:05.956+0000 7f5395f08640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 31765) UID: 0 2024-04-26T03:39:06.058 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:39:06.057+0000 7fba0d2ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31961) UID: 0 2024-04-26T03:39:06.102 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:06.102 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-26T03:39:06.102 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-26T03:39:06.102 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:06.111 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-04-26T03:39:06.160 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:39:06.158+0000 7fba0d2ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31961) UID: 0 2024-04-26T03:39:06.216 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:06.216 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-26T03:39:06.216 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-26T03:39:06.216 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-26T03:39:06.216 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:06.225 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-04-26T03:39:06.261 INFO:tasks.ceph.osd.10.smithi066.stderr:2024-04-26T03:39:06.260+0000 7f130fcc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 31763) UID: 0 2024-04-26T03:39:06.330 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:06.330 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-26T03:39:06.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-26T03:39:06.330 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-26T03:39:06.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.331 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:30 3.5s0 3:a64abe59:::smithi09235267-10:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:06.331 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.491045+0000", 2024-04-26T03:39:06.331 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.836631182, 2024-04-26T03:39:06.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.024868352, 2024-04-26T03:39:06.331 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:06.331 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:06.331 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:06.331 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:06.331 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:06.331 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:06.331 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 30 2024-04-26T03:39:06.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.332 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:06.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:06.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.491045+0000", 2024-04-26T03:39:06.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:06.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:06.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.491045+0000", 2024-04-26T03:39:06.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:06.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:06.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.491045+0000", 2024-04-26T03:39:06.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3799999999999997e-07 2024-04-26T03:39:06.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:06.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.491049+0000", 2024-04-26T03:39:06.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3359999999999999e-06 2024-04-26T03:39:06.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:06.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.491056+0000", 2024-04-26T03:39:06.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7549999999999994e-06 2024-04-26T03:39:06.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:06.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.491062+0000", 2024-04-26T03:39:06.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3730000000000001e-06 2024-04-26T03:39:06.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:06.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.493431+0000", 2024-04-26T03:39:06.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023688490000000001 2024-04-26T03:39:06.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:06.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.493515+0000", 2024-04-26T03:39:06.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4342e-05 2024-04-26T03:39:06.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:06.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499490+0000", 2024-04-26T03:39:06.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059748809999999996 2024-04-26T03:39:06.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:06.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499562+0000", 2024-04-26T03:39:06.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1942999999999999e-05 2024-04-26T03:39:06.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:06.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515913+0000", 2024-04-26T03:39:06.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016351335000000002 2024-04-26T03:39:06.337 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:06.337 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:06.337 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:06.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.338 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.5s0 35/32 ECSubReadReply(tid=3, attrs_read=0))", 2024-04-26T03:39:06.338 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.500548+0000", 2024-04-26T03:39:06.338 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.827127974, 2024-04-26T03:39:06.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014243212999999999, 2024-04-26T03:39:06.338 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:06.338 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:06.338 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:06.339 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:06.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:06.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500548+0000", 2024-04-26T03:39:06.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:06.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:06.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500548+0000", 2024-04-26T03:39:06.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:06.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:06.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500549+0000", 2024-04-26T03:39:06.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3300000000000002e-07 2024-04-26T03:39:06.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:06.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500560+0000", 2024-04-26T03:39:06.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1316e-05 2024-04-26T03:39:06.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:06.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500573+0000", 2024-04-26T03:39:06.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2853999999999999e-05 2024-04-26T03:39:06.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:06.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500579+0000", 2024-04-26T03:39:06.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3090000000000001e-06 2024-04-26T03:39:06.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:06.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.514618+0000", 2024-04-26T03:39:06.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014038861 2024-04-26T03:39:06.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:06.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.514791+0000", 2024-04-26T03:39:06.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017323999999999999 2024-04-26T03:39:06.342 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:06.342 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:06.342 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:06.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.342 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.5s0 35/32 ECSubReadReply(tid=3, attrs_read=0))", 2024-04-26T03:39:06.342 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.500609+0000", 2024-04-26T03:39:06.342 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.827066822999999, 2024-04-26T03:39:06.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014706643, 2024-04-26T03:39:06.343 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:06.343 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:06.343 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:06.344 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:06.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:06.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500609+0000", 2024-04-26T03:39:06.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:06.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:06.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500609+0000", 2024-04-26T03:39:06.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:06.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:06.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500610+0000", 2024-04-26T03:39:06.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1900000000000002e-07 2024-04-26T03:39:06.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:06.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500621+0000", 2024-04-26T03:39:06.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1102e-05 2024-04-26T03:39:06.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:06.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500631+0000", 2024-04-26T03:39:06.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0197000000000001e-05 2024-04-26T03:39:06.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:06.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500638+0000", 2024-04-26T03:39:06.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3770000000000001e-06 2024-04-26T03:39:06.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:06.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515285+0000", 2024-04-26T03:39:06.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014647004 2024-04-26T03:39:06.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:06.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515316+0000", 2024-04-26T03:39:06.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1244000000000001e-05 2024-04-26T03:39:06.347 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:06.347 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:06.347 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:06.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.347 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 35/32 ECSubWriteReply(tid=2, last_complete=34'2, committed=1, applied=1))", 2024-04-26T03:39:06.348 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.501067+0000", 2024-04-26T03:39:06.348 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.826609375, 2024-04-26T03:39:06.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013961199000000001, 2024-04-26T03:39:06.348 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:06.348 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:06.348 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:06.348 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:06.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:06.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501067+0000", 2024-04-26T03:39:06.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:06.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:06.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501067+0000", 2024-04-26T03:39:06.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:06.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:06.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501067+0000", 2024-04-26T03:39:06.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7599999999999996e-07 2024-04-26T03:39:06.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:06.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501072+0000", 2024-04-26T03:39:06.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7700000000000001e-06 2024-04-26T03:39:06.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:06.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501083+0000", 2024-04-26T03:39:06.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0652000000000001e-05 2024-04-26T03:39:06.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:06.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501090+0000", 2024-04-26T03:39:06.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5299999999999999e-06 2024-04-26T03:39:06.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:06.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515013+0000", 2024-04-26T03:39:06.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013922436999999999 2024-04-26T03:39:06.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:06.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515028+0000", 2024-04-26T03:39:06.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5034e-05 2024-04-26T03:39:06.354 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:06.354 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:06.354 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:06.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.355 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.5s0 35/32 ECSubReadReply(tid=3, attrs_read=0))", 2024-04-26T03:39:06.355 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.501907+0000", 2024-04-26T03:39:06.355 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.825768558, 2024-04-26T03:39:06.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013462257, 2024-04-26T03:39:06.355 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:06.355 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:06.355 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:06.355 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:06.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:06.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501907+0000", 2024-04-26T03:39:06.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:06.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.356 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:06.356 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501907+0000", 2024-04-26T03:39:06.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:06.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.356 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:06.356 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501908+0000", 2024-04-26T03:39:06.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2200000000000003e-07 2024-04-26T03:39:06.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:06.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501918+0000", 2024-04-26T03:39:06.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0329999999999999e-05 2024-04-26T03:39:06.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:06.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501945+0000", 2024-04-26T03:39:06.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6047999999999999e-05 2024-04-26T03:39:06.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:06.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501951+0000", 2024-04-26T03:39:06.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7560000000000002e-06 2024-04-26T03:39:06.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:06.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515339+0000", 2024-04-26T03:39:06.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013387731 2024-04-26T03:39:06.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:06.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515370+0000", 2024-04-26T03:39:06.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0670000000000003e-05 2024-04-26T03:39:06.359 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:06.359 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:06.359 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:06.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.359 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 35/32 ECSubWriteReply(tid=2, last_complete=34'2, committed=1, applied=1))", 2024-04-26T03:39:06.359 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.501982+0000", 2024-04-26T03:39:06.359 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.825693722, 2024-04-26T03:39:06.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013281477999999999, 2024-04-26T03:39:06.359 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:06.359 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:06.359 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:06.359 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:06.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:06.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501982+0000", 2024-04-26T03:39:06.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:06.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:06.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501982+0000", 2024-04-26T03:39:06.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:06.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:06.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501983+0000", 2024-04-26T03:39:06.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4900000000000001e-07 2024-04-26T03:39:06.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:06.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501987+0000", 2024-04-26T03:39:06.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1030000000000004e-06 2024-04-26T03:39:06.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:06.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501995+0000", 2024-04-26T03:39:06.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4859999999999997e-06 2024-04-26T03:39:06.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:06.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.502002+0000", 2024-04-26T03:39:06.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3400000000000003e-06 2024-04-26T03:39:06.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:06.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515231+0000", 2024-04-26T03:39:06.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013229757 2024-04-26T03:39:06.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:06.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515264+0000", 2024-04-26T03:39:06.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2443000000000001e-05 2024-04-26T03:39:06.362 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:06.362 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:06.362 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:06.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.363 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:67 3.5s0 3:a79a63ac:::smithi09235267-23:head [append 0~376832,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:06.363 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.518499+0000", 2024-04-26T03:39:06.363 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.809176681, 2024-04-26T03:39:06.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01578007, 2024-04-26T03:39:06.363 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:06.363 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:06.363 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:06.363 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:06.363 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:06.363 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:06.363 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 67 2024-04-26T03:39:06.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.364 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:06.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:06.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518499+0000", 2024-04-26T03:39:06.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:06.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:06.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518499+0000", 2024-04-26T03:39:06.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:06.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:06.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518501+0000", 2024-04-26T03:39:06.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9010000000000001e-06 2024-04-26T03:39:06.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:06.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518725+0000", 2024-04-26T03:39:06.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000224183 2024-04-26T03:39:06.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:06.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518740+0000", 2024-04-26T03:39:06.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4966e-05 2024-04-26T03:39:06.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:06.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518748+0000", 2024-04-26T03:39:06.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6399999999999997e-06 2024-04-26T03:39:06.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:06.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518798+0000", 2024-04-26T03:39:06.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0263000000000003e-05 2024-04-26T03:39:06.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:06.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518927+0000", 2024-04-26T03:39:06.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012898900000000001 2024-04-26T03:39:06.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:06.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519605+0000", 2024-04-26T03:39:06.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00067774899999999995 2024-04-26T03:39:06.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:06.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.524447+0000", 2024-04-26T03:39:06.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0048424319999999998 2024-04-26T03:39:06.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:06.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.534195+0000", 2024-04-26T03:39:06.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097473149999999995 2024-04-26T03:39:06.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:06.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.534279+0000", 2024-04-26T03:39:06.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4631999999999999e-05 2024-04-26T03:39:06.369 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:06.369 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:06.369 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:06.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.369 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:68 3.5s0 3:a79a63ac:::smithi09235267-23:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:06.369 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.518766+0000", 2024-04-26T03:39:06.369 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.808909760000001, 2024-04-26T03:39:06.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015709772, 2024-04-26T03:39:06.370 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:06.370 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:06.370 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:06.370 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:06.370 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:06.370 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:06.370 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 68 2024-04-26T03:39:06.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.370 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:06.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:06.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518766+0000", 2024-04-26T03:39:06.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:06.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:06.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518766+0000", 2024-04-26T03:39:06.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:06.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:06.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518767+0000", 2024-04-26T03:39:06.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6899999999999997e-07 2024-04-26T03:39:06.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:06.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518771+0000", 2024-04-26T03:39:06.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-06 2024-04-26T03:39:06.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:06.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518777+0000", 2024-04-26T03:39:06.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3600000000000004e-06 2024-04-26T03:39:06.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:06.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518783+0000", 2024-04-26T03:39:06.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9490000000000001e-06 2024-04-26T03:39:06.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:06.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.522888+0000", 2024-04-26T03:39:06.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004105577 2024-04-26T03:39:06.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:06.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.522976+0000", 2024-04-26T03:39:06.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400999999999999e-05 2024-04-26T03:39:06.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:06.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.523380+0000", 2024-04-26T03:39:06.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00040403599999999998 2024-04-26T03:39:06.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:06.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.524449+0000", 2024-04-26T03:39:06.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00106897 2024-04-26T03:39:06.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:06.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.534390+0000", 2024-04-26T03:39:06.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099415249999999997 2024-04-26T03:39:06.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:06.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.534476+0000", 2024-04-26T03:39:06.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5884999999999995e-05 2024-04-26T03:39:06.374 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:06.374 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:06.375 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:06.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.375 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:69 3.5s0 3:a79a63ac:::smithi09235267-23:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:06.375 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.518790+0000", 2024-04-26T03:39:06.375 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.808885527999999, 2024-04-26T03:39:06.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018453847999999998, 2024-04-26T03:39:06.375 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:06.375 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:06.375 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:06.375 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:06.375 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:06.375 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:06.375 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 69 2024-04-26T03:39:06.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.376 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:06.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:06.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518790+0000", 2024-04-26T03:39:06.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:06.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:06.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518790+0000", 2024-04-26T03:39:06.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:06.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:06.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518791+0000", 2024-04-26T03:39:06.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9999999999999998e-07 2024-04-26T03:39:06.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:06.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518793+0000", 2024-04-26T03:39:06.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0949999999999998e-06 2024-04-26T03:39:06.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:06.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518799+0000", 2024-04-26T03:39:06.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.117e-06 2024-04-26T03:39:06.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:06.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518804+0000", 2024-04-26T03:39:06.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8670000000000002e-06 2024-04-26T03:39:06.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:06.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.523575+0000", 2024-04-26T03:39:06.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004771155 2024-04-26T03:39:06.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:06.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.523646+0000", 2024-04-26T03:39:06.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0992000000000001e-05 2024-04-26T03:39:06.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:06.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.534812+0000", 2024-04-26T03:39:06.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011166099000000001 2024-04-26T03:39:06.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:06.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.534887+0000", 2024-04-26T03:39:06.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5094999999999998e-05 2024-04-26T03:39:06.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:06.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.537244+0000", 2024-04-26T03:39:06.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023569279999999999 2024-04-26T03:39:06.381 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:06.381 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:06.381 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:06.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.382 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.3s3 35/32 ECSubWrite(tid=4, reqid=client.4556.0:88, at_version=35'3, trim_to=0'0, roll_forward_to=34'2))", 2024-04-26T03:39:06.382 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.545346+0000", 2024-04-26T03:39:06.382 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.782329513000001, 2024-04-26T03:39:06.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013204087999999999, 2024-04-26T03:39:06.382 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:06.382 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:06.382 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:06.382 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:06.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:06.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.545346+0000", 2024-04-26T03:39:06.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:06.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:06.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.545346+0000", 2024-04-26T03:39:06.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:06.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:06.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.545348+0000", 2024-04-26T03:39:06.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.415e-06 2024-04-26T03:39:06.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:06.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.545798+0000", 2024-04-26T03:39:06.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00045054699999999998 2024-04-26T03:39:06.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:06.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.545811+0000", 2024-04-26T03:39:06.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2911000000000001e-05 2024-04-26T03:39:06.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:06.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.545820+0000", 2024-04-26T03:39:06.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.259e-06 2024-04-26T03:39:06.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:06.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.545884+0000", 2024-04-26T03:39:06.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4820999999999995e-05 2024-04-26T03:39:06.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:06.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.545893+0000", 2024-04-26T03:39:06.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9069999999999996e-06 2024-04-26T03:39:06.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:06.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.558498+0000", 2024-04-26T03:39:06.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012604667999999999 2024-04-26T03:39:06.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:06.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.558551+0000", 2024-04-26T03:39:06.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2559999999999998e-05 2024-04-26T03:39:06.386 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:06.386 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:06.386 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:06.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.386 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.2s4 35/32 ECSubWrite(tid=8, reqid=client.4556.0:139, at_version=35'5, trim_to=0'0, roll_forward_to=35'4))", 2024-04-26T03:39:06.386 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.621355+0000", 2024-04-26T03:39:06.386 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.706320899, 2024-04-26T03:39:06.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013204417, 2024-04-26T03:39:06.386 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:06.386 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:06.386 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:06.387 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:06.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:06.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.621355+0000", 2024-04-26T03:39:06.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:06.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:06.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.621355+0000", 2024-04-26T03:39:06.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:06.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:06.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.621357+0000", 2024-04-26T03:39:06.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.477e-06 2024-04-26T03:39:06.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:06.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.621680+0000", 2024-04-26T03:39:06.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00032331299999999999 2024-04-26T03:39:06.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:06.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.621698+0000", 2024-04-26T03:39:06.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8493000000000001e-05 2024-04-26T03:39:06.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:06.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.621710+0000", 2024-04-26T03:39:06.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1965e-05 2024-04-26T03:39:06.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:06.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.621790+0000", 2024-04-26T03:39:06.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9775000000000001e-05 2024-04-26T03:39:06.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:06.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.621799+0000", 2024-04-26T03:39:06.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.183e-06 2024-04-26T03:39:06.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:06.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.634425+0000", 2024-04-26T03:39:06.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012625945 2024-04-26T03:39:06.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:06.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.634559+0000", 2024-04-26T03:39:06.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013426599999999999 2024-04-26T03:39:06.392 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:06.392 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:06.392 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:06.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.392 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:256 3.5s0 3:a64abe59:::smithi09235267-10:head [append 0~0] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T03:39:06.392 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:56.508641+0000", 2024-04-26T03:39:06.392 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.8190350570000007, 2024-04-26T03:39:06.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013570576000000001, 2024-04-26T03:39:06.392 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:06.392 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:06.393 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:06.393 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:06.393 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:06.393 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:06.393 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 256 2024-04-26T03:39:06.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.393 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:06.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:06.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508641+0000", 2024-04-26T03:39:06.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:06.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:06.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508641+0000", 2024-04-26T03:39:06.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:06.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:06.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508644+0000", 2024-04-26T03:39:06.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7760000000000002e-06 2024-04-26T03:39:06.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:06.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508655+0000", 2024-04-26T03:39:06.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1049e-05 2024-04-26T03:39:06.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:06.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508727+0000", 2024-04-26T03:39:06.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1907000000000001e-05 2024-04-26T03:39:06.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:06.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508737+0000", 2024-04-26T03:39:06.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0691e-05 2024-04-26T03:39:06.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:06.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508889+0000", 2024-04-26T03:39:06.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015123099999999999 2024-04-26T03:39:06.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-04-26T03:39:06.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508905+0000", 2024-04-26T03:39:06.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6042999999999999e-05 2024-04-26T03:39:06.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:06.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.511938+0000", 2024-04-26T03:39:06.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030333119999999998 2024-04-26T03:39:06.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:06.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.512054+0000", 2024-04-26T03:39:06.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115742 2024-04-26T03:39:06.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:06.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.512811+0000", 2024-04-26T03:39:06.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00075763100000000004 2024-04-26T03:39:06.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:06.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.521838+0000", 2024-04-26T03:39:06.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0090266489999999994 2024-04-26T03:39:06.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:06.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.522144+0000", 2024-04-26T03:39:06.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030574600000000002 2024-04-26T03:39:06.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:06.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.522211+0000", 2024-04-26T03:39:06.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7799000000000006e-05 2024-04-26T03:39:06.398 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:06.398 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:06.398 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:06.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.399 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:257 3.5s0 3:a64abe59:::smithi09235267-10:head [rollback 2] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T03:39:06.399 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:56.508783+0000", 2024-04-26T03:39:06.399 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.8188930370000005, 2024-04-26T03:39:06.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014341474999999999, 2024-04-26T03:39:06.399 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:06.399 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:06.399 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:06.399 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:06.399 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:06.399 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:06.399 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 257 2024-04-26T03:39:06.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.400 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:06.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:06.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508783+0000", 2024-04-26T03:39:06.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:06.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:06.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508783+0000", 2024-04-26T03:39:06.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:06.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:06.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508784+0000", 2024-04-26T03:39:06.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9999999999999995e-07 2024-04-26T03:39:06.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:06.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508789+0000", 2024-04-26T03:39:06.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4280000000000004e-06 2024-04-26T03:39:06.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:06.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508800+0000", 2024-04-26T03:39:06.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.076e-05 2024-04-26T03:39:06.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:06.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508808+0000", 2024-04-26T03:39:06.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.593e-06 2024-04-26T03:39:06.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:06.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508993+0000", 2024-04-26T03:39:06.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018527999999999999 2024-04-26T03:39:06.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-04-26T03:39:06.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.509001+0000", 2024-04-26T03:39:06.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6529999999999994e-06 2024-04-26T03:39:06.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:06.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.513192+0000", 2024-04-26T03:39:06.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041913009999999997 2024-04-26T03:39:06.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:06.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.513348+0000", 2024-04-26T03:39:06.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000155654 2024-04-26T03:39:06.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:06.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.514509+0000", 2024-04-26T03:39:06.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011615169999999999 2024-04-26T03:39:06.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:06.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.521845+0000", 2024-04-26T03:39:06.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0073361140000000003 2024-04-26T03:39:06.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:06.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.523044+0000", 2024-04-26T03:39:06.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001198939 2024-04-26T03:39:06.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:06.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.523124+0000", 2024-04-26T03:39:06.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0235999999999995e-05 2024-04-26T03:39:06.404 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:06.405 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:06.405 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:06.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.405 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:258 3.5s0 3:a64abe59:::smithi09235267-10:head [append 0~0] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T03:39:06.405 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:56.508917+0000", 2024-04-26T03:39:06.405 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.818759193, 2024-04-26T03:39:06.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014562304999999999, 2024-04-26T03:39:06.405 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:06.405 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:06.405 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:06.405 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:06.405 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:06.405 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:06.406 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 258 2024-04-26T03:39:06.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.406 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:06.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:06.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508917+0000", 2024-04-26T03:39:06.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:06.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:06.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508917+0000", 2024-04-26T03:39:06.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:06.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:06.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508918+0000", 2024-04-26T03:39:06.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5600000000000004e-07 2024-04-26T03:39:06.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:06.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508922+0000", 2024-04-26T03:39:06.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4730000000000002e-06 2024-04-26T03:39:06.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:06.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508937+0000", 2024-04-26T03:39:06.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5114e-05 2024-04-26T03:39:06.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:06.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508948+0000", 2024-04-26T03:39:06.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1222e-05 2024-04-26T03:39:06.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:06.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.509035+0000", 2024-04-26T03:39:06.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7032000000000003e-05 2024-04-26T03:39:06.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-04-26T03:39:06.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.509040+0000", 2024-04-26T03:39:06.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6299999999999997e-06 2024-04-26T03:39:06.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:06.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.515348+0000", 2024-04-26T03:39:06.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0063082959999999997 2024-04-26T03:39:06.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:06.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.515613+0000", 2024-04-26T03:39:06.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026497100000000002 2024-04-26T03:39:06.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:06.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.516295+0000", 2024-04-26T03:39:06.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00068196499999999998 2024-04-26T03:39:06.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:06.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.521977+0000", 2024-04-26T03:39:06.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056815759999999998 2024-04-26T03:39:06.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:06.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.523407+0000", 2024-04-26T03:39:06.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014299409999999999 2024-04-26T03:39:06.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:06.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.523479+0000", 2024-04-26T03:39:06.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2229000000000003e-05 2024-04-26T03:39:06.411 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:06.411 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:06.412 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:06.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.412 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:318 3.5s0 3:a79a63ac:::smithi09235267-23:head [delete] snapc 7=[7,6,5] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e44)", 2024-04-26T03:39:06.412 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:00.528087+0000", 2024-04-26T03:39:06.412 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.7995885039999999, 2024-04-26T03:39:06.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.020004624, 2024-04-26T03:39:06.412 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:06.412 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:06.412 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:06.412 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:06.412 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:06.412 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:06.412 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 318 2024-04-26T03:39:06.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.413 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:06.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:06.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:00.528087+0000", 2024-04-26T03:39:06.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:06.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:06.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:00.528087+0000", 2024-04-26T03:39:06.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:06.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:06.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:00.528089+0000", 2024-04-26T03:39:06.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9120000000000001e-06 2024-04-26T03:39:06.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:06.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:00.528096+0000", 2024-04-26T03:39:06.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0829999999999998e-06 2024-04-26T03:39:06.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:06.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:00.528112+0000", 2024-04-26T03:39:06.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5619000000000001e-05 2024-04-26T03:39:06.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:06.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:00.528121+0000", 2024-04-26T03:39:06.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6789999999999994e-06 2024-04-26T03:39:06.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:06.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:00.528185+0000", 2024-04-26T03:39:06.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4156e-05 2024-04-26T03:39:06.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-04-26T03:39:06.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:00.528190+0000", 2024-04-26T03:39:06.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9010000000000002e-06 2024-04-26T03:39:06.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:06.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:00.534892+0000", 2024-04-26T03:39:06.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067026239999999999 2024-04-26T03:39:06.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-26T03:39:06.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:00.534897+0000", 2024-04-26T03:39:06.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9389999999999996e-06 2024-04-26T03:39:06.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:06.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.540918+0000", 2024-04-26T03:39:06.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0060203679999999 2024-04-26T03:39:06.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:06.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.541113+0000", 2024-04-26T03:39:06.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019529200000000001 2024-04-26T03:39:06.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:06.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.541689+0000", 2024-04-26T03:39:06.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00057578299999999996 2024-04-26T03:39:06.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:06.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.544502+0000", 2024-04-26T03:39:06.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028131950000000001 2024-04-26T03:39:06.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:06.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.547870+0000", 2024-04-26T03:39:06.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0033683039999999999 2024-04-26T03:39:06.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:06.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.548092+0000", 2024-04-26T03:39:06.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000221769 2024-04-26T03:39:06.418 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:06.418 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:06.418 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:06.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.419 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.3s3 49/44 ECSubWrite(tid=68, reqid=osd.10.0:66, at_version=0'0, trim_to=0'0, roll_forward_to=49'30))", 2024-04-26T03:39:06.419 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:04.689740+0000", 2024-04-26T03:39:06.419 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.6379357890000001, 2024-04-26T03:39:06.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013035526, 2024-04-26T03:39:06.420 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:06.420 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:06.420 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:06.420 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:06.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:06.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.689740+0000", 2024-04-26T03:39:06.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:06.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:06.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.689740+0000", 2024-04-26T03:39:06.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:06.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:06.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.689741+0000", 2024-04-26T03:39:06.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9500000000000001e-07 2024-04-26T03:39:06.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:06.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.689751+0000", 2024-04-26T03:39:06.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0308e-05 2024-04-26T03:39:06.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:06.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.689766+0000", 2024-04-26T03:39:06.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4810000000000001e-05 2024-04-26T03:39:06.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:06.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.689775+0000", 2024-04-26T03:39:06.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6940000000000001e-06 2024-04-26T03:39:06.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:06.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.689835+0000", 2024-04-26T03:39:06.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0361e-05 2024-04-26T03:39:06.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:06.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.689846+0000", 2024-04-26T03:39:06.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0648e-05 2024-04-26T03:39:06.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:06.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.701493+0000", 2024-04-26T03:39:06.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011647342 2024-04-26T03:39:06.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:06.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.702776+0000", 2024-04-26T03:39:06.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001282568 2024-04-26T03:39:06.423 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:06.423 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:06.423 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:06.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.424 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.13s3 50/44 ECSubWrite(tid=74, reqid=client.4556.0:446, at_version=50'37, trim_to=0'0, roll_forward_to=50'36))", 2024-04-26T03:39:06.424 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:05.702348+0000", 2024-04-26T03:39:06.424 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.62532790800000004, 2024-04-26T03:39:06.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013733293000000001, 2024-04-26T03:39:06.424 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:06.424 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:06.424 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:06.424 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:06.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:06.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.702348+0000", 2024-04-26T03:39:06.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:06.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:06.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.702348+0000", 2024-04-26T03:39:06.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:06.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:06.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.702349+0000", 2024-04-26T03:39:06.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5400000000000001e-07 2024-04-26T03:39:06.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:06.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.702837+0000", 2024-04-26T03:39:06.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00048857299999999996 2024-04-26T03:39:06.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:06.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.702851+0000", 2024-04-26T03:39:06.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3723999999999999e-05 2024-04-26T03:39:06.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:06.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.702862+0000", 2024-04-26T03:39:06.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1571e-05 2024-04-26T03:39:06.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:06.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.702925+0000", 2024-04-26T03:39:06.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2890000000000003e-05 2024-04-26T03:39:06.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:06.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.702942+0000", 2024-04-26T03:39:06.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6857000000000001e-05 2024-04-26T03:39:06.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:06.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716045+0000", 2024-04-26T03:39:06.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013102744 2024-04-26T03:39:06.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:06.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716081+0000", 2024-04-26T03:39:06.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6380000000000001e-05 2024-04-26T03:39:06.428 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:06.428 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:06.428 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:06.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.428 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:462 3.5s0 3:a64abe59:::smithi09235267-10:head [append 0~753664] snapc 8=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:06.428 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:05.749661+0000", 2024-04-26T03:39:06.428 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.57801499899999997, 2024-04-26T03:39:06.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014030855, 2024-04-26T03:39:06.428 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:06.429 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:06.429 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:06.429 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:06.429 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:06.429 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:06.429 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 462 2024-04-26T03:39:06.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.429 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:06.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:06.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.749661+0000", 2024-04-26T03:39:06.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:06.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:06.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.749661+0000", 2024-04-26T03:39:06.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:06.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:06.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.749663+0000", 2024-04-26T03:39:06.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.722e-06 2024-04-26T03:39:06.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:06.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.749971+0000", 2024-04-26T03:39:06.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000308475 2024-04-26T03:39:06.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:06.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.749985+0000", 2024-04-26T03:39:06.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4191e-05 2024-04-26T03:39:06.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:06.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.749995+0000", 2024-04-26T03:39:06.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0020000000000001e-05 2024-04-26T03:39:06.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:06.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.750050+0000", 2024-04-26T03:39:06.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5000000000000002e-05 2024-04-26T03:39:06.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:06.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.750121+0000", 2024-04-26T03:39:06.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0669999999999999e-05 2024-04-26T03:39:06.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:06.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.750769+0000", 2024-04-26T03:39:06.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00064844000000000002 2024-04-26T03:39:06.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:06.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.763254+0000", 2024-04-26T03:39:06.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012484215999999999 2024-04-26T03:39:06.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:06.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.763622+0000", 2024-04-26T03:39:06.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00036791699999999999 2024-04-26T03:39:06.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:06.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.763692+0000", 2024-04-26T03:39:06.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0203999999999994e-05 2024-04-26T03:39:06.434 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:06.434 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:06.434 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:06.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.434 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:466 3.5s0 3:a64abe59:::smithi09235267-10:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:06.434 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:05.755663+0000", 2024-04-26T03:39:06.434 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.57201250199999998, 2024-04-26T03:39:06.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021406392999999999, 2024-04-26T03:39:06.434 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:06.434 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:06.435 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:06.435 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:06.435 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:06.435 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:06.435 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 466 2024-04-26T03:39:06.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.435 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:06.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:06.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755663+0000", 2024-04-26T03:39:06.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:06.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:06.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755663+0000", 2024-04-26T03:39:06.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:06.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:06.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755664+0000", 2024-04-26T03:39:06.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6600000000000003e-07 2024-04-26T03:39:06.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:06.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755665+0000", 2024-04-26T03:39:06.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.389e-06 2024-04-26T03:39:06.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:06.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755669+0000", 2024-04-26T03:39:06.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2024-04-26T03:39:06.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:06.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755672+0000", 2024-04-26T03:39:06.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7190000000000001e-06 2024-04-26T03:39:06.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:06.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.764511+0000", 2024-04-26T03:39:06.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0088397730000000004 2024-04-26T03:39:06.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:06.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.764598+0000", 2024-04-26T03:39:06.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6438000000000001e-05 2024-04-26T03:39:06.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:06.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.766650+0000", 2024-04-26T03:39:06.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020521430000000002 2024-04-26T03:39:06.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:06.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.766698+0000", 2024-04-26T03:39:06.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8133000000000001e-05 2024-04-26T03:39:06.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:06.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.777070+0000", 2024-04-26T03:39:06.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010371672 2024-04-26T03:39:06.439 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:06.439 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:06.439 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:06.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.440 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:467 3.5s0 3:a79a63ac:::smithi09235267-23:head [stat,delete] snapc 8=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:06.440 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:05.755675+0000", 2024-04-26T03:39:06.440 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.57200074700000003, 2024-04-26T03:39:06.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020820686000000001, 2024-04-26T03:39:06.440 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:06.440 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:06.440 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:06.440 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:06.440 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:06.440 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:06.441 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 467 2024-04-26T03:39:06.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.441 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:06.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:06.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755675+0000", 2024-04-26T03:39:06.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:06.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:06.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755675+0000", 2024-04-26T03:39:06.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:06.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:06.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755675+0000", 2024-04-26T03:39:06.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0900000000000001e-07 2024-04-26T03:39:06.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:06.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755677+0000", 2024-04-26T03:39:06.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.406e-06 2024-04-26T03:39:06.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:06.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755680+0000", 2024-04-26T03:39:06.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2329999999999998e-06 2024-04-26T03:39:06.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:06.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755684+0000", 2024-04-26T03:39:06.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.472e-06 2024-04-26T03:39:06.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:06.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.764647+0000", 2024-04-26T03:39:06.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0089629900000000005 2024-04-26T03:39:06.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:06.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.764702+0000", 2024-04-26T03:39:06.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5748000000000002e-05 2024-04-26T03:39:06.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:06.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.765004+0000", 2024-04-26T03:39:06.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030170000000000002 2024-04-26T03:39:06.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:06.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.776390+0000", 2024-04-26T03:39:06.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011386454000000001 2024-04-26T03:39:06.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:06.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.776455+0000", 2024-04-26T03:39:06.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4304e-05 2024-04-26T03:39:06.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:06.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:06.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:06.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.776496+0000", 2024-04-26T03:39:06.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1170000000000001e-05 2024-04-26T03:39:06.445 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:06.445 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:06.445 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:06.445 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:06.445 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:06.445 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:06.447 INFO:tasks.ceph.osd.15.smithi092.stderr:2024-04-26T03:39:06.445+0000 7f6d12dec640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 31824) UID: 0 2024-04-26T03:39:06.447 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-26T03:39:06.549 INFO:tasks.ceph.osd.10.smithi066.stderr:2024-04-26T03:39:06.547+0000 7f130fcc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 31763) UID: 0 2024-04-26T03:39:06.567 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:39:06.567 INFO:teuthology.orchestra.run.smithi028.stdout: "ops": [], 2024-04-26T03:39:06.567 INFO:teuthology.orchestra.run.smithi028.stdout: "num_ops": 0 2024-04-26T03:39:06.567 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:39:06.576 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-26T03:39:06.625 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-26T03:39:06.649 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-04-26T03:39:06.648+0000 7fe9223c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31954) UID: 0 2024-04-26T03:39:06.686 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:39:06.686 INFO:teuthology.orchestra.run.smithi028.stdout: "ops": [], 2024-04-26T03:39:06.686 INFO:teuthology.orchestra.run.smithi028.stdout: "complaint_time": 30, 2024-04-26T03:39:06.686 INFO:teuthology.orchestra.run.smithi028.stdout: "num_blocked_ops": 0 2024-04-26T03:39:06.686 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:39:06.690 INFO:tasks.rados.rados.0.smithi092.stdout:185: done (1 left) 2024-04-26T03:39:06.691 INFO:tasks.rados.rados.0.smithi092.stdout:186: done (0 left) 2024-04-26T03:39:06.691 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 19 current snap is 8 2024-04-26T03:39:06.691 INFO:tasks.rados.rados.0.smithi092.stdout:187: seq_num 94 ranges {1425408=425984,1851392=786432,2637824=376832} 2024-04-26T03:39:06.697 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-26T03:39:06.700 INFO:tasks.rados.rados.0.smithi092.stdout:187: writing smithi09235267-19 from 1425408 to 1851392 tid 1 2024-04-26T03:39:06.704 INFO:tasks.rados.rados.0.smithi092.stdout:187: writing smithi09235267-19 from 1851392 to 2637824 tid 2 2024-04-26T03:39:06.707 INFO:tasks.rados.rados.0.smithi092.stdout:187: writing smithi09235267-19 from 2637824 to 3014656 tid 3 2024-04-26T03:39:06.707 INFO:tasks.rados.rados.0.smithi092.stdout:188: snap_remove snap 7 2024-04-26T03:39:06.750 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:39:06.749+0000 7f60d3c4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31757) UID: 0 2024-04-26T03:39:06.755 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0: {} 2024-04-26T03:39:06.756 INFO:teuthology.orchestra.run.smithi028.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T03:39:06.774 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1: {} 2024-04-26T03:39:06.775 INFO:teuthology.orchestra.run.smithi028.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T03:39:06.794 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2: {} 2024-04-26T03:39:06.794 INFO:teuthology.orchestra.run.smithi028.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T03:39:06.807 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:06.807 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:06.811 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3: {} 2024-04-26T03:39:06.811 INFO:teuthology.orchestra.run.smithi028.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T03:39:06.817 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-26T03:39:06.830 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4: {} 2024-04-26T03:39:06.830 INFO:teuthology.orchestra.run.smithi028.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T03:39:06.850 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-26T03:39:06.849+0000 7f610b3a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31816) UID: 0 2024-04-26T03:39:06.853 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5: {} 2024-04-26T03:39:06.854 INFO:teuthology.orchestra.run.smithi028.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T03:39:06.872 INFO:teuthology.orchestra.run.smithi028.stdout:osd.6: {} 2024-04-26T03:39:06.872 INFO:teuthology.orchestra.run.smithi028.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T03:39:06.891 INFO:teuthology.orchestra.run.smithi028.stdout:osd.7: {} 2024-04-26T03:39:06.891 INFO:teuthology.orchestra.run.smithi028.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T03:39:06.908 INFO:teuthology.orchestra.run.smithi028.stdout:osd.8: {} 2024-04-26T03:39:06.908 INFO:teuthology.orchestra.run.smithi028.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T03:39:06.927 INFO:teuthology.orchestra.run.smithi028.stdout:osd.9: {} 2024-04-26T03:39:06.927 INFO:teuthology.orchestra.run.smithi028.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T03:39:06.929 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:06.929 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:06.939 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-26T03:39:06.946 INFO:teuthology.orchestra.run.smithi028.stdout:osd.10: {} 2024-04-26T03:39:06.946 INFO:teuthology.orchestra.run.smithi028.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T03:39:06.951 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:39:06.950+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:39:06.964 INFO:teuthology.orchestra.run.smithi028.stdout:osd.11: {} 2024-04-26T03:39:06.964 INFO:teuthology.orchestra.run.smithi028.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T03:39:06.983 INFO:teuthology.orchestra.run.smithi028.stdout:osd.12: {} 2024-04-26T03:39:06.983 INFO:teuthology.orchestra.run.smithi028.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T03:39:07.001 INFO:teuthology.orchestra.run.smithi028.stdout:osd.13: {} 2024-04-26T03:39:07.001 INFO:teuthology.orchestra.run.smithi028.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T03:39:07.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:39:06.999+0000 7f12b1d9b640 0 -- 172.21.15.28:0/3960833284 >> [v2:172.21.15.66:6824/1534905602,v1:172.21.15.66:6825/1534905602] conn(0x7f12b421d8e0 msgr2=0x7f12b421fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-26T03:39:07.053 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:39:07.051+0000 7fb78c544640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31956) UID: 0 2024-04-26T03:39:07.054 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:07.054 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:07.063 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-26T03:39:07.154 INFO:tasks.ceph.osd.10.smithi066.stderr:2024-04-26T03:39:07.153+0000 7f130fcc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 31763) UID: 0 2024-04-26T03:39:07.214 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:07.214 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:07.220 INFO:teuthology.orchestra.run.smithi028.stdout:osd.14: {} 2024-04-26T03:39:07.220 INFO:teuthology.orchestra.run.smithi028.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T03:39:07.223 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-04-26T03:39:07.238 INFO:teuthology.orchestra.run.smithi028.stdout:osd.15: {} 2024-04-26T03:39:07.239 INFO:teuthology.orchestra.run.smithi028.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T03:39:07.255 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-04-26T03:39:07.253+0000 7f745118d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 31821) UID: 0 2024-04-26T03:39:07.332 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:07.332 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:07.341 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-04-26T03:39:07.385 INFO:tasks.ceph.osd.8.smithi028.stderr:2024-04-26T03:39:07.384+0000 7f5bb75d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 31968) UID: 0 2024-04-26T03:39:07.448 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:07.449 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:07.457 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-04-26T03:39:07.458 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-04-26T03:39:07.455+0000 7fdc2d21c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31819) UID: 0 2024-04-26T03:39:07.558 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:39:07.557+0000 7f60d3c4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31757) UID: 0 2024-04-26T03:39:07.605 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:07.605 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:07.613 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-04-26T03:39:07.658 INFO:tasks.ceph.osd.9.smithi042.stderr:2024-04-26T03:39:07.657+0000 7fdac63b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 31960) UID: 0 2024-04-26T03:39:07.695 INFO:tasks.rados.rados.0.smithi092.stdout:187: finishing write tid 1 to smithi09235267-19 2024-04-26T03:39:07.695 INFO:tasks.rados.rados.0.smithi092.stdout:187: finishing write tid 2 to smithi09235267-19 2024-04-26T03:39:07.695 INFO:tasks.rados.rados.0.smithi092.stdout:187: finishing write tid 3 to smithi09235267-19 2024-04-26T03:39:07.695 INFO:tasks.rados.rados.0.smithi092.stdout:187: finishing write tid 4 to smithi09235267-19 2024-04-26T03:39:07.695 INFO:tasks.rados.rados.0.smithi092.stdout:187: finishing write tid 5 to smithi09235267-19 2024-04-26T03:39:07.695 INFO:tasks.rados.rados.0.smithi092.stdout:187: oid 19 updating version 0 to 45 2024-04-26T03:39:07.695 INFO:tasks.rados.rados.0.smithi092.stdout:187: oid 19 updating version 45 to 46 2024-04-26T03:39:07.695 INFO:tasks.rados.rados.0.smithi092.stdout:187: oid 19 updating version 46 to 47 2024-04-26T03:39:07.695 INFO:tasks.rados.rados.0.smithi092.stdout:187: oid 19 updating version 47 to 48 2024-04-26T03:39:07.695 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 19 v 48 (ObjNum 94 snap 8 seq_num 94) dirty exists 2024-04-26T03:39:07.695 INFO:tasks.rados.rados.0.smithi092.stdout:187: left oid 19 (ObjNum 94 snap 8 seq_num 94) 2024-04-26T03:39:07.695 INFO:tasks.rados.rados.0.smithi092.stdout:187: done (1 left) 2024-04-26T03:39:07.695 INFO:tasks.rados.rados.0.smithi092.stdout:188: done (0 left) 2024-04-26T03:39:07.696 INFO:tasks.rados.rados.0.smithi092.stdout:189: rollback oid 41 current snap is 8 2024-04-26T03:39:07.696 INFO:tasks.rados.rados.0.smithi092.stdout:189: done (0 left) 2024-04-26T03:39:07.696 INFO:tasks.rados.rados.0.smithi092.stdout:190: delete oid 48 current snap is 8 2024-04-26T03:39:07.704 INFO:tasks.rados.rados.0.smithi092.stdout:190: done (0 left) 2024-04-26T03:39:07.704 INFO:tasks.rados.rados.0.smithi092.stdout:191: rmattr oid 33 current snap is 8 2024-04-26T03:39:07.704 INFO:tasks.rados.rados.0.smithi092.stdout:191: done (0 left) 2024-04-26T03:39:07.705 INFO:tasks.rados.rados.0.smithi092.stdout:192: read oid 25 snap -1 2024-04-26T03:39:07.705 INFO:tasks.rados.rados.0.smithi092.stdout:192: expect (ObjNum 48 snap 0 seq_num 48) 2024-04-26T03:39:07.705 INFO:tasks.rados.rados.0.smithi092.stdout:193: rollback oid 19 current snap is 8 2024-04-26T03:39:07.705 INFO:tasks.rados.rados.0.smithi092.stdout:193: done (1 left) 2024-04-26T03:39:07.705 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 16 current snap is 8 2024-04-26T03:39:07.705 INFO:tasks.rados.rados.0.smithi092.stdout:194: seq_num 96 ranges {0=458752,458752=688128,1146880=475136} 2024-04-26T03:39:07.707 INFO:tasks.rados.rados.0.smithi092.stdout:194: writing smithi09235267-16 from 0 to 458752 tid 1 2024-04-26T03:39:07.709 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-26T03:39:07.711 INFO:tasks.rados.rados.0.smithi092.stdout:194: writing smithi09235267-16 from 458752 to 1146880 tid 2 2024-04-26T03:39:07.714 INFO:tasks.rados.rados.0.smithi092.stdout:194: writing smithi09235267-16 from 1146880 to 1622016 tid 3 2024-04-26T03:39:07.714 INFO:tasks.rados.rados.0.smithi092.stdout:195: setattr oid 5 current snap is 8 2024-04-26T03:39:07.715 INFO:tasks.rados.rados.0.smithi092.stdout:192: expect (ObjNum 48 snap 0 seq_num 48) 2024-04-26T03:39:07.734 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:07.735 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:07.739 INFO:tasks.rados.rados.0.smithi092.stdout:194: finishing write tid 1 to smithi09235267-16 2024-04-26T03:39:07.739 INFO:tasks.rados.rados.0.smithi092.stdout:194: finishing write tid 2 to smithi09235267-16 2024-04-26T03:39:07.739 INFO:tasks.rados.rados.0.smithi092.stdout:194: finishing write tid 3 to smithi09235267-16 2024-04-26T03:39:07.739 INFO:tasks.rados.rados.0.smithi092.stdout:194: finishing write tid 4 to smithi09235267-16 2024-04-26T03:39:07.739 INFO:tasks.rados.rados.0.smithi092.stdout:194: finishing write tid 5 to smithi09235267-16 2024-04-26T03:39:07.739 INFO:tasks.rados.rados.0.smithi092.stdout:194: oid 16 updating version 0 to 23 2024-04-26T03:39:07.739 INFO:tasks.rados.rados.0.smithi092.stdout:194: oid 16 version 23 is already newer than 21 2024-04-26T03:39:07.739 INFO:tasks.rados.rados.0.smithi092.stdout:194: oid 16 version 23 is already newer than 22 2024-04-26T03:39:07.739 INFO:tasks.rados.rados.0.smithi092.stdout:194: oid 16 updating version 23 to 24 2024-04-26T03:39:07.739 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 16 v 24 (ObjNum 96 snap 8 seq_num 96) dirty exists 2024-04-26T03:39:07.739 INFO:tasks.rados.rados.0.smithi092.stdout:194: left oid 16 (ObjNum 96 snap 8 seq_num 96) 2024-04-26T03:39:07.739 INFO:tasks.rados.rados.0.smithi092.stdout:192: done (2 left) 2024-04-26T03:39:07.740 INFO:tasks.rados.rados.0.smithi092.stdout:194: done (1 left) 2024-04-26T03:39:07.740 INFO:tasks.rados.rados.0.smithi092.stdout:196: delete oid 9 current snap is 8 2024-04-26T03:39:07.740 INFO:tasks.rados.rados.0.smithi092.stdout:196: done (1 left) 2024-04-26T03:39:07.740 INFO:tasks.rados.rados.0.smithi092.stdout:197: read oid 38 snap -1 2024-04-26T03:39:07.740 INFO:tasks.rados.rados.0.smithi092.stdout:197: expect (ObjNum 37 snap 0 seq_num 37) 2024-04-26T03:39:07.740 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 43 current snap is 8 2024-04-26T03:39:07.741 INFO:tasks.rados.rados.0.smithi092.stdout:198: seq_num 98 ranges {1343488=491520,1835008=589824,2424832=557056,2981888=16384} 2024-04-26T03:39:07.748 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-04-26T03:39:07.750 INFO:tasks.rados.rados.0.smithi092.stdout:198: writing smithi09235267-43 from 1343488 to 1835008 tid 1 2024-04-26T03:39:07.754 INFO:tasks.rados.rados.0.smithi092.stdout:198: writing smithi09235267-43 from 1835008 to 2424832 tid 2 2024-04-26T03:39:07.758 INFO:tasks.rados.rados.0.smithi092.stdout:198: writing smithi09235267-43 from 2424832 to 2981888 tid 3 2024-04-26T03:39:07.758 INFO:tasks.rados.rados.0.smithi092.stdout:198: writing smithi09235267-43 from 2981888 to 2998272 tid 4 2024-04-26T03:39:07.758 INFO:tasks.rados.rados.0.smithi092.stdout:199: read oid 33 snap -1 2024-04-26T03:39:07.758 INFO:tasks.rados.rados.0.smithi092.stdout:199: expect (ObjNum 32 snap 0 seq_num 32) 2024-04-26T03:39:07.758 INFO:tasks.rados.rados.0.smithi092.stdout:200: read oid 28 snap -1 2024-04-26T03:39:07.758 INFO:tasks.rados.rados.0.smithi092.stdout:200: expect (ObjNum 87 snap 7 seq_num 87) 2024-04-26T03:39:07.758 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 1 current snap is 8 2024-04-26T03:39:07.758 INFO:tasks.rados.rados.0.smithi092.stdout:201: seq_num 99 ranges {2179072=507904,2686976=606208,3293184=557056} 2024-04-26T03:39:07.760 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:39:07.758+0000 7fb78c544640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31956) UID: 0 2024-04-26T03:39:07.773 INFO:tasks.rados.rados.0.smithi092.stdout:201: writing smithi09235267-1 from 2179072 to 2686976 tid 1 2024-04-26T03:39:07.776 INFO:tasks.rados.rados.0.smithi092.stdout:201: writing smithi09235267-1 from 2686976 to 3293184 tid 2 2024-04-26T03:39:07.780 INFO:tasks.rados.rados.0.smithi092.stdout:201: writing smithi09235267-1 from 3293184 to 3850240 tid 3 2024-04-26T03:39:07.780 INFO:tasks.rados.rados.0.smithi092.stdout:202: rmattr oid 26 current snap is 8 2024-04-26T03:39:07.780 INFO:tasks.rados.rados.0.smithi092.stdout:202: done (6 left) 2024-04-26T03:39:07.780 INFO:tasks.rados.rados.0.smithi092.stdout:203: delete oid 20 current snap is 8 2024-04-26T03:39:07.780 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 5 v 10 (ObjNum 161 snap 0 seq_num 1) dirty exists 2024-04-26T03:39:07.780 INFO:tasks.rados.rados.0.smithi092.stdout:197: expect (ObjNum 37 snap 0 seq_num 37) 2024-04-26T03:39:07.798 INFO:tasks.rados.rados.0.smithi092.stdout:199: expect (ObjNum 32 snap 0 seq_num 32) 2024-04-26T03:39:07.811 INFO:tasks.rados.rados.0.smithi092.stdout:198: finishing write tid 1 to smithi09235267-43 2024-04-26T03:39:07.811 INFO:tasks.rados.rados.0.smithi092.stdout:198: finishing write tid 2 to smithi09235267-43 2024-04-26T03:39:07.812 INFO:tasks.rados.rados.0.smithi092.stdout:198: finishing write tid 3 to smithi09235267-43 2024-04-26T03:39:07.812 INFO:tasks.rados.rados.0.smithi092.stdout:198: finishing write tid 4 to smithi09235267-43 2024-04-26T03:39:07.812 INFO:tasks.rados.rados.0.smithi092.stdout:198: finishing write tid 5 to smithi09235267-43 2024-04-26T03:39:07.812 INFO:tasks.rados.rados.0.smithi092.stdout:200: expect (ObjNum 87 snap 7 seq_num 87) 2024-04-26T03:39:07.860 INFO:tasks.ceph.osd.14.smithi066.stderr:2024-04-26T03:39:07.859+0000 7f5395f08640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 31765) UID: 0 2024-04-26T03:39:07.897 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:07.897 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:07.906 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-04-26T03:39:07.919 INFO:tasks.rados.rados.0.smithi092.stdout:198: finishing write tid 6 to smithi09235267-43 2024-04-26T03:39:07.919 INFO:tasks.rados.rados.0.smithi092.stdout:198: oid 43 updating version 0 to 52 2024-04-26T03:39:07.920 INFO:tasks.rados.rados.0.smithi092.stdout:198: oid 43 version 52 is already newer than 50 2024-04-26T03:39:07.920 INFO:tasks.rados.rados.0.smithi092.stdout:198: oid 43 version 52 is already newer than 51 2024-04-26T03:39:07.920 INFO:tasks.rados.rados.0.smithi092.stdout:198: oid 43 updating version 52 to 53 2024-04-26T03:39:07.920 INFO:tasks.rados.rados.0.smithi092.stdout:198: oid 43 version 53 is already newer than 49 2024-04-26T03:39:07.920 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 43 v 53 (ObjNum 98 snap 8 seq_num 98) dirty exists 2024-04-26T03:39:07.920 INFO:tasks.rados.rados.0.smithi092.stdout:198: left oid 43 (ObjNum 98 snap 8 seq_num 98) 2024-04-26T03:39:07.920 INFO:tasks.rados.rados.0.smithi092.stdout:201: finishing write tid 1 to smithi09235267-1 2024-04-26T03:39:07.920 INFO:tasks.rados.rados.0.smithi092.stdout:201: finishing write tid 2 to smithi09235267-1 2024-04-26T03:39:07.920 INFO:tasks.rados.rados.0.smithi092.stdout:201: finishing write tid 3 to smithi09235267-1 2024-04-26T03:39:07.921 INFO:tasks.rados.rados.0.smithi092.stdout:195: done (6 left) 2024-04-26T03:39:07.921 INFO:tasks.rados.rados.0.smithi092.stdout:197: done (5 left) 2024-04-26T03:39:07.921 INFO:tasks.rados.rados.0.smithi092.stdout:198: done (4 left) 2024-04-26T03:39:07.921 INFO:tasks.rados.rados.0.smithi092.stdout:199: done (3 left) 2024-04-26T03:39:07.921 INFO:tasks.rados.rados.0.smithi092.stdout:200: done (2 left) 2024-04-26T03:39:07.921 INFO:tasks.rados.rados.0.smithi092.stdout:203: done (1 left) 2024-04-26T03:39:07.921 INFO:tasks.rados.rados.0.smithi092.stdout:204: rmattr oid 37 current snap is 8 2024-04-26T03:39:07.921 INFO:tasks.rados.rados.0.smithi092.stdout:204: done (1 left) 2024-04-26T03:39:07.921 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 33 current snap is 8 2024-04-26T03:39:07.921 INFO:tasks.rados.rados.0.smithi092.stdout:201: finishing write tid 4 to smithi09235267-1 2024-04-26T03:39:07.921 INFO:tasks.rados.rados.0.smithi092.stdout:201: finishing write tid 5 to smithi09235267-1 2024-04-26T03:39:07.921 INFO:tasks.rados.rados.0.smithi092.stdout:201: oid 1 updating version 0 to 34 2024-04-26T03:39:07.922 INFO:tasks.rados.rados.0.smithi092.stdout:201: oid 1 version 34 is already newer than 33 2024-04-26T03:39:07.922 INFO:tasks.rados.rados.0.smithi092.stdout:201: oid 1 version 34 is already newer than 32 2024-04-26T03:39:07.922 INFO:tasks.rados.rados.0.smithi092.stdout:201: oid 1 updating version 34 to 35 2024-04-26T03:39:07.922 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 1 v 35 (ObjNum 99 snap 8 seq_num 99) dirty exists 2024-04-26T03:39:07.922 INFO:tasks.rados.rados.0.smithi092.stdout:201: left oid 1 (ObjNum 99 snap 8 seq_num 99) 2024-04-26T03:39:07.922 INFO:tasks.rados.rados.0.smithi092.stdout:205: seq_num 101 ranges {540672=540672,1081344=507904,1589248=622592,2211840=32768} 2024-04-26T03:39:07.924 INFO:tasks.rados.rados.0.smithi092.stdout:205: writing smithi09235267-33 from 540672 to 1081344 tid 1 2024-04-26T03:39:07.927 INFO:tasks.rados.rados.0.smithi092.stdout:205: writing smithi09235267-33 from 1081344 to 1589248 tid 2 2024-04-26T03:39:07.930 INFO:tasks.rados.rados.0.smithi092.stdout:205: writing smithi09235267-33 from 1589248 to 2211840 tid 3 2024-04-26T03:39:07.931 INFO:tasks.rados.rados.0.smithi092.stdout:205: writing smithi09235267-33 from 2211840 to 2244608 tid 4 2024-04-26T03:39:07.931 INFO:tasks.rados.rados.0.smithi092.stdout:201: done (1 left) 2024-04-26T03:39:07.931 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 6 current snap is 8 2024-04-26T03:39:07.931 INFO:tasks.rados.rados.0.smithi092.stdout:206: seq_num 102 ranges {2949120=557056,3506176=638976,4145152=524288} 2024-04-26T03:39:07.951 INFO:tasks.rados.rados.0.smithi092.stdout:206: writing smithi09235267-6 from 2949120 to 3506176 tid 1 2024-04-26T03:39:07.954 INFO:tasks.rados.rados.0.smithi092.stdout:206: writing smithi09235267-6 from 3506176 to 4145152 tid 2 2024-04-26T03:39:07.960 INFO:tasks.rados.rados.0.smithi092.stdout:206: writing smithi09235267-6 from 4145152 to 4669440 tid 3 2024-04-26T03:39:07.960 INFO:tasks.rados.rados.0.smithi092.stdout:207: copy_from oid 31 from oid 47 current snap is 8 2024-04-26T03:39:07.960 INFO:tasks.rados.rados.0.smithi092.stdout:205: finishing write tid 1 to smithi09235267-33 2024-04-26T03:39:07.960 INFO:tasks.rados.rados.0.smithi092.stdout:205: finishing write tid 2 to smithi09235267-33 2024-04-26T03:39:07.960 INFO:tasks.rados.rados.0.smithi092.stdout:205: finishing write tid 3 to smithi09235267-33 2024-04-26T03:39:07.960 INFO:tasks.rados.rados.0.smithi092.stdout:205: finishing write tid 4 to smithi09235267-33 2024-04-26T03:39:07.960 INFO:tasks.rados.rados.0.smithi092.stdout:205: finishing write tid 5 to smithi09235267-33 2024-04-26T03:39:07.960 INFO:tasks.rados.rados.0.smithi092.stdout:205: finishing write tid 6 to smithi09235267-33 2024-04-26T03:39:07.961 INFO:tasks.rados.rados.0.smithi092.stdout:205: oid 33 updating version 0 to 11 2024-04-26T03:39:07.961 INFO:tasks.rados.rados.0.smithi092.stdout:205: oid 33 version 11 is already newer than 10 2024-04-26T03:39:07.961 INFO:tasks.rados.rados.0.smithi092.stdout:205: oid 33 version 11 is already newer than 9 2024-04-26T03:39:07.961 INFO:tasks.rados.rados.0.smithi092.stdout:205: oid 33 version 11 is already newer than 7 2024-04-26T03:39:07.961 INFO:tasks.rados.rados.0.smithi092.stdout:205: oid 33 version 11 is already newer than 8 2024-04-26T03:39:07.961 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 33 v 11 (ObjNum 101 snap 8 seq_num 101) dirty exists 2024-04-26T03:39:07.961 INFO:tasks.rados.rados.0.smithi092.stdout:205: left oid 33 (ObjNum 101 snap 8 seq_num 101) 2024-04-26T03:39:07.961 INFO:tasks.rados.rados.0.smithi092.stdout:205: done (2 left) 2024-04-26T03:39:07.961 INFO:tasks.rados.rados.0.smithi092.stdout:208: setattr oid 20 current snap is 8 2024-04-26T03:39:07.961 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 23 current snap is 8 2024-04-26T03:39:07.961 INFO:tasks.rados.rados.0.smithi092.stdout:209: seq_num 103 ranges {0=573440,573440=786432,1359872=376832} 2024-04-26T03:39:07.962 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:39:07.961+0000 7fb78c544640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31956) UID: 0 2024-04-26T03:39:07.963 INFO:tasks.rados.rados.0.smithi092.stdout:209: writing smithi09235267-23 from 0 to 573440 tid 1 2024-04-26T03:39:07.969 INFO:tasks.rados.rados.0.smithi092.stdout:209: writing smithi09235267-23 from 573440 to 1359872 tid 2 2024-04-26T03:39:07.972 INFO:tasks.rados.rados.0.smithi092.stdout:209: writing smithi09235267-23 from 1359872 to 1736704 tid 3 2024-04-26T03:39:07.972 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 1 current snap is 8 2024-04-26T03:39:07.972 INFO:tasks.rados.rados.0.smithi092.stdout:210: seq_num 104 ranges {3850240=589824,4440064=540672,4980736=622592} 2024-04-26T03:39:07.996 INFO:tasks.rados.rados.0.smithi092.stdout:210: writing smithi09235267-1 from 3850240 to 4440064 tid 1 2024-04-26T03:39:07.999 INFO:tasks.rados.rados.0.smithi092.stdout:210: writing smithi09235267-1 from 4440064 to 4980736 tid 2 2024-04-26T03:39:08.005 INFO:tasks.rados.rados.0.smithi092.stdout:210: writing smithi09235267-1 from 4980736 to 5603328 tid 3 2024-04-26T03:39:08.005 INFO:tasks.rados.rados.0.smithi092.stdout:211: read oid 42 snap -1 2024-04-26T03:39:08.005 INFO:tasks.rados.rados.0.smithi092.stdout:211: expect (ObjNum 60 snap 0 seq_num 60) 2024-04-26T03:39:08.005 INFO:tasks.rados.rados.0.smithi092.stdout:212: read oid 10 snap -1 2024-04-26T03:39:08.005 INFO:tasks.rados.rados.0.smithi092.stdout:212: expect (ObjNum 90 snap 7 seq_num 90) 2024-04-26T03:39:08.005 INFO:tasks.rados.rados.0.smithi092.stdout:213: read oid 45 snap -1 2024-04-26T03:39:08.005 INFO:tasks.rados.rados.0.smithi092.stdout:213: expect (ObjNum 44 snap 0 seq_num 44) 2024-04-26T03:39:08.005 INFO:tasks.rados.rados.0.smithi092.stdout:214: read oid 27 snap -1 2024-04-26T03:39:08.006 INFO:tasks.rados.rados.0.smithi092.stdout:214: expect (ObjNum 66 snap 3 seq_num 66) 2024-04-26T03:39:08.006 INFO:tasks.rados.rados.0.smithi092.stdout:215: delete oid 34 current snap is 8 2024-04-26T03:39:08.006 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 20 v 42 (ObjNum 113 snap 0 seq_num 3583255664) dirty exists 2024-04-26T03:39:08.006 INFO:tasks.rados.rados.0.smithi092.stdout:206: finishing write tid 1 to smithi09235267-6 2024-04-26T03:39:08.006 INFO:tasks.rados.rados.0.smithi092.stdout:206: finishing write tid 2 to smithi09235267-6 2024-04-26T03:39:08.006 INFO:tasks.rados.rados.0.smithi092.stdout:206: finishing write tid 3 to smithi09235267-6 2024-04-26T03:39:08.006 INFO:tasks.rados.rados.0.smithi092.stdout:206: finishing write tid 4 to smithi09235267-6 2024-04-26T03:39:08.006 INFO:tasks.rados.rados.0.smithi092.stdout:206: finishing write tid 5 to smithi09235267-6 2024-04-26T03:39:08.006 INFO:tasks.rados.rados.0.smithi092.stdout:206: oid 6 updating version 0 to 41 2024-04-26T03:39:08.006 INFO:tasks.rados.rados.0.smithi092.stdout:206: oid 6 updating version 41 to 43 2024-04-26T03:39:08.006 INFO:tasks.rados.rados.0.smithi092.stdout:206: oid 6 version 43 is already newer than 42 2024-04-26T03:39:08.006 INFO:tasks.rados.rados.0.smithi092.stdout:206: oid 6 updating version 43 to 44 2024-04-26T03:39:08.006 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 6 v 44 (ObjNum 102 snap 8 seq_num 102) dirty exists 2024-04-26T03:39:08.006 INFO:tasks.rados.rados.0.smithi092.stdout:206: left oid 6 (ObjNum 102 snap 8 seq_num 102) 2024-04-26T03:39:08.007 INFO:tasks.rados.rados.0.smithi092.stdout:209: finishing write tid 1 to smithi09235267-23 2024-04-26T03:39:08.007 INFO:tasks.rados.rados.0.smithi092.stdout:207: finishing copy_from racing read to smithi09235267-31 2024-04-26T03:39:08.007 INFO:tasks.rados.rados.0.smithi092.stdout:209: finishing write tid 2 to smithi09235267-23 2024-04-26T03:39:08.007 INFO:tasks.rados.rados.0.smithi092.stdout:209: finishing write tid 3 to smithi09235267-23 2024-04-26T03:39:08.007 INFO:tasks.rados.rados.0.smithi092.stdout:209: finishing write tid 4 to smithi09235267-23 2024-04-26T03:39:08.007 INFO:tasks.rados.rados.0.smithi092.stdout:209: finishing write tid 5 to smithi09235267-23 2024-04-26T03:39:08.007 INFO:tasks.rados.rados.0.smithi092.stdout:209: oid 23 updating version 0 to 53 2024-04-26T03:39:08.007 INFO:tasks.rados.rados.0.smithi092.stdout:209: oid 23 updating version 53 to 55 2024-04-26T03:39:08.007 INFO:tasks.rados.rados.0.smithi092.stdout:209: oid 23 version 55 is already newer than 54 2024-04-26T03:39:08.007 INFO:tasks.rados.rados.0.smithi092.stdout:209: oid 23 version 55 is already newer than 52 2024-04-26T03:39:08.007 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 23 v 55 (ObjNum 103 snap 8 seq_num 103) dirty exists 2024-04-26T03:39:08.007 INFO:tasks.rados.rados.0.smithi092.stdout:209: left oid 23 (ObjNum 103 snap 8 seq_num 103) 2024-04-26T03:39:08.007 INFO:tasks.rados.rados.0.smithi092.stdout:207: finishing copy_from to smithi09235267-31 2024-04-26T03:39:08.007 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 31 v 31 (ObjNum 93 snap 7 seq_num 93) dirty exists 2024-04-26T03:39:08.008 INFO:tasks.rados.rados.0.smithi092.stdout:206: done (9 left) 2024-04-26T03:39:08.008 INFO:tasks.rados.rados.0.smithi092.stdout:207: done (8 left) 2024-04-26T03:39:08.008 INFO:tasks.rados.rados.0.smithi092.stdout:208: done (7 left) 2024-04-26T03:39:08.008 INFO:tasks.rados.rados.0.smithi092.stdout:209: done (6 left) 2024-04-26T03:39:08.008 INFO:tasks.rados.rados.0.smithi092.stdout:215: done (5 left) 2024-04-26T03:39:08.008 INFO:tasks.rados.rados.0.smithi092.stdout:216: delete oid 32 current snap is 8 2024-04-26T03:39:08.011 INFO:tasks.rados.rados.0.smithi092.stdout:216: done (5 left) 2024-04-26T03:39:08.011 INFO:tasks.rados.rados.0.smithi092.stdout:217: delete oid 9 current snap is 8 2024-04-26T03:39:08.013 INFO:tasks.rados.rados.0.smithi092.stdout:210: finishing write tid 1 to smithi09235267-1 2024-04-26T03:39:08.013 INFO:tasks.rados.rados.0.smithi092.stdout:213: expect (ObjNum 44 snap 0 seq_num 44) 2024-04-26T03:39:08.031 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:08.031 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:08.036 INFO:tasks.rados.rados.0.smithi092.stdout:212: expect (ObjNum 90 snap 7 seq_num 90) 2024-04-26T03:39:08.040 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-26T03:39:08.063 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:39:08.062+0000 7f60d3c4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31757) UID: 0 2024-04-26T03:39:08.121 INFO:tasks.rados.rados.0.smithi092.stdout:214: expect (ObjNum 66 snap 3 seq_num 66) 2024-04-26T03:39:08.148 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:08.148 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:08.157 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-26T03:39:08.166 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-04-26T03:39:08.163+0000 7fdc2d21c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31819) UID: 0 2024-04-26T03:39:08.166 INFO:tasks.rados.rados.0.smithi092.stdout:210: finishing write tid 2 to smithi09235267-1 2024-04-26T03:39:08.167 INFO:tasks.rados.rados.0.smithi092.stdout:210: finishing write tid 3 to smithi09235267-1 2024-04-26T03:39:08.167 INFO:tasks.rados.rados.0.smithi092.stdout:210: finishing write tid 4 to smithi09235267-1 2024-04-26T03:39:08.167 INFO:tasks.rados.rados.0.smithi092.stdout:210: finishing write tid 5 to smithi09235267-1 2024-04-26T03:39:08.167 INFO:tasks.rados.rados.0.smithi092.stdout:210: oid 1 updating version 0 to 36 2024-04-26T03:39:08.167 INFO:tasks.rados.rados.0.smithi092.stdout:210: oid 1 updating version 36 to 38 2024-04-26T03:39:08.167 INFO:tasks.rados.rados.0.smithi092.stdout:210: oid 1 updating version 38 to 39 2024-04-26T03:39:08.167 INFO:tasks.rados.rados.0.smithi092.stdout:210: oid 1 version 39 is already newer than 37 2024-04-26T03:39:08.167 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 1 v 39 (ObjNum 104 snap 8 seq_num 104) dirty exists 2024-04-26T03:39:08.167 INFO:tasks.rados.rados.0.smithi092.stdout:210: left oid 1 (ObjNum 104 snap 8 seq_num 104) 2024-04-26T03:39:08.167 INFO:tasks.rados.rados.0.smithi092.stdout:211: expect (ObjNum 60 snap 0 seq_num 60) 2024-04-26T03:39:08.265 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:08.265 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:08.266 INFO:tasks.ceph.osd.8.smithi028.stderr:2024-04-26T03:39:08.265+0000 7f5bb75d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 31968) UID: 0 2024-04-26T03:39:08.268 INFO:tasks.rados.rados.0.smithi092.stdout:210: done (5 left) 2024-04-26T03:39:08.268 INFO:tasks.rados.rados.0.smithi092.stdout:211: done (4 left) 2024-04-26T03:39:08.268 INFO:tasks.rados.rados.0.smithi092.stdout:212: done (3 left) 2024-04-26T03:39:08.269 INFO:tasks.rados.rados.0.smithi092.stdout:213: done (2 left) 2024-04-26T03:39:08.269 INFO:tasks.rados.rados.0.smithi092.stdout:214: done (1 left) 2024-04-26T03:39:08.269 INFO:tasks.rados.rados.0.smithi092.stdout:217: done (0 left) 2024-04-26T03:39:08.269 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 3 current snap is 8 2024-04-26T03:39:08.269 INFO:tasks.rados.rados.0.smithi092.stdout:218: seq_num 108 ranges {49152=655360,704512=606208,1310720=475136,1785856=81920} 2024-04-26T03:39:08.272 INFO:tasks.rados.rados.0.smithi092.stdout:218: writing smithi09235267-3 from 49152 to 704512 tid 1 2024-04-26T03:39:08.274 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-26T03:39:08.275 INFO:tasks.rados.rados.0.smithi092.stdout:218: writing smithi09235267-3 from 704512 to 1310720 tid 2 2024-04-26T03:39:08.278 INFO:tasks.rados.rados.0.smithi092.stdout:218: writing smithi09235267-3 from 1310720 to 1785856 tid 3 2024-04-26T03:39:08.278 INFO:tasks.rados.rados.0.smithi092.stdout:218: writing smithi09235267-3 from 1785856 to 1867776 tid 4 2024-04-26T03:39:08.278 INFO:tasks.rados.rados.0.smithi092.stdout:219: delete oid 11 current snap is 8 2024-04-26T03:39:08.280 INFO:tasks.rados.rados.0.smithi092.stdout:219: done (1 left) 2024-04-26T03:39:08.281 INFO:tasks.rados.rados.0.smithi092.stdout:220: read oid 41 snap -1 2024-04-26T03:39:08.281 INFO:tasks.rados.rados.0.smithi092.stdout:220: expect (ObjNum 40 snap 0 seq_num 40) 2024-04-26T03:39:08.281 INFO:tasks.rados.rados.0.smithi092.stdout:221: read oid 7 snap -1 2024-04-26T03:39:08.281 INFO:tasks.rados.rados.0.smithi092.stdout:221: expect (ObjNum 59 snap 0 seq_num 59) 2024-04-26T03:39:08.281 INFO:tasks.rados.rados.0.smithi092.stdout:222: setattr oid 12 current snap is 8 2024-04-26T03:39:08.282 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 2 current snap is 8 2024-04-26T03:39:08.282 INFO:tasks.rados.rados.0.smithi092.stdout:223: seq_num 110 ranges {819200=688128,1507328=507904,2015232=655360} 2024-04-26T03:39:08.291 INFO:tasks.rados.rados.0.smithi092.stdout:223: writing smithi09235267-2 from 819200 to 1507328 tid 1 2024-04-26T03:39:08.294 INFO:tasks.rados.rados.0.smithi092.stdout:223: writing smithi09235267-2 from 1507328 to 2015232 tid 2 2024-04-26T03:39:08.300 INFO:tasks.rados.rados.0.smithi092.stdout:223: writing smithi09235267-2 from 2015232 to 2670592 tid 3 2024-04-26T03:39:08.300 INFO:tasks.rados.rados.0.smithi092.stdout:224: read oid 18 snap -1 2024-04-26T03:39:08.300 INFO:tasks.rados.rados.0.smithi092.stdout:224: expect deleted 2024-04-26T03:39:08.300 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 28 current snap is 8 2024-04-26T03:39:08.301 INFO:tasks.rados.rados.0.smithi092.stdout:225: seq_num 111 ranges {3031040=704512,3735552=655360,4390912=507904} 2024-04-26T03:39:08.320 INFO:tasks.rados.rados.0.smithi092.stdout:225: writing smithi09235267-28 from 3031040 to 3735552 tid 1 2024-04-26T03:39:08.323 INFO:tasks.rados.rados.0.smithi092.stdout:225: writing smithi09235267-28 from 3735552 to 4390912 tid 2 2024-04-26T03:39:08.326 INFO:tasks.rados.rados.0.smithi092.stdout:225: writing smithi09235267-28 from 4390912 to 4898816 tid 3 2024-04-26T03:39:08.326 INFO:tasks.rados.rados.0.smithi092.stdout:226: read oid 23 snap -1 2024-04-26T03:39:08.326 INFO:tasks.rados.rados.0.smithi092.stdout:226: expect (ObjNum 103 snap 8 seq_num 103) 2024-04-26T03:39:08.326 INFO:tasks.rados.rados.0.smithi092.stdout:227: snap_create 2024-04-26T03:39:08.326 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 12 v 31 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-04-26T03:39:08.326 INFO:tasks.rados.rados.0.smithi092.stdout:220: expect (ObjNum 40 snap 0 seq_num 40) 2024-04-26T03:39:08.346 INFO:tasks.rados.rados.0.smithi092.stdout:221: expect (ObjNum 59 snap 0 seq_num 59) 2024-04-26T03:39:08.366 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:39:08.364+0000 7fb78c544640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31956) UID: 0 2024-04-26T03:39:08.375 INFO:tasks.rados.rados.0.smithi092.stdout:218: finishing write tid 1 to smithi09235267-3 2024-04-26T03:39:08.375 INFO:tasks.rados.rados.0.smithi092.stdout:218: finishing write tid 2 to smithi09235267-3 2024-04-26T03:39:08.376 INFO:tasks.rados.rados.0.smithi092.stdout:218: finishing write tid 3 to smithi09235267-3 2024-04-26T03:39:08.376 INFO:tasks.rados.rados.0.smithi092.stdout:218: finishing write tid 4 to smithi09235267-3 2024-04-26T03:39:08.376 INFO:tasks.rados.rados.0.smithi092.stdout:218: finishing write tid 5 to smithi09235267-3 2024-04-26T03:39:08.376 INFO:tasks.rados.rados.0.smithi092.stdout:218: finishing write tid 6 to smithi09235267-3 2024-04-26T03:39:08.376 INFO:tasks.rados.rados.0.smithi092.stdout:218: oid 3 updating version 0 to 58 2024-04-26T03:39:08.376 INFO:tasks.rados.rados.0.smithi092.stdout:218: oid 3 version 58 is already newer than 56 2024-04-26T03:39:08.376 INFO:tasks.rados.rados.0.smithi092.stdout:218: oid 3 version 58 is already newer than 57 2024-04-26T03:39:08.376 INFO:tasks.rados.rados.0.smithi092.stdout:218: oid 3 version 58 is already newer than 55 2024-04-26T03:39:08.376 INFO:tasks.rados.rados.0.smithi092.stdout:218: oid 3 version 58 is already newer than 54 2024-04-26T03:39:08.376 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 3 v 58 (ObjNum 108 snap 8 seq_num 108) dirty exists 2024-04-26T03:39:08.376 INFO:tasks.rados.rados.0.smithi092.stdout:218: left oid 3 (ObjNum 108 snap 8 seq_num 108) 2024-04-26T03:39:08.376 INFO:tasks.rados.rados.0.smithi092.stdout:223: finishing write tid 1 to smithi09235267-2 2024-04-26T03:39:08.376 INFO:tasks.rados.rados.0.smithi092.stdout:223: finishing write tid 2 to smithi09235267-2 2024-04-26T03:39:08.376 INFO:tasks.rados.rados.0.smithi092.stdout:223: finishing write tid 3 to smithi09235267-2 2024-04-26T03:39:08.376 INFO:tasks.rados.rados.0.smithi092.stdout:223: finishing write tid 4 to smithi09235267-2 2024-04-26T03:39:08.377 INFO:tasks.rados.rados.0.smithi092.stdout:223: finishing write tid 5 to smithi09235267-2 2024-04-26T03:39:08.377 INFO:tasks.rados.rados.0.smithi092.stdout:223: oid 2 updating version 0 to 53 2024-04-26T03:39:08.377 INFO:tasks.rados.rados.0.smithi092.stdout:223: oid 2 version 53 is already newer than 52 2024-04-26T03:39:08.377 INFO:tasks.rados.rados.0.smithi092.stdout:223: oid 2 updating version 53 to 55 2024-04-26T03:39:08.377 INFO:tasks.rados.rados.0.smithi092.stdout:223: oid 2 version 55 is already newer than 54 2024-04-26T03:39:08.377 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 2 v 55 (ObjNum 110 snap 8 seq_num 110) dirty exists 2024-04-26T03:39:08.377 INFO:tasks.rados.rados.0.smithi092.stdout:223: left oid 2 (ObjNum 110 snap 8 seq_num 110) 2024-04-26T03:39:08.377 INFO:tasks.rados.rados.0.smithi092.stdout:226: expect (ObjNum 103 snap 8 seq_num 103) 2024-04-26T03:39:08.386 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:08.386 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:08.395 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-26T03:39:08.418 INFO:tasks.rados.rados.0.smithi092.stdout:225: finishing write tid 1 to smithi09235267-28 2024-04-26T03:39:08.419 INFO:tasks.rados.rados.0.smithi092.stdout:225: finishing write tid 2 to smithi09235267-28 2024-04-26T03:39:08.419 INFO:tasks.rados.rados.0.smithi092.stdout:225: finishing write tid 3 to smithi09235267-28 2024-04-26T03:39:08.419 INFO:tasks.rados.rados.0.smithi092.stdout:225: finishing write tid 4 to smithi09235267-28 2024-04-26T03:39:08.419 INFO:tasks.rados.rados.0.smithi092.stdout:225: finishing write tid 5 to smithi09235267-28 2024-04-26T03:39:08.419 INFO:tasks.rados.rados.0.smithi092.stdout:225: oid 28 updating version 0 to 32 2024-04-26T03:39:08.419 INFO:tasks.rados.rados.0.smithi092.stdout:225: oid 28 updating version 32 to 35 2024-04-26T03:39:08.419 INFO:tasks.rados.rados.0.smithi092.stdout:225: oid 28 version 35 is already newer than 34 2024-04-26T03:39:08.419 INFO:tasks.rados.rados.0.smithi092.stdout:225: oid 28 version 35 is already newer than 33 2024-04-26T03:39:08.419 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 28 v 35 (ObjNum 111 snap 8 seq_num 111) dirty exists 2024-04-26T03:39:08.420 INFO:tasks.rados.rados.0.smithi092.stdout:225: left oid 28 (ObjNum 111 snap 8 seq_num 111) 2024-04-26T03:39:08.479 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:39:08.478+0000 7fb78c544640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31956) UID: 0 2024-04-26T03:39:08.543 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:08.543 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:08.551 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-26T03:39:08.593 INFO:tasks.ceph.osd.9.smithi042.stderr:2024-04-26T03:39:08.591+0000 7fdac63b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 31960) UID: 0 2024-04-26T03:39:08.669 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:39:08.668+0000 7f60d3c4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31757) UID: 0 2024-04-26T03:39:08.697 INFO:teuthology.orchestra.run.smithi028.stderr:nodeep-scrub is unset 2024-04-26T03:39:08.698 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:08.698 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:08.706 INFO:tasks.rados.rados.0.smithi092.stdout:218: done (8 left) 2024-04-26T03:39:08.707 INFO:tasks.rados.rados.0.smithi092.stdout:220: done (7 left) 2024-04-26T03:39:08.707 INFO:tasks.rados.rados.0.smithi092.stdout:221: done (6 left) 2024-04-26T03:39:08.707 INFO:tasks.rados.rados.0.smithi092.stdout:222: done (5 left) 2024-04-26T03:39:08.707 INFO:tasks.rados.rados.0.smithi092.stdout:223: done (4 left) 2024-04-26T03:39:08.707 INFO:tasks.rados.rados.0.smithi092.stdout:224: done (3 left) 2024-04-26T03:39:08.707 INFO:tasks.rados.rados.0.smithi092.stdout:225: done (2 left) 2024-04-26T03:39:08.707 INFO:tasks.rados.rados.0.smithi092.stdout:226: done (1 left) 2024-04-26T03:39:08.707 INFO:tasks.rados.rados.0.smithi092.stdout:227: done (0 left) 2024-04-26T03:39:08.707 INFO:tasks.rados.rados.0.smithi092.stdout:228: snap_create 2024-04-26T03:39:08.708 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-26T03:39:08.770 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:39:08.769+0000 7f60d3c4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31757) UID: 0 2024-04-26T03:39:08.815 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:08.816 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:08.824 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-04-26T03:39:08.871 INFO:tasks.ceph.osd.15.smithi092.stderr:2024-04-26T03:39:08.870+0000 7f6d12dec640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 31824) UID: 0 2024-04-26T03:39:08.931 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:08.931 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:08.940 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-04-26T03:39:08.985 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-04-26T03:39:08.982+0000 7fe9223c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31954) UID: 0 2024-04-26T03:39:09.047 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:09.047 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:09.056 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-04-26T03:39:09.073 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-04-26T03:39:09.072+0000 7f745118d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 31821) UID: 0 2024-04-26T03:39:09.173 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:39:09.173+0000 7fba0d2ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31961) UID: 0 2024-04-26T03:39:09.203 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:09.204 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:09.212 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-04-26T03:39:09.251 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-26T03:39:09.296 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:39:09.295+0000 7fba0d2ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31961) UID: 0 2024-04-26T03:39:09.319 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:09.319 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:09.328 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-04-26T03:39:09.376 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-04-26T03:39:09.375+0000 7fee562fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31759) UID: 0 2024-04-26T03:39:09.380 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0: {} 2024-04-26T03:39:09.380 INFO:teuthology.orchestra.run.smithi028.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T03:39:09.400 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1: {} 2024-04-26T03:39:09.400 INFO:teuthology.orchestra.run.smithi028.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T03:39:09.417 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2: {} 2024-04-26T03:39:09.418 INFO:teuthology.orchestra.run.smithi028.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T03:39:09.435 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:09.435 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:09.435 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3: {} 2024-04-26T03:39:09.435 INFO:teuthology.orchestra.run.smithi028.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T03:39:09.443 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-04-26T03:39:09.452 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4: {} 2024-04-26T03:39:09.452 INFO:teuthology.orchestra.run.smithi028.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T03:39:09.469 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5: {} 2024-04-26T03:39:09.469 INFO:teuthology.orchestra.run.smithi028.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T03:39:09.477 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:39:09.476+0000 7fba0d2ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31961) UID: 0 2024-04-26T03:39:09.486 INFO:teuthology.orchestra.run.smithi028.stdout:osd.6: {} 2024-04-26T03:39:09.487 INFO:teuthology.orchestra.run.smithi028.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T03:39:09.504 INFO:teuthology.orchestra.run.smithi028.stdout:osd.7: {} 2024-04-26T03:39:09.505 INFO:teuthology.orchestra.run.smithi028.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T03:39:09.521 INFO:teuthology.orchestra.run.smithi028.stdout:osd.8: {} 2024-04-26T03:39:09.521 INFO:teuthology.orchestra.run.smithi028.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T03:39:09.540 INFO:teuthology.orchestra.run.smithi028.stdout:osd.9: {} 2024-04-26T03:39:09.540 INFO:teuthology.orchestra.run.smithi028.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T03:39:09.553 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:09.553 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:09.560 INFO:teuthology.orchestra.run.smithi028.stdout:osd.10: {} 2024-04-26T03:39:09.560 INFO:teuthology.orchestra.run.smithi028.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T03:39:09.562 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-26T03:39:09.575 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-04-26T03:39:09.576 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-26T03:39:09.576 INFO:tasks.thrashosds.thrasher:Shrinking pool unique_pool_0 2024-04-26T03:39:09.576 INFO:tasks.ceph.ceph_manager.ceph:contract_pool unique_pool_0 by 10 min 16 2024-04-26T03:39:09.576 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T03:39:09.578 INFO:teuthology.orchestra.run.smithi028.stdout:osd.11: {} 2024-04-26T03:39:09.578 INFO:teuthology.orchestra.run.smithi028.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T03:39:09.596 INFO:teuthology.orchestra.run.smithi028.stdout:osd.12: {} 2024-04-26T03:39:09.597 INFO:teuthology.orchestra.run.smithi028.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T03:39:09.606 INFO:tasks.ceph.osd.10.smithi066.stderr:2024-04-26T03:39:09.605+0000 7f130fcc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 31763) UID: 0 2024-04-26T03:39:09.613 INFO:teuthology.orchestra.run.smithi028.stdout:osd.13: {} 2024-04-26T03:39:09.614 INFO:teuthology.orchestra.run.smithi028.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T03:39:09.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:39:09.613+0000 7f6e86ffd640 0 -- 172.21.15.28:0/1227168705 >> [v2:172.21.15.66:6824/1534905602,v1:172.21.15.66:6825/1534905602] conn(0x7f6e8821b250 msgr2=0x7f6e8821d640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-26T03:39:09.678 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:39:09.679 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-04-26T03:39:09.679 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 0 2024-04-26T03:39:09.679 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:39:09.679 INFO:tasks.ceph.osd.12.smithi028.stderr:2024-04-26T03:39:09.678+0000 7f114fe14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 31967) UID: 0 2024-04-26T03:39:09.687 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-26T03:39:09.708 INFO:tasks.rados.rados.0.smithi092.stdout:228: done (0 left) 2024-04-26T03:39:09.708 INFO:tasks.rados.rados.0.smithi092.stdout:229: rmattr oid 40 current snap is 10 2024-04-26T03:39:09.709 INFO:tasks.rados.rados.0.smithi092.stdout:229: done (0 left) 2024-04-26T03:39:09.709 INFO:tasks.rados.rados.0.smithi092.stdout:230: snap_create 2024-04-26T03:39:09.780 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-26T03:39:09.779+0000 7f610b3a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31816) UID: 0 2024-04-26T03:39:09.832 INFO:teuthology.orchestra.run.smithi028.stdout:osd.14: {} 2024-04-26T03:39:09.833 INFO:teuthology.orchestra.run.smithi028.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T03:39:09.836 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:39:09.836 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-04-26T03:39:09.836 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-04-26T03:39:09.836 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-04-26T03:39:09.836 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:39:09.845 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-26T03:39:09.850 INFO:teuthology.orchestra.run.smithi028.stdout:osd.15: {} 2024-04-26T03:39:09.850 INFO:teuthology.orchestra.run.smithi028.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T03:39:09.876 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:39:09.876 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-04-26T03:39:09.888 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":58,"stamp":"2024-04-26T03:39:09.700808+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":62857760,"num_objects":49,"num_object_clones":1,"num_object_copies":333,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":331,"num_read_kb":51001,"num_write":535,"num_write_kb":51196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":772,"ondisk_log_size":772,"up":200,"acting":200,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":201,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":565136,"kb_used_data":111916,"kb_used_omap":24,"kb_used_meta":452903,"kb_avail":1509384304,"statfs":{"total":1546188226560,"available":1545609527296,"internally_reserved":0,"allocated":114601984,"data_stored":114272248,"data_compressed":900671,"data_compressed_allocated":1900544,"data_compressed_original":4694016,"omap_allocated":25522,"internal_metadata":463772750},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21618688,"num_objects":-12,"num_object_clones":-14,"num_object_copies":-84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-12,"num_whiteouts":-1,"num_read":-13,"num_read_kb":9019,"num_write":3,"num_write_kb":2872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.018260"},"pg_stats":[{"pgid":"3.19","version":"50'20","reported_seq":137,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T03:39:06.739866+0000","last_change":"2024-04-26T03:39:06.343371+0000","last_active":"2024-04-26T03:39:06.739866+0000","last_peered":"2024-04-26T03:39:06.739866+0000","last_clean":"2024-04-26T03:39:06.739866+0000","last_became_active":"2024-04-26T03:39:01.544381+0000","last_became_peered":"2024-04-26T03:39:01.544381+0000","last_unstale":"2024-04-26T03:39:06.739866+0000","last_undegraded":"2024-04-26T03:39:06.739866+0000","last_fullsized":"2024-04-26T03:39:06.739866+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":5,"last_scrub":"50'20","last_scrub_stamp":"2024-04-26T03:39:06.343250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:06.343250+0000","objects_scrubbed":1,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:17.797527+0000","scrub_duration":4,"objects_trimmed":2,"snaptrim_duration":0.0094688140000000007,"stat_sum":{"num_bytes":425984,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2,10,9,15,8,4],"acting":[13,2,10,9,15,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"5","length":"4"}]},{"pgid":"3.18","version":"51'48","reported_seq":168,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T03:39:06.721045+0000","last_change":"2024-04-26T03:39:06.156453+0000","last_active":"2024-04-26T03:39:06.721045+0000","last_peered":"2024-04-26T03:39:06.721045+0000","last_clean":"2024-04-26T03:39:06.721045+0000","last_became_active":"2024-04-26T03:39:01.553702+0000","last_became_peered":"2024-04-26T03:39:01.553702+0000","last_unstale":"2024-04-26T03:39:06.721045+0000","last_undegraded":"2024-04-26T03:39:06.721045+0000","last_fullsized":"2024-04-26T03:39:06.721045+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":5,"last_scrub":"50'43","last_scrub_stamp":"2024-04-26T03:39:06.156322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:06.156322+0000","objects_scrubbed":3,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:18.490269+0000","scrub_duration":3,"objects_trimmed":3,"snaptrim_duration":0.018434015000000002,"stat_sum":{"num_bytes":4341760,"num_objects":4,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,10,11,15,5],"acting":[6,0,1,10,11,15,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"5","length":"4"}]},{"pgid":"3.17","version":"43'6","reported_seq":126,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T03:39:07.700246+0000","last_change":"2024-04-26T03:39:07.700601+0000","last_active":"2024-04-26T03:39:07.700246+0000","last_peered":"2024-04-26T03:39:07.700246+0000","last_clean":"2024-04-26T03:39:07.700246+0000","last_became_active":"2024-04-26T03:39:01.550697+0000","last_became_peered":"2024-04-26T03:39:01.550697+0000","last_unstale":"2024-04-26T03:39:07.700246+0000","last_undegraded":"2024-04-26T03:39:07.700246+0000","last_fullsized":"2024-04-26T03:39:07.700246+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":5,"last_scrub":"43'6","last_scrub_stamp":"2024-04-26T03:39:07.292706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:07.292706+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:11.228640+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00069869099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,13,6,0,14,10,11],"acting":[4,13,6,0,14,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"5","length":"5"}]},{"pgid":"3.16","version":"35'3","reported_seq":124,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T03:39:07.714713+0000","last_change":"2024-04-26T03:39:07.715366+0000","last_active":"2024-04-26T03:39:07.714713+0000","last_peered":"2024-04-26T03:39:07.714713+0000","last_clean":"2024-04-26T03:39:07.714713+0000","last_became_active":"2024-04-26T03:39:01.546594+0000","last_became_peered":"2024-04-26T03:39:01.546594+0000","last_unstale":"2024-04-26T03:39:07.714713+0000","last_undegraded":"2024-04-26T03:39:07.714713+0000","last_fullsized":"2024-04-26T03:39:07.714713+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":5,"last_scrub":"35'3","last_scrub_stamp":"2024-04-26T03:39:06.216135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:06.216135+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:25.080494+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0010693810000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,14,6,2,12,7,1],"acting":[0,14,6,2,12,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"5","length":"5"}]},{"pgid":"3.15","version":"42'24","reported_seq":139,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T03:39:07.185979+0000","last_change":"2024-04-26T03:39:07.186016+0000","last_active":"2024-04-26T03:39:07.185979+0000","last_peered":"2024-04-26T03:39:07.185979+0000","last_clean":"2024-04-26T03:39:07.185979+0000","last_became_active":"2024-04-26T03:39:01.543191+0000","last_became_peered":"2024-04-26T03:39:01.543191+0000","last_unstale":"2024-04-26T03:39:07.185979+0000","last_undegraded":"2024-04-26T03:39:07.185979+0000","last_fullsized":"2024-04-26T03:39:07.185979+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":5,"last_scrub":"42'24","last_scrub_stamp":"2024-04-26T03:39:07.185893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:07.185893+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:35.070439+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00043816599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,8,4,2,7,0,5],"acting":[15,8,4,2,7,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"5","length":"4"}]},{"pgid":"3.14","version":"50'29","reported_seq":152,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T03:39:06.740100+0000","last_change":"2024-04-26T03:39:06.353527+0000","last_active":"2024-04-26T03:39:06.740100+0000","last_peered":"2024-04-26T03:39:06.740100+0000","last_clean":"2024-04-26T03:39:06.740100+0000","last_became_active":"2024-04-26T03:39:01.550254+0000","last_became_peered":"2024-04-26T03:39:01.550254+0000","last_unstale":"2024-04-26T03:39:06.740100+0000","last_undegraded":"2024-04-26T03:39:06.740100+0000","last_fullsized":"2024-04-26T03:39:06.740100+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":5,"last_scrub":"50'29","last_scrub_stamp":"2024-04-26T03:39:06.353377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:06.353377+0000","objects_scrubbed":5,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:25.281623+0000","scrub_duration":4,"objects_trimmed":1,"snaptrim_duration":0.0089029780000000006,"stat_sum":{"num_bytes":5636096,"num_objects":5,"num_object_clones":0,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":577,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,13,4,2,6,10,5],"acting":[8,13,4,2,6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"5","length":"4"}]},{"pgid":"2.7","version":"0'0","reported_seq":58,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T03:39:06.692725+0000","last_change":"2024-04-26T03:38:37.359252+0000","last_active":"2024-04-26T03:39:06.692725+0000","last_peered":"2024-04-26T03:39:06.692725+0000","last_clean":"2024-04-26T03:39:06.692725+0000","last_became_active":"2024-04-26T03:38:37.358910+0000","last_became_peered":"2024-04-26T03:38:37.358910+0000","last_unstale":"2024-04-26T03:39:06.692725+0000","last_undegraded":"2024-04-26T03:39:06.692725+0000","last_fullsized":"2024-04-26T03:39:06.692725+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:04.186001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"3.6","version":"50'9","reported_seq":130,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T03:39:07.715433+0000","last_change":"2024-04-26T03:39:07.715748+0000","last_active":"2024-04-26T03:39:07.715433+0000","last_peered":"2024-04-26T03:39:07.715433+0000","last_clean":"2024-04-26T03:39:07.715433+0000","last_became_active":"2024-04-26T03:39:01.543735+0000","last_became_peered":"2024-04-26T03:39:01.543735+0000","last_unstale":"2024-04-26T03:39:07.715433+0000","last_undegraded":"2024-04-26T03:39:07.715433+0000","last_fullsized":"2024-04-26T03:39:07.715433+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'9","last_scrub_stamp":"2024-04-26T03:39:07.177824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:07.177824+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:29.421879+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0011943170000000001,"stat_sum":{"num_bytes":819200,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,14,6,2,12,7,1],"acting":[0,14,6,2,12,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"5","length":"5"}]},{"pgid":"2.6","version":"0'0","reported_seq":58,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T03:39:06.740389+0000","last_change":"2024-04-26T03:38:36.484782+0000","last_active":"2024-04-26T03:39:06.740389+0000","last_peered":"2024-04-26T03:39:06.740389+0000","last_clean":"2024-04-26T03:39:06.740389+0000","last_became_active":"2024-04-26T03:38:36.484388+0000","last_became_peered":"2024-04-26T03:38:36.484388+0000","last_unstale":"2024-04-26T03:39:06.740389+0000","last_undegraded":"2024-04-26T03:39:06.740389+0000","last_fullsized":"2024-04-26T03:39:06.740389+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:36.318287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,13],"acting":[8,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.7","version":"49'10","reported_seq":123,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T03:39:07.700451+0000","last_change":"2024-04-26T03:39:07.700791+0000","last_active":"2024-04-26T03:39:07.700451+0000","last_peered":"2024-04-26T03:39:07.700451+0000","last_clean":"2024-04-26T03:39:07.700451+0000","last_became_active":"2024-04-26T03:39:01.549431+0000","last_became_peered":"2024-04-26T03:39:01.549431+0000","last_unstale":"2024-04-26T03:39:07.700451+0000","last_undegraded":"2024-04-26T03:39:07.700451+0000","last_fullsized":"2024-04-26T03:39:07.700451+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061997599999999995,"stat_sum":{"num_bytes":65536,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,13,6,0,14,10,11],"acting":[4,13,6,0,14,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"5","length":"5"}]},{"pgid":"2.5","version":"0'0","reported_seq":57,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T03:39:06.694038+0000","last_change":"2024-04-26T03:38:38.187714+0000","last_active":"2024-04-26T03:39:06.694038+0000","last_peered":"2024-04-26T03:39:06.694038+0000","last_clean":"2024-04-26T03:39:06.694038+0000","last_became_active":"2024-04-26T03:38:38.187383+0000","last_became_peered":"2024-04-26T03:38:38.187383+0000","last_unstale":"2024-04-26T03:39:06.694038+0000","last_undegraded":"2024-04-26T03:39:06.694038+0000","last_fullsized":"2024-04-26T03:39:06.694038+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:59.062623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4],"acting":[9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.4","version":"42'18","reported_seq":128,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T03:39:06.740375+0000","last_change":"2024-04-26T03:39:05.686024+0000","last_active":"2024-04-26T03:39:06.740375+0000","last_peered":"2024-04-26T03:39:06.740375+0000","last_clean":"2024-04-26T03:39:06.740375+0000","last_became_active":"2024-04-26T03:39:01.543050+0000","last_became_peered":"2024-04-26T03:39:01.543050+0000","last_unstale":"2024-04-26T03:39:06.740375+0000","last_undegraded":"2024-04-26T03:39:06.740375+0000","last_fullsized":"2024-04-26T03:39:06.740375+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00157,"stat_sum":{"num_bytes":1376256,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":7,"num_write_kb":1768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,13,4,2,6,10,5],"acting":[8,13,4,2,6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"5","length":"4"}]},{"pgid":"2.4","version":"0'0","reported_seq":59,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T03:39:07.699646+0000","last_change":"2024-04-26T03:38:37.255389+0000","last_active":"2024-04-26T03:39:07.699646+0000","last_peered":"2024-04-26T03:39:07.699646+0000","last_clean":"2024-04-26T03:39:07.699646+0000","last_became_active":"2024-04-26T03:38:37.255066+0000","last_became_peered":"2024-04-26T03:38:37.255066+0000","last_unstale":"2024-04-26T03:39:07.699646+0000","last_undegraded":"2024-04-26T03:39:07.699646+0000","last_fullsized":"2024-04-26T03:39:07.699646+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:43.860480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.5","version":"50'51","reported_seq":163,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T03:39:06.692802+0000","last_change":"2024-04-26T03:39:06.149862+0000","last_active":"2024-04-26T03:39:06.692802+0000","last_peered":"2024-04-26T03:39:06.692802+0000","last_clean":"2024-04-26T03:39:06.692802+0000","last_became_active":"2024-04-26T03:39:01.540394+0000","last_became_peered":"2024-04-26T03:39:01.540394+0000","last_unstale":"2024-04-26T03:39:06.692802+0000","last_undegraded":"2024-04-26T03:39:06.692802+0000","last_fullsized":"2024-04-26T03:39:06.692802+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'51","last_scrub_stamp":"2024-04-26T03:39:06.149763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:06.149763+0000","objects_scrubbed":1,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:29.746233+0000","scrub_duration":3,"objects_trimmed":1,"snaptrim_duration":0.0057982800000000003,"stat_sum":{"num_bytes":2605056,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,8,4,2,7,0,5],"acting":[15,8,4,2,7,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"5","length":"4"}]},{"pgid":"3.3","version":"50'32","reported_seq":149,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T03:39:06.689282+0000","last_change":"2024-04-26T03:39:05.694594+0000","last_active":"2024-04-26T03:39:06.689282+0000","last_peered":"2024-04-26T03:39:06.689282+0000","last_clean":"2024-04-26T03:39:06.689282+0000","last_became_active":"2024-04-26T03:39:01.550287+0000","last_became_peered":"2024-04-26T03:39:01.550287+0000","last_unstale":"2024-04-26T03:39:06.689282+0000","last_undegraded":"2024-04-26T03:39:06.689282+0000","last_fullsized":"2024-04-26T03:39:06.689282+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0088994090000000005,"stat_sum":{"num_bytes":3825664,"num_objects":3,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":3533,"num_write":13,"num_write_kb":3736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,7,4,15,1,2,3],"acting":[10,7,4,15,1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"5","length":"4"}]},{"pgid":"2.2","version":"0'0","reported_seq":46,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T03:39:05.701191+0000","last_change":"2024-04-26T03:38:48.475569+0000","last_active":"2024-04-26T03:39:05.701191+0000","last_peered":"2024-04-26T03:39:05.701191+0000","last_clean":"2024-04-26T03:39:05.701191+0000","last_became_active":"2024-04-26T03:38:48.475155+0000","last_became_peered":"2024-04-26T03:38:48.475155+0000","last_unstale":"2024-04-26T03:39:05.701191+0000","last_undegraded":"2024-04-26T03:39:05.701191+0000","last_fullsized":"2024-04-26T03:39:05.701191+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:38.214686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,12],"acting":[11,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.0","version":"35'9","reported_seq":132,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T03:39:07.694808+0000","last_change":"2024-04-26T03:39:07.695314+0000","last_active":"2024-04-26T03:39:07.694808+0000","last_peered":"2024-04-26T03:39:07.694808+0000","last_clean":"2024-04-26T03:39:07.694808+0000","last_became_active":"2024-04-26T03:39:01.540712+0000","last_became_peered":"2024-04-26T03:39:01.540712+0000","last_unstale":"2024-04-26T03:39:07.694808+0000","last_undegraded":"2024-04-26T03:39:07.694808+0000","last_fullsized":"2024-04-26T03:39:07.694808+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'9","last_scrub_stamp":"2024-04-26T03:39:06.119088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:06.119088+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:26.193662+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00075975399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,9,15,2,6,14,4],"acting":[1,9,15,2,6,14,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"5","length":"5"}]},{"pgid":"2.1","version":"0'0","reported_seq":58,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T03:39:06.694030+0000","last_change":"2024-04-26T03:38:37.255450+0000","last_active":"2024-04-26T03:39:06.694030+0000","last_peered":"2024-04-26T03:39:06.694030+0000","last_clean":"2024-04-26T03:39:06.694030+0000","last_became_active":"2024-04-26T03:38:37.255181+0000","last_became_peered":"2024-04-26T03:38:37.255181+0000","last_unstale":"2024-04-26T03:39:06.694030+0000","last_undegraded":"2024-04-26T03:39:06.694030+0000","last_fullsized":"2024-04-26T03:39:06.694030+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:00.449388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0],"acting":[14,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.1","version":"42'8","reported_seq":128,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T03:39:08.263969+0000","last_change":"2024-04-26T03:39:08.263995+0000","last_active":"2024-04-26T03:39:08.263969+0000","last_peered":"2024-04-26T03:39:08.263969+0000","last_clean":"2024-04-26T03:39:08.263969+0000","last_became_active":"2024-04-26T03:39:01.547451+0000","last_became_peered":"2024-04-26T03:39:01.547451+0000","last_unstale":"2024-04-26T03:39:08.263969+0000","last_undegraded":"2024-04-26T03:39:08.263969+0000","last_fullsized":"2024-04-26T03:39:08.263969+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"42'8","last_scrub_stamp":"2024-04-26T03:39:08.263903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:08.263903+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:37.907097+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00049398599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,14,10,9,1,2],"acting":[4,11,14,10,9,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"5","length":"5"}]},{"pgid":"2.0","version":"0'0","reported_seq":61,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T03:39:08.704286+0000","last_change":"2024-04-26T03:38:36.431325+0000","last_active":"2024-04-26T03:39:08.704286+0000","last_peered":"2024-04-26T03:39:08.704286+0000","last_clean":"2024-04-26T03:39:08.704286+0000","last_became_active":"2024-04-26T03:38:36.431121+0000","last_became_peered":"2024-04-26T03:38:36.431121+0000","last_unstale":"2024-04-26T03:39:08.704286+0000","last_undegraded":"2024-04-26T03:39:08.704286+0000","last_fullsized":"2024-04-26T03:39:08.704286+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:53.294619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"50'31","reported_seq":145,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T03:39:06.693336+0000","last_change":"2024-04-26T03:39:05.706338+0000","last_active":"2024-04-26T03:39:06.693336+0000","last_peered":"2024-04-26T03:39:06.693336+0000","last_clean":"2024-04-26T03:39:06.693336+0000","last_became_active":"2024-04-26T03:39:01.543967+0000","last_became_peered":"2024-04-26T03:39:01.543967+0000","last_unstale":"2024-04-26T03:39:06.693336+0000","last_undegraded":"2024-04-26T03:39:06.693336+0000","last_fullsized":"2024-04-26T03:39:06.693336+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.010062454,"stat_sum":{"num_bytes":5169152,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":4,"num_write":35,"num_write_kb":6720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,5,3,9,15,10,11],"acting":[14,5,3,9,15,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"5","length":"4"}]},{"pgid":"2.3","version":"0'0","reported_seq":58,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T03:39:06.693238+0000","last_change":"2024-04-26T03:38:36.431667+0000","last_active":"2024-04-26T03:39:06.693238+0000","last_peered":"2024-04-26T03:39:06.693238+0000","last_clean":"2024-04-26T03:39:06.693238+0000","last_became_active":"2024-04-26T03:38:36.431449+0000","last_became_peered":"2024-04-26T03:38:36.431449+0000","last_unstale":"2024-04-26T03:39:06.693238+0000","last_undegraded":"2024-04-26T03:39:06.693238+0000","last_fullsized":"2024-04-26T03:39:06.693238+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:47.562648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"27'157","reported_seq":259,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T03:39:06.703561+0000","last_change":"2024-04-26T03:38:34.390206+0000","last_active":"2024-04-26T03:39:06.703561+0000","last_peered":"2024-04-26T03:39:06.703561+0000","last_clean":"2024-04-26T03:39:06.703561+0000","last_became_active":"2024-04-26T03:38:34.389099+0000","last_became_peered":"2024-04-26T03:38:34.389099+0000","last_unstale":"2024-04-26T03:39:06.703561+0000","last_undegraded":"2024-04-26T03:39:06.703561+0000","last_fullsized":"2024-04-26T03:39:06.703561+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:32.306810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:32.306810+0000","last_clean_scrub_stamp":"2024-04-26T03:38:32.306810+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:40.216975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.8","version":"42'12","reported_seq":129,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T03:39:06.703534+0000","last_change":"2024-04-26T03:39:05.695492+0000","last_active":"2024-04-26T03:39:06.703534+0000","last_peered":"2024-04-26T03:39:06.703534+0000","last_clean":"2024-04-26T03:39:06.703534+0000","last_became_active":"2024-04-26T03:39:01.546346+0000","last_became_peered":"2024-04-26T03:39:01.546346+0000","last_unstale":"2024-04-26T03:39:06.703534+0000","last_undegraded":"2024-04-26T03:39:06.703534+0000","last_fullsized":"2024-04-26T03:39:06.703534+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004749072,"stat_sum":{"num_bytes":983040,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":963,"num_write":4,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,10,11,15,5],"acting":[6,0,1,10,11,15,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"5","length":"4"}]},{"pgid":"3.9","version":"35'6","reported_seq":121,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T03:39:06.739877+0000","last_change":"2024-04-26T03:39:05.699102+0000","last_active":"2024-04-26T03:39:06.739877+0000","last_peered":"2024-04-26T03:39:06.739877+0000","last_clean":"2024-04-26T03:39:06.739877+0000","last_became_active":"2024-04-26T03:39:01.541605+0000","last_became_peered":"2024-04-26T03:39:01.541605+0000","last_unstale":"2024-04-26T03:39:06.739877+0000","last_undegraded":"2024-04-26T03:39:06.739877+0000","last_fullsized":"2024-04-26T03:39:06.739877+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0077164019999999998,"stat_sum":{"num_bytes":614400,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":2211,"num_write":3,"num_write_kb":600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2,10,9,15,8,4],"acting":[13,2,10,9,15,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"5","length":"4"}]},{"pgid":"3.a","version":"50'44","reported_seq":180,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T03:39:08.705273+0000","last_change":"2024-04-26T03:39:07.703520+0000","last_active":"2024-04-26T03:39:08.705273+0000","last_peered":"2024-04-26T03:39:08.705273+0000","last_clean":"2024-04-26T03:39:08.705273+0000","last_became_active":"2024-04-26T03:38:49.486365+0000","last_became_peered":"2024-04-26T03:38:49.486365+0000","last_unstale":"2024-04-26T03:39:08.705273+0000","last_undegraded":"2024-04-26T03:39:08.705273+0000","last_fullsized":"2024-04-26T03:39:08.705273+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:51.988955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067771699999999999,"stat_sum":{"num_bytes":7061504,"num_objects":6,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":50,"num_read_kb":15714,"num_write":49,"num_write_kb":8672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,10,13,1,11],"acting":[3,7,6,10,13,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"5","length":"5"}]},{"pgid":"3.b","version":"52'53","reported_seq":168,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T03:39:08.008174+0000","last_change":"2024-04-26T03:39:07.700277+0000","last_active":"2024-04-26T03:39:08.008174+0000","last_peered":"2024-04-26T03:39:08.008174+0000","last_clean":"2024-04-26T03:39:08.008174+0000","last_became_active":"2024-04-26T03:38:49.940561+0000","last_became_peered":"2024-04-26T03:38:49.940561+0000","last_unstale":"2024-04-26T03:39:08.008174+0000","last_undegraded":"2024-04-26T03:39:08.008174+0000","last_fullsized":"2024-04-26T03:39:08.008174+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:01.637777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000510053,"stat_sum":{"num_bytes":4145152,"num_objects":4,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":28,"num_read_kb":6541,"num_write":91,"num_write_kb":7200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,10,12,1,7,14],"acting":[4,0,10,12,1,7,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"5","length":"5"}]},{"pgid":"3.c","version":"50'39","reported_seq":133,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T03:39:05.708666+0000","last_change":"2024-04-26T03:39:05.709704+0000","last_active":"2024-04-26T03:39:05.708666+0000","last_peered":"2024-04-26T03:39:05.708666+0000","last_clean":"2024-04-26T03:39:05.708666+0000","last_became_active":"2024-04-26T03:38:49.941017+0000","last_became_peered":"2024-04-26T03:38:49.941017+0000","last_unstale":"2024-04-26T03:39:05.708666+0000","last_undegraded":"2024-04-26T03:39:05.708666+0000","last_fullsized":"2024-04-26T03:39:05.708666+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:10.977890+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0088329659999999994,"stat_sum":{"num_bytes":5586944,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":5136,"num_write":37,"num_write_kb":5696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,0,1,9,12,14],"acting":[11,8,0,1,9,12,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"5","length":"4"}]},{"pgid":"3.d","version":"50'23","reported_seq":118,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T03:39:06.740057+0000","last_change":"2024-04-26T03:39:05.695786+0000","last_active":"2024-04-26T03:39:06.740057+0000","last_peered":"2024-04-26T03:39:06.740057+0000","last_clean":"2024-04-26T03:39:06.740057+0000","last_became_active":"2024-04-26T03:38:49.511783+0000","last_became_peered":"2024-04-26T03:38:49.511783+0000","last_unstale":"2024-04-26T03:39:06.740057+0000","last_undegraded":"2024-04-26T03:39:06.740057+0000","last_fullsized":"2024-04-26T03:39:06.740057+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:05.236982+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0086157260000000006,"stat_sum":{"num_bytes":606208,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":3,"num_write":29,"num_write_kb":880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,9,1,5,8,4],"acting":[7,3,9,1,5,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"5","length":"4"}]},{"pgid":"3.e","version":"50'40","reported_seq":150,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T03:39:06.740070+0000","last_change":"2024-04-26T03:39:05.698125+0000","last_active":"2024-04-26T03:39:06.740070+0000","last_peered":"2024-04-26T03:39:06.740070+0000","last_clean":"2024-04-26T03:39:06.740070+0000","last_became_active":"2024-04-26T03:38:49.941260+0000","last_became_peered":"2024-04-26T03:38:49.941260+0000","last_unstale":"2024-04-26T03:39:06.740070+0000","last_undegraded":"2024-04-26T03:39:06.740070+0000","last_fullsized":"2024-04-26T03:39:06.740070+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:57.000160+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0073978990000000003,"stat_sum":{"num_bytes":6029312,"num_objects":3,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":29,"num_read_kb":12540,"num_write":23,"num_write_kb":6208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,14,6,11,8,9,12],"acting":[13,14,6,11,8,9,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"5","length":"4"}]},{"pgid":"3.f","version":"50'32","reported_seq":133,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T03:39:05.710514+0000","last_change":"2024-04-26T03:39:05.711577+0000","last_active":"2024-04-26T03:39:05.710514+0000","last_peered":"2024-04-26T03:39:05.710514+0000","last_clean":"2024-04-26T03:39:05.710514+0000","last_became_active":"2024-04-26T03:38:49.486850+0000","last_became_peered":"2024-04-26T03:38:49.486850+0000","last_unstale":"2024-04-26T03:39:05.710514+0000","last_undegraded":"2024-04-26T03:39:05.710514+0000","last_fullsized":"2024-04-26T03:39:05.710514+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:52.225320+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.010058294000000001,"stat_sum":{"num_bytes":2490368,"num_objects":3,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":3655,"num_write":35,"num_write_kb":3680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,13,5,14,2,1,0],"acting":[11,13,5,14,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"5","length":"4"}]},{"pgid":"3.10","version":"35'9","reported_seq":58,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T03:39:07.110244+0000","last_change":"2024-04-26T03:39:07.110244+0000","last_active":"2024-04-26T03:39:07.110244+0000","last_peered":"2024-04-26T03:39:07.110244+0000","last_clean":"2024-04-26T03:39:07.110244+0000","last_became_active":"2024-04-26T03:39:04.704276+0000","last_became_peered":"2024-04-26T03:39:04.704276+0000","last_unstale":"2024-04-26T03:39:07.110244+0000","last_undegraded":"2024-04-26T03:39:07.110244+0000","last_fullsized":"2024-04-26T03:39:07.110244+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":5,"last_scrub":"35'9","last_scrub_stamp":"2024-04-26T03:39:07.110098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:07.110098+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:26.150021+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0033218890000000002,"stat_sum":{"num_bytes":1015808,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,7,4,8,0,11],"acting":[6,9,7,4,8,0,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"5","length":"4"}]},{"pgid":"3.11","version":"42'8","reported_seq":43,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T03:39:08.698721+0000","last_change":"2024-04-26T03:39:08.233857+0000","last_active":"2024-04-26T03:39:08.698721+0000","last_peered":"2024-04-26T03:39:08.698721+0000","last_clean":"2024-04-26T03:39:08.698721+0000","last_became_active":"2024-04-26T03:39:07.719054+0000","last_became_peered":"2024-04-26T03:39:07.719054+0000","last_unstale":"2024-04-26T03:39:08.698721+0000","last_undegraded":"2024-04-26T03:39:08.698721+0000","last_fullsized":"2024-04-26T03:39:08.698721+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":5,"last_scrub":"42'8","last_scrub_stamp":"2024-04-26T03:39:08.233750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:08.233750+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:24.269626+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.000434886,"stat_sum":{"num_bytes":2621440,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,15,10,0,14,2],"acting":[5,11,15,10,0,14,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"5","length":"5"}]},{"pgid":"3.12","version":"35'11","reported_seq":128,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T03:39:06.693532+0000","last_change":"2024-04-26T03:39:06.494118+0000","last_active":"2024-04-26T03:39:06.693532+0000","last_peered":"2024-04-26T03:39:06.693532+0000","last_clean":"2024-04-26T03:39:06.693532+0000","last_became_active":"2024-04-26T03:39:01.545694+0000","last_became_peered":"2024-04-26T03:39:01.545694+0000","last_unstale":"2024-04-26T03:39:06.693532+0000","last_undegraded":"2024-04-26T03:39:06.693532+0000","last_fullsized":"2024-04-26T03:39:06.693532+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":5,"last_scrub":"35'11","last_scrub_stamp":"2024-04-26T03:39:06.494007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:06.494007+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:15.517716+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.00099163700000000007,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,5,3,9,15,10,11],"acting":[14,5,3,9,15,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"5","length":"4"}]},{"pgid":"3.13","version":"50'40","reported_seq":168,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T03:39:06.689159+0000","last_change":"2024-04-26T03:39:06.466204+0000","last_active":"2024-04-26T03:39:06.689159+0000","last_peered":"2024-04-26T03:39:06.689159+0000","last_clean":"2024-04-26T03:39:06.689159+0000","last_became_active":"2024-04-26T03:39:01.561931+0000","last_became_peered":"2024-04-26T03:39:01.561931+0000","last_unstale":"2024-04-26T03:39:06.689159+0000","last_undegraded":"2024-04-26T03:39:06.689159+0000","last_fullsized":"2024-04-26T03:39:06.689159+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":5,"last_scrub":"50'40","last_scrub_stamp":"2024-04-26T03:39:06.466060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:06.466060+0000","objects_scrubbed":4,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:22.901000+0000","scrub_duration":4,"objects_trimmed":1,"snaptrim_duration":0.0093714220000000008,"stat_sum":{"num_bytes":6848512,"num_objects":4,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,7,4,15,1,2,3],"acting":[10,7,4,15,1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"5","length":"4"}]}],"pool_stats":[{"poolid":3,"num_pg":26,"stat_sum":{"num_bytes":62267392,"num_objects":47,"num_object_clones":1,"num_object_copies":329,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":195,"num_read_kb":50883,"num_write":332,"num_write_kb":47736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106049536,"data_stored":107118592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":615,"ondisk_log_size":615,"up":182,"acting":182,"num_store_stats":16},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":13852,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":24,"seq":103079215116,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40044,"kb_used_data":8864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331796,"statfs":{"total":96636764160,"available":96595759104,"internally_reserved":0,"allocated":9076736,"data_stored":9352643,"data_compressed":66393,"data_compressed_allocated":155648,"data_compressed_original":757760,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[4,6,7,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215116,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33636,"kb_used_data":6616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338204,"statfs":{"total":96636764160,"available":96602320896,"internally_reserved":0,"allocated":6774784,"data_stored":6577029,"data_compressed":57448,"data_compressed_allocated":118784,"data_compressed_original":237568,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":23,"seq":98784247821,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38484,"kb_used_data":7300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94333356,"statfs":{"total":96636764160,"available":96597356544,"internally_reserved":0,"allocated":7475200,"data_stored":7277445,"data_compressed":57448,"data_compressed_allocated":118784,"data_compressed_original":237568,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280522,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33724,"kb_used_data":6700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338116,"statfs":{"total":96636764160,"available":96602230784,"internally_reserved":0,"allocated":6860800,"data_stored":7497031,"data_compressed":62353,"data_compressed_allocated":147456,"data_compressed_original":741376,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,4,5,7,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280522,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33348,"kb_used_data":6316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338492,"statfs":{"total":96636764160,"available":96602615808,"internally_reserved":0,"allocated":6467584,"data_stored":6271783,"data_compressed":55427,"data_compressed_allocated":114688,"data_compressed_original":229376,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,3,4,5,6,8,9,10,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":8,"up_from":22,"seq":94489280523,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33032,"kb_used_data":6004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338808,"statfs":{"total":96636764160,"available":96602939392,"internally_reserved":0,"allocated":6148096,"data_stored":5954313,"data_compressed":53404,"data_compressed_allocated":110592,"data_compressed_original":221184,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[2,4,5,6,7,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280521,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33056,"kb_used_data":6028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":6172672,"data_stored":5978889,"data_compressed":53404,"data_compressed_allocated":110592,"data_compressed_original":221184,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,2,4,7,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280522,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36452,"kb_used_data":9424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94335388,"statfs":{"total":96636764160,"available":96599437312,"internally_reserved":0,"allocated":9650176,"data_stored":9456393,"data_compressed":53404,"data_compressed_allocated":110592,"data_compressed_original":221184,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[1,2,3,4,7,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313227,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35228,"kb_used_data":8204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94336612,"statfs":{"total":96636764160,"available":96600690688,"internally_reserved":0,"allocated":8400896,"data_stored":9347763,"data_compressed":51384,"data_compressed_allocated":106496,"data_compressed_original":212992,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,5,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":12,"up_from":22,"seq":94489280521,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31212,"kb_used_data":4180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340628,"statfs":{"total":96636764160,"available":96604803072,"internally_reserved":0,"allocated":4280320,"data_stored":4084519,"data_compressed":55427,"data_compressed_allocated":114688,"data_compressed_original":229376,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,7,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280521,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33000,"kb_used_data":5972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338840,"statfs":{"total":96636764160,"available":96602972160,"internally_reserved":0,"allocated":6115328,"data_stored":5921545,"data_compressed":53404,"data_compressed_allocated":110592,"data_compressed_original":221184,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[2,4,6,7,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247819,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38312,"kb_used_data":7128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94333528,"statfs":{"total":96636764160,"available":96597532672,"internally_reserved":0,"allocated":7299072,"data_stored":7101317,"data_compressed":57448,"data_compressed_allocated":118784,"data_compressed_original":237568,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,6,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280521,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33872,"kb_used_data":6844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337968,"statfs":{"total":96636764160,"available":96602079232,"internally_reserved":0,"allocated":7008256,"data_stored":6814473,"data_compressed":53404,"data_compressed_allocated":110592,"data_compressed_original":221184,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,3,5,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":23,"seq":98784247821,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40168,"kb_used_data":8984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331672,"statfs":{"total":96636764160,"available":96595632128,"internally_reserved":0,"allocated":9199616,"data_stored":9001861,"data_compressed":57448,"data_compressed_allocated":118784,"data_compressed_original":237568,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,4,6,7,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280521,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33112,"kb_used_data":6080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338728,"statfs":{"total":96636764160,"available":96602857472,"internally_reserved":0,"allocated":6225920,"data_stored":6386471,"data_compressed":55427,"data_compressed_allocated":114688,"data_compressed_original":229376,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,4,5,7,8,9,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247819,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38456,"kb_used_data":7272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94333384,"statfs":{"total":96636764160,"available":96597385216,"internally_reserved":0,"allocated":7446528,"data_stored":7248773,"data_compressed":57448,"data_compressed_allocated":118784,"data_compressed_original":237568,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6397952,"data_stored":6397952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":7987200,"data_stored":7987200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6885376,"data_stored":6885376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6213632,"data_stored":6213632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":7368704,"data_stored":7368704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8450048,"data_stored":8450048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":6258688,"data_stored":6615040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":5918720,"data_stored":5918720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":5611520,"data_stored":5611520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":5636096,"data_stored":5636096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":9510912,"data_stored":9867264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8351744,"data_stored":8351744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":3731456,"data_stored":3731456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":5578752,"data_stored":5578752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":6471680,"data_stored":6471680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":5677056,"data_stored":6033408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T03:39:09.890 INFO:tasks.ceph.ceph_manager.ceph:decrease pool size by 10 2024-04-26T03:39:09.890 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set unique_pool_0 pg_num 16 2024-04-26T03:39:09.891 INFO:tasks.ceph.osd.14.smithi066.stderr:2024-04-26T03:39:09.888+0000 7f5395f08640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 31765) UID: 0 2024-04-26T03:39:09.954 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:39:09.955 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-04-26T03:39:09.955 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-04-26T03:39:09.955 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-04-26T03:39:09.955 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.955 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.9s1 35/32 ECSubWrite(tid=20, reqid=client.4556.0:97, at_version=35'5, trim_to=0'0, roll_forward_to=35'4))", 2024-04-26T03:39:09.955 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.555957+0000", 2024-04-26T03:39:09.955 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 18.396367262999998, 2024-04-26T03:39:09.955 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.006868023, 2024-04-26T03:39:09.955 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:09.955 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:09.955 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:09.955 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:09.955 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.956 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:09.956 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.555957+0000", 2024-04-26T03:39:09.956 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:09.956 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.956 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.956 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:09.956 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.555957+0000", 2024-04-26T03:39:09.956 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:09.956 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.956 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.956 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:09.956 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.555964+0000", 2024-04-26T03:39:09.956 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.7360000000000001e-06 2024-04-26T03:39:09.956 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.957 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.957 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:09.957 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.556167+0000", 2024-04-26T03:39:09.957 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000202986 2024-04-26T03:39:09.957 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.957 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.957 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:09.957 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.556183+0000", 2024-04-26T03:39:09.957 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6175999999999999e-05 2024-04-26T03:39:09.957 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.957 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.957 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:09.957 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.556191+0000", 2024-04-26T03:39:09.957 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9899999999999997e-06 2024-04-26T03:39:09.958 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.958 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.958 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:09.958 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.556259+0000", 2024-04-26T03:39:09.958 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7986999999999998e-05 2024-04-26T03:39:09.958 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.958 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.958 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:09.958 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.556280+0000", 2024-04-26T03:39:09.958 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0392999999999999e-05 2024-04-26T03:39:09.958 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.958 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.958 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:09.959 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.562792+0000", 2024-04-26T03:39:09.959 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0065119560000000002 2024-04-26T03:39:09.959 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.959 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.959 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:09.959 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.562825+0000", 2024-04-26T03:39:09.959 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.2799000000000002e-05 2024-04-26T03:39:09.959 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:09.959 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:09.959 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:09.959 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.959 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.959 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.4s3 35/32 ECSubWrite(tid=12, reqid=client.4556.0:101, at_version=35'8, trim_to=0'0, roll_forward_to=35'6))", 2024-04-26T03:39:09.960 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.564690+0000", 2024-04-26T03:39:09.960 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 18.387633374, 2024-04-26T03:39:09.960 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0069190800000000002, 2024-04-26T03:39:09.960 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:09.960 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:09.960 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:09.960 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:09.960 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.960 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:09.960 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.564690+0000", 2024-04-26T03:39:09.960 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:09.960 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.960 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.960 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:09.961 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.564690+0000", 2024-04-26T03:39:09.961 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:09.961 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.961 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.961 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:09.961 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.564694+0000", 2024-04-26T03:39:09.961 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.6449999999999998e-06 2024-04-26T03:39:09.961 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.961 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.961 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:09.961 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.564710+0000", 2024-04-26T03:39:09.961 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5979999999999999e-05 2024-04-26T03:39:09.961 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.961 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.962 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:09.962 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.564725+0000", 2024-04-26T03:39:09.962 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4876e-05 2024-04-26T03:39:09.962 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.962 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.962 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:09.962 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.564734+0000", 2024-04-26T03:39:09.962 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7220000000000005e-06 2024-04-26T03:39:09.962 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.962 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.962 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:09.962 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.565327+0000", 2024-04-26T03:39:09.962 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000592849 2024-04-26T03:39:09.962 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.963 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.963 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:09.963 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.565334+0000", 2024-04-26T03:39:09.963 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9270000000000005e-06 2024-04-26T03:39:09.963 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.963 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.963 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:09.963 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.571566+0000", 2024-04-26T03:39:09.963 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0062319070000000001 2024-04-26T03:39:09.963 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.963 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.963 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:09.963 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.571610+0000", 2024-04-26T03:39:09.963 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.3173999999999997e-05 2024-04-26T03:39:09.963 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:09.965 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:09.965 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:09.965 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.965 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.965 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.4s3 35/32 ECSubWrite(tid=13, reqid=client.4556.0:106, at_version=35'9, trim_to=0'0, roll_forward_to=35'6))", 2024-04-26T03:39:09.965 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.567072+0000", 2024-04-26T03:39:09.965 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 18.385252218000002, 2024-04-26T03:39:09.965 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0074599230000000002, 2024-04-26T03:39:09.965 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:09.966 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:09.966 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:09.966 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:09.966 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.966 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:09.966 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.567072+0000", 2024-04-26T03:39:09.966 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:09.966 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.966 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.966 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:09.966 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.567072+0000", 2024-04-26T03:39:09.966 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:09.966 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.966 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.967 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:09.967 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.567072+0000", 2024-04-26T03:39:09.967 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.8100000000000003e-07 2024-04-26T03:39:09.967 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.967 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.967 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:09.967 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.567302+0000", 2024-04-26T03:39:09.967 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000229457 2024-04-26T03:39:09.967 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.967 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.967 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:09.967 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.567315+0000", 2024-04-26T03:39:09.967 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2845e-05 2024-04-26T03:39:09.967 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.968 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.968 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:09.968 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.567322+0000", 2024-04-26T03:39:09.968 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8979999999999996e-06 2024-04-26T03:39:09.968 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.968 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.968 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:09.968 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.567396+0000", 2024-04-26T03:39:09.968 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3398000000000004e-05 2024-04-26T03:39:09.968 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.968 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.968 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:09.968 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.567413+0000", 2024-04-26T03:39:09.969 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7e-05 2024-04-26T03:39:09.969 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.969 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.969 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:09.969 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.574497+0000", 2024-04-26T03:39:09.969 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0070843640000000001 2024-04-26T03:39:09.969 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.969 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.969 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:09.969 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.574532+0000", 2024-04-26T03:39:09.969 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.4379999999999999e-05 2024-04-26T03:39:09.969 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:09.969 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:09.969 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:09.970 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.970 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.970 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.6s3 35/32 ECSubWrite(tid=0, reqid=client.4556.0:146, at_version=35'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:39:09.970 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.630308+0000", 2024-04-26T03:39:09.970 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 18.322015537999999, 2024-04-26T03:39:09.970 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0078713510000000004, 2024-04-26T03:39:09.970 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:09.970 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:09.970 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:09.970 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:09.970 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.970 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:09.970 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.630308+0000", 2024-04-26T03:39:09.971 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:09.971 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.971 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.971 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:09.971 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.630308+0000", 2024-04-26T03:39:09.971 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:09.971 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.971 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.971 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:09.971 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.630309+0000", 2024-04-26T03:39:09.971 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.9500000000000002e-07 2024-04-26T03:39:09.971 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.971 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.971 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:09.972 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.630752+0000", 2024-04-26T03:39:09.972 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00044313600000000001 2024-04-26T03:39:09.972 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.972 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.972 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:09.972 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.630769+0000", 2024-04-26T03:39:09.972 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6541999999999998e-05 2024-04-26T03:39:09.972 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.972 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.972 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:09.972 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.630777+0000", 2024-04-26T03:39:09.972 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.3410000000000003e-06 2024-04-26T03:39:09.972 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.972 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.973 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:09.973 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.630840+0000", 2024-04-26T03:39:09.973 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.3303e-05 2024-04-26T03:39:09.973 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.973 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.973 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:09.973 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.630848+0000", 2024-04-26T03:39:09.973 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2630000000000003e-06 2024-04-26T03:39:09.973 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.973 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.973 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:09.973 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.638146+0000", 2024-04-26T03:39:09.973 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0072979819999999997 2024-04-26T03:39:09.973 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.973 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.974 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:09.974 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.638180+0000", 2024-04-26T03:39:09.974 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.3188999999999997e-05 2024-04-26T03:39:09.974 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:09.974 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:09.974 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:09.974 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.974 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.974 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.6s3 35/32 ECSubWrite(tid=2, reqid=client.4556.0:148, at_version=35'3, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:39:09.974 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.636626+0000", 2024-04-26T03:39:09.974 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 18.315697849999999, 2024-04-26T03:39:09.974 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0066727840000000002, 2024-04-26T03:39:09.974 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:09.975 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:09.975 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:09.975 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:09.975 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.975 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:09.975 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.636626+0000", 2024-04-26T03:39:09.975 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:09.975 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.975 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.975 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:09.975 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.636626+0000", 2024-04-26T03:39:09.975 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:09.975 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.975 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.976 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:09.976 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.636627+0000", 2024-04-26T03:39:09.976 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7299999999999995e-07 2024-04-26T03:39:09.976 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.976 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.976 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:09.976 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.636643+0000", 2024-04-26T03:39:09.976 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6765999999999998e-05 2024-04-26T03:39:09.976 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.976 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.976 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:09.976 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.636659+0000", 2024-04-26T03:39:09.976 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5831000000000001e-05 2024-04-26T03:39:09.976 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.977 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.977 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:09.977 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.636668+0000", 2024-04-26T03:39:09.977 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5280000000000003e-06 2024-04-26T03:39:09.977 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.977 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.977 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:09.977 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.637982+0000", 2024-04-26T03:39:09.977 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001314684 2024-04-26T03:39:09.977 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.977 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.977 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:09.977 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.637990+0000", 2024-04-26T03:39:09.977 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5279999999999998e-06 2024-04-26T03:39:09.978 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.978 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.978 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:09.978 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.643262+0000", 2024-04-26T03:39:09.978 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0052716009999999999 2024-04-26T03:39:09.978 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.978 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.978 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:09.978 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.643299+0000", 2024-04-26T03:39:09.978 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7172999999999998e-05 2024-04-26T03:39:09.978 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:09.978 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:09.978 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:09.978 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.979 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.979 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.4s3 40/32 ECSubWrite(tid=22, reqid=client.4556.0:262, at_version=40'14, trim_to=0'0, roll_forward_to=35'12))", 2024-04-26T03:39:09.979 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:56.520557+0000", 2024-04-26T03:39:09.979 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 13.431766493, 2024-04-26T03:39:09.979 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.015564889, 2024-04-26T03:39:09.979 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:09.979 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:09.979 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:09.979 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:09.979 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.979 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:09.979 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.520557+0000", 2024-04-26T03:39:09.979 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:09.979 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.980 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.980 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:09.980 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.520557+0000", 2024-04-26T03:39:09.980 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:09.980 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.980 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.980 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:09.980 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.520558+0000", 2024-04-26T03:39:09.980 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7399999999999999e-07 2024-04-26T03:39:09.980 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.980 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.980 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:09.980 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.520818+0000", 2024-04-26T03:39:09.980 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00026045899999999999 2024-04-26T03:39:09.980 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.981 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.981 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:09.981 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.520829+0000", 2024-04-26T03:39:09.981 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0701e-05 2024-04-26T03:39:09.981 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.981 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.981 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:09.981 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.520836+0000", 2024-04-26T03:39:09.981 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.9199999999999998e-06 2024-04-26T03:39:09.981 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.981 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.981 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:09.981 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.520901+0000", 2024-04-26T03:39:09.981 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5327000000000007e-05 2024-04-26T03:39:09.982 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.982 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.982 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:09.982 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.520912+0000", 2024-04-26T03:39:09.982 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0747000000000001e-05 2024-04-26T03:39:09.982 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.982 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.982 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:09.982 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.536082+0000", 2024-04-26T03:39:09.982 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.015169775 2024-04-26T03:39:09.982 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.982 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.982 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:09.982 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.536122+0000", 2024-04-26T03:39:09.983 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.0586000000000001e-05 2024-04-26T03:39:09.983 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:09.983 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:09.983 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:09.983 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.983 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.983 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.4s3 40/32 ECSubWrite(tid=23, reqid=client.4556.0:263, at_version=40'15, trim_to=0'0, roll_forward_to=35'12))", 2024-04-26T03:39:09.983 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:56.529067+0000", 2024-04-26T03:39:09.983 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 13.423256863000001, 2024-04-26T03:39:09.983 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0074032129999999996, 2024-04-26T03:39:09.983 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:09.983 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:09.983 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:09.983 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:09.983 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.984 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:09.984 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.529067+0000", 2024-04-26T03:39:09.984 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:09.984 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.984 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.984 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:09.984 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.529067+0000", 2024-04-26T03:39:09.984 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:09.984 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.984 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.984 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:09.984 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.529068+0000", 2024-04-26T03:39:09.984 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.4700000000000001e-07 2024-04-26T03:39:09.985 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.985 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.985 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:09.985 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.529233+0000", 2024-04-26T03:39:09.985 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000165544 2024-04-26T03:39:09.985 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.985 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.985 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:09.985 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.529245+0000", 2024-04-26T03:39:09.985 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1905999999999999e-05 2024-04-26T03:39:09.985 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.985 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.985 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:09.985 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.529251+0000", 2024-04-26T03:39:09.985 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.5330000000000002e-06 2024-04-26T03:39:09.986 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.986 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.986 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:09.986 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.529303+0000", 2024-04-26T03:39:09.986 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.2766000000000003e-05 2024-04-26T03:39:09.986 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.986 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.986 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:09.986 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.529310+0000", 2024-04-26T03:39:09.986 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.2809999999999997e-06 2024-04-26T03:39:09.986 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.986 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.986 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:09.987 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.536437+0000", 2024-04-26T03:39:09.987 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0071270090000000001 2024-04-26T03:39:09.987 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.987 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.987 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:09.987 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.536470+0000", 2024-04-26T03:39:09.987 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.3526999999999999e-05 2024-04-26T03:39:09.987 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:09.987 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:09.987 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:09.987 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.987 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.987 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.3s5 43/32 ECSubWrite(tid=29, reqid=client.4556.0:313, at_version=43'17, trim_to=0'0, roll_forward_to=35'14))", 2024-04-26T03:39:09.987 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:59.554982+0000", 2024-04-26T03:39:09.988 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 10.397341444, 2024-04-26T03:39:09.988 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0068486409999999999, 2024-04-26T03:39:09.988 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:09.988 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:09.988 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:09.988 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:09.988 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.988 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:09.988 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.554982+0000", 2024-04-26T03:39:09.988 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:09.988 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.988 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.988 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:09.988 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.554982+0000", 2024-04-26T03:39:09.989 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:09.989 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.989 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.989 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:09.989 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.554984+0000", 2024-04-26T03:39:09.989 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2419999999999999e-06 2024-04-26T03:39:09.989 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.989 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.989 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:09.989 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.555294+0000", 2024-04-26T03:39:09.989 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00031052299999999999 2024-04-26T03:39:09.989 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.989 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.989 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:09.989 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.555325+0000", 2024-04-26T03:39:09.990 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.0976000000000003e-05 2024-04-26T03:39:09.990 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.990 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.990 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:09.990 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.555332+0000", 2024-04-26T03:39:09.990 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.9410000000000001e-06 2024-04-26T03:39:09.990 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.990 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.990 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:09.990 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.555423+0000", 2024-04-26T03:39:09.990 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.0946000000000006e-05 2024-04-26T03:39:09.990 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.990 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.990 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:09.991 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.555435+0000", 2024-04-26T03:39:09.991 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1735e-05 2024-04-26T03:39:09.991 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.991 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.991 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:09.991 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.561783+0000", 2024-04-26T03:39:09.991 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0063486810000000001 2024-04-26T03:39:09.991 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.991 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.991 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:09.991 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.561831+0000", 2024-04-26T03:39:09.991 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.7596999999999998e-05 2024-04-26T03:39:09.991 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:09.991 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:09.992 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:09.992 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.992 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.992 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.3s5 43/32 ECSubWrite(tid=30, reqid=client.4556.0:314, at_version=43'18, trim_to=0'0, roll_forward_to=43'16))", 2024-04-26T03:39:09.992 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:59.562615+0000", 2024-04-26T03:39:09.992 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 10.389708616, 2024-04-26T03:39:09.992 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.020210662000000001, 2024-04-26T03:39:09.992 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:09.992 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:09.992 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:09.992 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:09.992 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.992 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:09.992 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.562615+0000", 2024-04-26T03:39:09.992 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:09.993 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.993 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.993 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:09.993 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.562615+0000", 2024-04-26T03:39:09.993 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:09.993 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.993 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.993 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:09.993 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.562616+0000", 2024-04-26T03:39:09.993 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1699999999999997e-07 2024-04-26T03:39:09.993 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.993 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.993 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:09.993 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.562662+0000", 2024-04-26T03:39:09.994 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.5809999999999997e-05 2024-04-26T03:39:09.994 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.994 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.994 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:09.994 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.562675+0000", 2024-04-26T03:39:09.994 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.332e-05 2024-04-26T03:39:09.994 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.994 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.994 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:09.994 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.562683+0000", 2024-04-26T03:39:09.994 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3889999999999999e-06 2024-04-26T03:39:09.994 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.994 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.994 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:09.995 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.562753+0000", 2024-04-26T03:39:09.995 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.0747999999999994e-05 2024-04-26T03:39:09.995 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.995 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.995 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:09.995 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.562761+0000", 2024-04-26T03:39:09.995 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.4019999999999996e-06 2024-04-26T03:39:09.995 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.995 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.995 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:09.995 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.582798+0000", 2024-04-26T03:39:09.995 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.020036838000000001 2024-04-26T03:39:09.995 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.995 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.995 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:09.996 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.582826+0000", 2024-04-26T03:39:09.996 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8238e-05 2024-04-26T03:39:09.996 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:09.996 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:09.996 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:09.996 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.996 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.996 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.3s5 43/32 ECSubWrite(tid=31, reqid=client.4556.0:315, at_version=43'19, trim_to=0'0, roll_forward_to=43'16))", 2024-04-26T03:39:09.996 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:59.564400+0000", 2024-04-26T03:39:09.996 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 10.387923658, 2024-04-26T03:39:09.996 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.018851940000000001, 2024-04-26T03:39:09.996 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:09.996 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:09.996 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:09.997 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:09.997 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.997 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:09.997 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.564400+0000", 2024-04-26T03:39:09.997 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:09.997 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.997 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.997 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:09.997 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.564400+0000", 2024-04-26T03:39:09.997 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:09.997 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.997 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.997 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:09.998 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.564401+0000", 2024-04-26T03:39:09.998 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.16e-07 2024-04-26T03:39:09.998 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.998 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.998 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:09.998 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.564436+0000", 2024-04-26T03:39:09.998 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.5463000000000002e-05 2024-04-26T03:39:09.998 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.998 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.998 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:09.998 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.582521+0000", 2024-04-26T03:39:09.998 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.018084287000000001 2024-04-26T03:39:09.998 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.998 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.999 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:09.999 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.582532+0000", 2024-04-26T03:39:09.999 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.166e-05 2024-04-26T03:39:09.999 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.999 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.999 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:09.999 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.582599+0000", 2024-04-26T03:39:09.999 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.6721e-05 2024-04-26T03:39:09.999 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:09.999 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:09.999 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:09.999 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.582609+0000", 2024-04-26T03:39:09.999 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.8959999999999998e-06 2024-04-26T03:39:09.999 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.000 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.000 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:10.000 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.583226+0000", 2024-04-26T03:39:10.000 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00061683500000000004 2024-04-26T03:39:10.000 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.000 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.000 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:10.000 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.583252+0000", 2024-04-26T03:39:10.000 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6262e-05 2024-04-26T03:39:10.000 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.000 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.000 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.000 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.000 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.001 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.fs4 45/32 ECSubWrite(tid=49, reqid=client.4556.0:325, at_version=45'17, trim_to=0'0, roll_forward_to=42'15))", 2024-04-26T03:39:10.001 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:01.554516+0000", 2024-04-26T03:39:10.001 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 8.3978082399999998, 2024-04-26T03:39:10.001 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.012414408, 2024-04-26T03:39:10.001 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:10.001 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:10.001 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:10.001 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:10.001 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.001 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:10.001 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.554516+0000", 2024-04-26T03:39:10.001 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.001 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.001 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.002 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:10.002 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.554516+0000", 2024-04-26T03:39:10.002 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.002 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.002 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.002 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:10.002 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.554516+0000", 2024-04-26T03:39:10.002 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.03e-07 2024-04-26T03:39:10.002 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.002 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.002 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:10.002 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.555044+0000", 2024-04-26T03:39:10.002 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00052827400000000002 2024-04-26T03:39:10.003 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.003 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.003 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:10.003 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.555067+0000", 2024-04-26T03:39:10.003 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2359999999999999e-05 2024-04-26T03:39:10.003 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.003 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.003 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:10.003 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.555077+0000", 2024-04-26T03:39:10.003 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0623e-05 2024-04-26T03:39:10.003 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.003 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.003 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.003 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.555156+0000", 2024-04-26T03:39:10.003 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8478000000000003e-05 2024-04-26T03:39:10.004 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.004 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.004 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:10.004 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.555167+0000", 2024-04-26T03:39:10.004 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1243000000000001e-05 2024-04-26T03:39:10.004 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.004 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.004 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:10.004 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.566750+0000", 2024-04-26T03:39:10.004 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.011583115 2024-04-26T03:39:10.004 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.004 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.004 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:10.004 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.566930+0000", 2024-04-26T03:39:10.005 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00017991200000000001 2024-04-26T03:39:10.005 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.005 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.005 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.005 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.005 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.005 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.fs4 45/32 ECSubWrite(tid=50, reqid=client.4556.0:326, at_version=45'18, trim_to=0'0, roll_forward_to=42'15))", 2024-04-26T03:39:10.005 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:01.561230+0000", 2024-04-26T03:39:10.005 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 8.3910934410000007, 2024-04-26T03:39:10.005 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0088042480000000006, 2024-04-26T03:39:10.005 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:10.005 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:10.005 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:10.005 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:10.005 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.006 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:10.006 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.561230+0000", 2024-04-26T03:39:10.006 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.006 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.006 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.006 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:10.006 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.561230+0000", 2024-04-26T03:39:10.006 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.006 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.006 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.006 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:10.006 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.561231+0000", 2024-04-26T03:39:10.006 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.3099999999999996e-07 2024-04-26T03:39:10.006 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.007 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.007 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:10.007 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.561601+0000", 2024-04-26T03:39:10.007 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000370069 2024-04-26T03:39:10.007 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.007 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.007 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:10.007 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.561616+0000", 2024-04-26T03:39:10.007 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4250000000000001e-05 2024-04-26T03:39:10.007 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.007 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.007 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:10.007 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.561625+0000", 2024-04-26T03:39:10.007 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.9679999999999995e-06 2024-04-26T03:39:10.008 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.008 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.008 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.008 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.561850+0000", 2024-04-26T03:39:10.008 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00022588600000000001 2024-04-26T03:39:10.008 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.008 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.008 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:10.008 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.561870+0000", 2024-04-26T03:39:10.008 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9887000000000001e-05 2024-04-26T03:39:10.008 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.008 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.008 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:10.008 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.569991+0000", 2024-04-26T03:39:10.009 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0081205619999999996 2024-04-26T03:39:10.009 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.009 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.009 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:10.009 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.570035+0000", 2024-04-26T03:39:10.009 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.3795000000000003e-05 2024-04-26T03:39:10.009 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.009 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.009 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.009 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.009 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.009 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.5s3 48/44 ECSubWrite(tid=45, reqid=osd.15.0:44, at_version=0'0, trim_to=0'0, roll_forward_to=48'40))", 2024-04-26T03:39:10.009 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:04.571316+0000", 2024-04-26T03:39:10.009 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 5.3810079890000004, 2024-04-26T03:39:10.010 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0084868870000000003, 2024-04-26T03:39:10.010 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:10.010 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:10.010 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:10.010 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:10.010 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.010 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:10.010 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.571316+0000", 2024-04-26T03:39:10.010 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.010 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.010 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.010 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:10.010 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.571316+0000", 2024-04-26T03:39:10.010 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.011 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.011 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.011 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:10.011 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.571317+0000", 2024-04-26T03:39:10.011 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.9299999999999996e-07 2024-04-26T03:39:10.011 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.011 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.011 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:10.011 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.571327+0000", 2024-04-26T03:39:10.011 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.9159999999999996e-06 2024-04-26T03:39:10.011 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.011 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.011 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:10.011 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.571339+0000", 2024-04-26T03:39:10.011 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2522e-05 2024-04-26T03:39:10.012 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.012 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.012 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:10.012 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.571348+0000", 2024-04-26T03:39:10.012 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.6400000000000003e-06 2024-04-26T03:39:10.012 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.012 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.012 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.012 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.571399+0000", 2024-04-26T03:39:10.012 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.1564999999999998e-05 2024-04-26T03:39:10.012 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.012 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.012 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:10.012 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.571416+0000", 2024-04-26T03:39:10.013 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6410999999999999e-05 2024-04-26T03:39:10.013 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.013 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.013 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:10.013 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.579774+0000", 2024-04-26T03:39:10.013 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0083583630000000006 2024-04-26T03:39:10.013 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.013 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.013 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:10.013 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.579803+0000", 2024-04-26T03:39:10.013 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8577000000000001e-05 2024-04-26T03:39:10.013 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.013 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.013 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.013 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.014 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.014 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.13s5 49/44 ECSubWrite(tid=67, reqid=osd.10.0:67, at_version=49'34, trim_to=0'0, roll_forward_to=48'30))", 2024-04-26T03:39:10.014 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:04.685012+0000", 2024-04-26T03:39:10.014 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 5.2673121429999998, 2024-04-26T03:39:10.014 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0092065640000000004, 2024-04-26T03:39:10.014 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:10.014 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:10.014 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:10.014 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:10.014 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.014 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:10.014 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.685012+0000", 2024-04-26T03:39:10.014 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.014 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.015 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.015 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:10.015 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.685012+0000", 2024-04-26T03:39:10.015 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.015 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.015 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.015 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:10.015 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.685012+0000", 2024-04-26T03:39:10.015 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.2500000000000005e-07 2024-04-26T03:39:10.015 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.015 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.015 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:10.015 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.685038+0000", 2024-04-26T03:39:10.015 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5989000000000001e-05 2024-04-26T03:39:10.016 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.016 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.016 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:10.016 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.685050+0000", 2024-04-26T03:39:10.016 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1702000000000001e-05 2024-04-26T03:39:10.016 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.016 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.016 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:10.016 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.685067+0000", 2024-04-26T03:39:10.016 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7147e-05 2024-04-26T03:39:10.016 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.016 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.016 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.016 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.685191+0000", 2024-04-26T03:39:10.016 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012426000000000001 2024-04-26T03:39:10.017 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.017 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.017 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:10.017 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.685202+0000", 2024-04-26T03:39:10.017 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0908e-05 2024-04-26T03:39:10.017 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.017 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.017 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:10.017 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.694180+0000", 2024-04-26T03:39:10.017 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0089774060000000003 2024-04-26T03:39:10.017 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.017 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.017 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:10.017 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.694218+0000", 2024-04-26T03:39:10.018 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8526999999999999e-05 2024-04-26T03:39:10.018 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.018 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.018 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.018 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.018 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.018 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.6s3 49/44 ECSubWrite(tid=6, reqid=client.4556.0:430, at_version=49'5, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:39:10.018 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:04.689166+0000", 2024-04-26T03:39:10.018 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 5.2631579740000003, 2024-04-26T03:39:10.018 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0091018130000000003, 2024-04-26T03:39:10.018 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:10.018 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:10.018 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:10.018 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:10.019 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.019 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:10.019 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.689166+0000", 2024-04-26T03:39:10.019 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.019 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.019 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.019 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:10.019 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.689166+0000", 2024-04-26T03:39:10.019 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.019 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.019 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.019 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:10.019 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.689166+0000", 2024-04-26T03:39:10.019 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9999999999999998e-07 2024-04-26T03:39:10.020 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.020 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.020 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:10.020 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.689185+0000", 2024-04-26T03:39:10.020 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9137e-05 2024-04-26T03:39:10.020 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.020 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.020 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:10.020 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.689196+0000", 2024-04-26T03:39:10.020 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0963e-05 2024-04-26T03:39:10.020 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.020 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.020 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:10.020 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.689205+0000", 2024-04-26T03:39:10.021 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5469999999999997e-06 2024-04-26T03:39:10.021 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.021 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.021 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.021 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.689261+0000", 2024-04-26T03:39:10.021 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.6100999999999997e-05 2024-04-26T03:39:10.021 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.021 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.021 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:10.021 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.689270+0000", 2024-04-26T03:39:10.021 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.2410000000000001e-06 2024-04-26T03:39:10.021 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.021 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.021 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:10.021 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.698221+0000", 2024-04-26T03:39:10.022 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0089505560000000001 2024-04-26T03:39:10.022 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.022 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.022 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:10.022 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.698268+0000", 2024-04-26T03:39:10.022 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6867999999999999e-05 2024-04-26T03:39:10.022 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.022 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.022 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.022 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.022 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.022 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.3s5 49/44 ECSubWrite(tid=68, reqid=osd.10.0:66, at_version=0'0, trim_to=0'0, roll_forward_to=49'30))", 2024-04-26T03:39:10.022 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:04.689742+0000", 2024-04-26T03:39:10.022 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 5.2625816250000002, 2024-04-26T03:39:10.023 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0085702499999999997, 2024-04-26T03:39:10.023 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:10.023 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:10.023 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:10.023 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:10.023 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.023 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:10.023 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.689742+0000", 2024-04-26T03:39:10.023 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.023 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.023 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.023 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:10.023 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.689742+0000", 2024-04-26T03:39:10.023 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.024 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.024 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.024 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:10.024 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.689743+0000", 2024-04-26T03:39:10.024 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7000000000000004e-07 2024-04-26T03:39:10.024 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.024 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.024 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:10.024 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.689753+0000", 2024-04-26T03:39:10.024 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0013000000000001e-05 2024-04-26T03:39:10.024 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.024 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.024 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:10.024 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.689775+0000", 2024-04-26T03:39:10.025 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1633000000000001e-05 2024-04-26T03:39:10.025 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.025 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.025 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:10.025 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.689784+0000", 2024-04-26T03:39:10.025 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.696e-06 2024-04-26T03:39:10.025 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.025 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.025 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.025 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.689837+0000", 2024-04-26T03:39:10.025 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.2617000000000001e-05 2024-04-26T03:39:10.025 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.025 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.025 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:10.026 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.689848+0000", 2024-04-26T03:39:10.026 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1003e-05 2024-04-26T03:39:10.026 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.026 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.026 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:10.026 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.698260+0000", 2024-04-26T03:39:10.026 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0084119409999999992 2024-04-26T03:39:10.026 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.026 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.026 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:10.026 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.698312+0000", 2024-04-26T03:39:10.026 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.2676999999999997e-05 2024-04-26T03:39:10.026 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.026 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.027 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.027 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.027 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.027 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.6s3 49/44 ECSubWrite(tid=7, reqid=client.4556.0:431, at_version=49'6, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:39:10.027 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:04.690515+0000", 2024-04-26T03:39:10.027 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 5.2618088209999998, 2024-04-26T03:39:10.027 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.010272303, 2024-04-26T03:39:10.027 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:10.027 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:10.027 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:10.027 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:10.027 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.027 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:10.027 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.690515+0000", 2024-04-26T03:39:10.028 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.028 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.028 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.028 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:10.028 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.690515+0000", 2024-04-26T03:39:10.028 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.028 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.028 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.028 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:10.028 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.690516+0000", 2024-04-26T03:39:10.028 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.1900000000000002e-07 2024-04-26T03:39:10.028 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.028 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.028 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:10.028 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.690533+0000", 2024-04-26T03:39:10.029 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7017e-05 2024-04-26T03:39:10.029 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.029 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.029 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:10.029 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.690546+0000", 2024-04-26T03:39:10.029 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2894999999999999e-05 2024-04-26T03:39:10.029 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.029 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.029 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:10.029 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.690555+0000", 2024-04-26T03:39:10.029 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.0650000000000005e-06 2024-04-26T03:39:10.029 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.029 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.029 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.030 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.690599+0000", 2024-04-26T03:39:10.030 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.4081e-05 2024-04-26T03:39:10.030 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.030 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.030 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:10.030 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.690605+0000", 2024-04-26T03:39:10.030 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.3820000000000001e-06 2024-04-26T03:39:10.030 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.030 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.030 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:10.030 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700747+0000", 2024-04-26T03:39:10.030 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.010142134000000001 2024-04-26T03:39:10.030 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.031 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.031 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:10.031 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700787+0000", 2024-04-26T03:39:10.031 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.0110000000000001e-05 2024-04-26T03:39:10.031 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.031 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.031 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.031 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.031 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.031 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.6s3 49/44 ECSubWrite(tid=8, reqid=client.4556.0:432, at_version=49'7, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:39:10.031 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:04.692710+0000", 2024-04-26T03:39:10.031 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 5.2596138080000001, 2024-04-26T03:39:10.031 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0081017780000000005, 2024-04-26T03:39:10.031 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:10.031 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:10.032 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:10.032 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:10.032 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.032 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:10.032 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.692710+0000", 2024-04-26T03:39:10.032 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.032 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.032 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.032 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:10.032 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.692710+0000", 2024-04-26T03:39:10.032 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.032 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.032 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.032 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:10.033 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.692710+0000", 2024-04-26T03:39:10.033 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.91e-07 2024-04-26T03:39:10.033 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.033 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.033 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:10.033 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.692729+0000", 2024-04-26T03:39:10.033 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8766e-05 2024-04-26T03:39:10.033 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.033 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.033 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:10.033 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.692747+0000", 2024-04-26T03:39:10.033 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7410000000000001e-05 2024-04-26T03:39:10.033 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.034 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.034 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:10.034 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.692759+0000", 2024-04-26T03:39:10.034 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2001000000000001e-05 2024-04-26T03:39:10.034 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.034 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.034 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.035 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.694804+0000", 2024-04-26T03:39:10.035 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0020459010000000001 2024-04-26T03:39:10.035 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.035 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.035 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:10.035 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.694814+0000", 2024-04-26T03:39:10.035 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0081999999999999e-05 2024-04-26T03:39:10.035 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.035 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.035 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:10.035 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700792+0000", 2024-04-26T03:39:10.035 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0059770680000000003 2024-04-26T03:39:10.035 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.035 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.035 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:10.036 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700812+0000", 2024-04-26T03:39:10.036 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0259e-05 2024-04-26T03:39:10.036 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.036 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.036 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.036 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.036 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.036 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.14s3 49/44 ECSubWrite(tid=36, reqid=client.4556.0:439, at_version=49'25, trim_to=0'0, roll_forward_to=46'22))", 2024-04-26T03:39:10.036 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:04.734614+0000", 2024-04-26T03:39:10.036 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 5.2177100110000003, 2024-04-26T03:39:10.036 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0076171040000000004, 2024-04-26T03:39:10.036 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:10.036 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:10.036 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:10.037 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:10.037 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.037 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:10.037 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.734614+0000", 2024-04-26T03:39:10.037 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.037 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.037 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.037 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:10.037 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.734614+0000", 2024-04-26T03:39:10.037 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.037 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.037 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.037 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:10.037 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.734615+0000", 2024-04-26T03:39:10.037 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1739999999999999e-06 2024-04-26T03:39:10.038 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.038 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.038 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:10.038 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.734797+0000", 2024-04-26T03:39:10.038 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00018156300000000001 2024-04-26T03:39:10.038 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.038 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.038 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:10.038 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.734811+0000", 2024-04-26T03:39:10.038 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4106e-05 2024-04-26T03:39:10.038 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.038 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.038 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:10.038 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.734817+0000", 2024-04-26T03:39:10.039 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.6869999999999997e-06 2024-04-26T03:39:10.039 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.039 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.039 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.039 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.734895+0000", 2024-04-26T03:39:10.039 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.7619000000000003e-05 2024-04-26T03:39:10.039 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.039 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.039 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:10.039 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.734906+0000", 2024-04-26T03:39:10.040 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1102e-05 2024-04-26T03:39:10.040 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.040 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.040 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:10.040 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.742201+0000", 2024-04-26T03:39:10.040 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0072949199999999999 2024-04-26T03:39:10.040 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.040 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.040 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:10.040 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.742231+0000", 2024-04-26T03:39:10.040 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.9933e-05 2024-04-26T03:39:10.040 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.040 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.041 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.041 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.041 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.041 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.13s5 50/44 ECSubWrite(tid=74, reqid=client.4556.0:446, at_version=50'37, trim_to=0'0, roll_forward_to=50'36))", 2024-04-26T03:39:10.041 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:05.702175+0000", 2024-04-26T03:39:10.041 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 4.2501489430000001, 2024-04-26T03:39:10.041 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.012835628, 2024-04-26T03:39:10.041 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:10.041 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:10.041 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:10.041 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:10.041 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.041 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:10.041 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.702175+0000", 2024-04-26T03:39:10.042 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.042 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.042 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.042 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:10.042 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.702175+0000", 2024-04-26T03:39:10.042 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.042 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.042 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.042 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:10.042 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.702176+0000", 2024-04-26T03:39:10.042 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1429999999999999e-06 2024-04-26T03:39:10.042 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.042 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.042 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:10.043 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.702378+0000", 2024-04-26T03:39:10.043 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020168399999999999 2024-04-26T03:39:10.043 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.043 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.043 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:10.043 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.702402+0000", 2024-04-26T03:39:10.043 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4139e-05 2024-04-26T03:39:10.043 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.043 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.043 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:10.043 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.702421+0000", 2024-04-26T03:39:10.043 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9012e-05 2024-04-26T03:39:10.043 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.043 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.043 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.044 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.702532+0000", 2024-04-26T03:39:10.044 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00011152199999999999 2024-04-26T03:39:10.044 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.044 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.044 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:10.044 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.702544+0000", 2024-04-26T03:39:10.044 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1719000000000001e-05 2024-04-26T03:39:10.044 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.044 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.044 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:10.044 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.714976+0000", 2024-04-26T03:39:10.044 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.012432241 2024-04-26T03:39:10.044 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.044 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.044 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:10.045 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.715011+0000", 2024-04-26T03:39:10.045 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.4168000000000002e-05 2024-04-26T03:39:10.045 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.045 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.045 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.045 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.045 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.045 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:39:10.047 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-26T03:39:10.047 INFO:tasks.ceph.osd.10.smithi066.stderr:2024-04-26T03:39:10.045+0000 7f130fcc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 31763) UID: 0 2024-04-26T03:39:10.147 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:39:10.146+0000 7fb78c544640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31956) UID: 0 2024-04-26T03:39:10.194 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:39:10.194 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-04-26T03:39:10.195 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 0 2024-04-26T03:39:10.195 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:39:10.203 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-26T03:39:10.248 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:39:10.247+0000 7fba0d2ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31961) UID: 0 2024-04-26T03:39:10.309 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:39:10.309 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-04-26T03:39:10.310 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-04-26T03:39:10.310 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-04-26T03:39:10.310 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:39:10.318 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-26T03:39:10.348 INFO:tasks.ceph.osd.15.smithi092.stderr:2024-04-26T03:39:10.347+0000 7f6d12dec640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 31824) UID: 0 2024-04-26T03:39:10.427 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:39:10.427 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-04-26T03:39:10.427 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-04-26T03:39:10.427 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-04-26T03:39:10.427 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.427 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4213.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-04-26T03:39:10.427 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:33.315386+0000", 2024-04-26T03:39:10.428 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 37.10923889, 2024-04-26T03:39:10.428 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.075394833, 2024-04-26T03:39:10.428 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:10.428 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:10.428 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:10.428 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:10.428 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4213", 2024-04-26T03:39:10.428 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/331201604", 2024-04-26T03:39:10.428 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 1 2024-04-26T03:39:10.428 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.428 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:10.428 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.428 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:10.428 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:33.315386+0000", 2024-04-26T03:39:10.428 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.429 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.429 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.429 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:10.429 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:33.315386+0000", 2024-04-26T03:39:10.429 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.429 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.429 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.429 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:10.429 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:33.315388+0000", 2024-04-26T03:39:10.429 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.897e-06 2024-04-26T03:39:10.429 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.429 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.429 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:10.429 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:33.315417+0000", 2024-04-26T03:39:10.429 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8983e-05 2024-04-26T03:39:10.430 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.430 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.430 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:10.430 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:33.315447+0000", 2024-04-26T03:39:10.430 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.9920000000000002e-05 2024-04-26T03:39:10.430 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.430 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.430 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:10.430 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:33.315460+0000", 2024-04-26T03:39:10.430 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3064999999999999e-05 2024-04-26T03:39:10.431 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.431 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.431 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.431 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:33.315770+0000", 2024-04-26T03:39:10.431 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000309687 2024-04-26T03:39:10.431 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.431 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.431 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for peered", 2024-04-26T03:39:10.431 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:33.315784+0000", 2024-04-26T03:39:10.431 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3767e-05 2024-04-26T03:39:10.432 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.432 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.432 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.432 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.389505+0000", 2024-04-26T03:39:10.432 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0737211259999999 2024-04-26T03:39:10.432 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.432 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.432 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:10.432 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.389667+0000", 2024-04-26T03:39:10.432 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000162435 2024-04-26T03:39:10.432 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.432 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.432 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 5", 2024-04-26T03:39:10.433 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.389837+0000", 2024-04-26T03:39:10.433 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00016988999999999999 2024-04-26T03:39:10.433 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.433 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.433 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-04-26T03:39:10.433 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.390620+0000", 2024-04-26T03:39:10.433 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000782941 2024-04-26T03:39:10.433 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.433 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.433 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-04-26T03:39:10.433 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.390712+0000", 2024-04-26T03:39:10.433 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.2119000000000004e-05 2024-04-26T03:39:10.433 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.433 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.433 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:10.434 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.390751+0000", 2024-04-26T03:39:10.434 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9168999999999997e-05 2024-04-26T03:39:10.434 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.434 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.434 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:10.434 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.390781+0000", 2024-04-26T03:39:10.434 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.9833999999999999e-05 2024-04-26T03:39:10.434 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.434 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.434 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.434 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.434 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.434 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:5 3.8s0 3:1de5c702:::smithi09235267-2:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:10.434 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.480991+0000", 2024-04-26T03:39:10.435 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 18.943633921, 2024-04-26T03:39:10.435 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.010602502999999999, 2024-04-26T03:39:10.435 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:10.435 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:10.435 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:10.435 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:10.435 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:10.435 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:10.435 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 5 2024-04-26T03:39:10.435 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.435 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:10.436 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.436 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:10.436 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.480991+0000", 2024-04-26T03:39:10.436 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.436 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.436 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.436 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:10.436 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.480991+0000", 2024-04-26T03:39:10.436 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.436 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.436 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.436 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:10.437 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.480992+0000", 2024-04-26T03:39:10.437 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.4700000000000001e-07 2024-04-26T03:39:10.437 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.437 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.437 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:10.437 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.481004+0000", 2024-04-26T03:39:10.437 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2182e-05 2024-04-26T03:39:10.437 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.437 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.437 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:10.437 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.481012+0000", 2024-04-26T03:39:10.437 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2190000000000004e-06 2024-04-26T03:39:10.437 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.437 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.438 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:10.438 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.481019+0000", 2024-04-26T03:39:10.438 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.8630000000000002e-06 2024-04-26T03:39:10.438 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.438 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.438 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.438 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.482455+0000", 2024-04-26T03:39:10.438 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0014360180000000001 2024-04-26T03:39:10.438 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.438 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.438 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:10.438 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.482538+0000", 2024-04-26T03:39:10.438 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2437999999999999e-05 2024-04-26T03:39:10.439 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.439 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.439 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:10.439 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.482991+0000", 2024-04-26T03:39:10.439 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00045318000000000002 2024-04-26T03:39:10.439 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.439 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.439 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:10.439 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.485509+0000", 2024-04-26T03:39:10.439 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0025184130000000002 2024-04-26T03:39:10.439 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.439 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.439 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:10.439 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.490366+0000", 2024-04-26T03:39:10.439 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0048569570000000003 2024-04-26T03:39:10.439 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.440 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.440 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:10.440 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.491594+0000", 2024-04-26T03:39:10.440 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0012275859999999999 2024-04-26T03:39:10.440 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.440 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.440 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.440 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.440 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.440 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:6 3.8s0 3:1de5c702:::smithi09235267-2:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:10.440 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.481043+0000", 2024-04-26T03:39:10.440 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 18.943581632000001, 2024-04-26T03:39:10.440 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.013192016000000001, 2024-04-26T03:39:10.440 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:10.441 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:10.441 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:10.441 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:10.441 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:10.441 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:10.441 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 6 2024-04-26T03:39:10.441 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.441 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:10.441 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.441 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:10.441 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.481043+0000", 2024-04-26T03:39:10.441 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.441 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.441 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.441 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:10.442 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.481043+0000", 2024-04-26T03:39:10.442 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.442 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.442 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.442 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:10.442 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.481044+0000", 2024-04-26T03:39:10.442 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.4000000000000001e-07 2024-04-26T03:39:10.442 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.442 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.442 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:10.442 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.481048+0000", 2024-04-26T03:39:10.442 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.3000000000000002e-06 2024-04-26T03:39:10.442 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.443 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.443 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:10.443 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.481059+0000", 2024-04-26T03:39:10.443 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1090999999999999e-05 2024-04-26T03:39:10.443 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.443 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.443 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:10.443 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.481065+0000", 2024-04-26T03:39:10.443 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7859999999999999e-06 2024-04-26T03:39:10.443 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.443 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.443 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.443 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.483301+0000", 2024-04-26T03:39:10.443 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0022357509999999998 2024-04-26T03:39:10.443 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.444 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.444 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:10.444 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.483380+0000", 2024-04-26T03:39:10.444 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8579999999999996e-05 2024-04-26T03:39:10.444 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.444 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.444 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.444 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.492132+0000", 2024-04-26T03:39:10.444 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0087520900000000006 2024-04-26T03:39:10.444 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.444 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.444 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:10.444 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.492213+0000", 2024-04-26T03:39:10.444 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.1335000000000006e-05 2024-04-26T03:39:10.444 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.445 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.445 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:10.445 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.494235+0000", 2024-04-26T03:39:10.445 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0020223429999999998 2024-04-26T03:39:10.445 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.445 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.445 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.445 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.445 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.445 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:57 3.8s0 3:1b3fe483:::smithi09235267-19:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:10.445 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.508374+0000", 2024-04-26T03:39:10.445 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 18.916250981000001, 2024-04-26T03:39:10.445 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.010394835, 2024-04-26T03:39:10.445 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:10.446 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:10.446 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:10.446 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:10.446 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:10.446 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:10.446 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 57 2024-04-26T03:39:10.446 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.446 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:10.446 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.446 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:10.446 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.508374+0000", 2024-04-26T03:39:10.446 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.446 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.446 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.447 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:10.447 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.508374+0000", 2024-04-26T03:39:10.447 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.447 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.447 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.447 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:10.447 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.508375+0000", 2024-04-26T03:39:10.447 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7000000000000005e-07 2024-04-26T03:39:10.447 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.447 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.447 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:10.447 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.508379+0000", 2024-04-26T03:39:10.447 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8879999999999999e-06 2024-04-26T03:39:10.447 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.447 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.448 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:10.448 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.508386+0000", 2024-04-26T03:39:10.448 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3030000000000002e-06 2024-04-26T03:39:10.448 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.448 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.448 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:10.448 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.508391+0000", 2024-04-26T03:39:10.448 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.5650000000000001e-06 2024-04-26T03:39:10.448 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.448 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.448 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.448 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.513571+0000", 2024-04-26T03:39:10.448 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0051797570000000001 2024-04-26T03:39:10.448 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.448 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.449 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:10.449 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.513659+0000", 2024-04-26T03:39:10.449 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7720999999999996e-05 2024-04-26T03:39:10.449 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.449 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.449 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.449 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.516421+0000", 2024-04-26T03:39:10.449 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0027619670000000002 2024-04-26T03:39:10.449 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.449 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.449 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:10.449 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.516512+0000", 2024-04-26T03:39:10.449 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1145999999999998e-05 2024-04-26T03:39:10.449 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.449 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.450 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:10.450 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.518769+0000", 2024-04-26T03:39:10.450 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0022569180000000001 2024-04-26T03:39:10.450 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.450 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.450 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.450 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.450 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.450 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:93 3.8s0 3:1a0b5f13:::smithi09235267-31:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:10.450 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.547482+0000", 2024-04-26T03:39:10.450 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 18.877142684999999, 2024-04-26T03:39:10.450 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.011400515999999999, 2024-04-26T03:39:10.450 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:10.450 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:10.451 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:10.451 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:10.451 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:10.451 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:10.451 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 93 2024-04-26T03:39:10.451 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.451 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:10.451 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.451 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:10.451 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.547482+0000", 2024-04-26T03:39:10.451 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.451 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.451 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.451 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:10.451 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.547482+0000", 2024-04-26T03:39:10.452 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.452 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.452 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.452 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:10.452 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.547483+0000", 2024-04-26T03:39:10.452 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.1800000000000001e-07 2024-04-26T03:39:10.452 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.452 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.452 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:10.452 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.547485+0000", 2024-04-26T03:39:10.452 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.278e-06 2024-04-26T03:39:10.452 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.452 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.452 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:10.452 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.547490+0000", 2024-04-26T03:39:10.453 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.2179999999999998e-06 2024-04-26T03:39:10.453 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.453 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.453 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:10.453 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.547494+0000", 2024-04-26T03:39:10.453 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.6820000000000001e-06 2024-04-26T03:39:10.453 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.453 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.453 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.453 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.554032+0000", 2024-04-26T03:39:10.453 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0065384329999999997 2024-04-26T03:39:10.453 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.453 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.453 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:10.453 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.554109+0000", 2024-04-26T03:39:10.454 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.6097e-05 2024-04-26T03:39:10.454 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.454 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.454 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.454 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.557384+0000", 2024-04-26T03:39:10.454 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0032751820000000002 2024-04-26T03:39:10.454 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.454 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.454 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:10.454 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.557526+0000", 2024-04-26T03:39:10.454 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014266999999999999 2024-04-26T03:39:10.454 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.454 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.454 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:10.455 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.558883+0000", 2024-04-26T03:39:10.455 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001356538 2024-04-26T03:39:10.455 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.455 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.455 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.455 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.455 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.455 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.4s4 35/32 ECSubWrite(tid=11, reqid=client.4556.0:100, at_version=35'7, trim_to=0'0, roll_forward_to=35'6))", 2024-04-26T03:39:10.455 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.560677+0000", 2024-04-26T03:39:10.455 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 18.863948351000001, 2024-04-26T03:39:10.455 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.011392509, 2024-04-26T03:39:10.455 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:10.455 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:10.455 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:10.455 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:10.456 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.456 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:10.456 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.560677+0000", 2024-04-26T03:39:10.456 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.456 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.456 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.456 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:10.456 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.560677+0000", 2024-04-26T03:39:10.456 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.456 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.456 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.456 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:10.456 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.560678+0000", 2024-04-26T03:39:10.456 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.8299999999999995e-07 2024-04-26T03:39:10.456 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.457 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.457 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:10.457 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.560997+0000", 2024-04-26T03:39:10.457 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00031924600000000003 2024-04-26T03:39:10.457 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.457 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.457 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:10.457 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.561013+0000", 2024-04-26T03:39:10.457 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6263e-05 2024-04-26T03:39:10.457 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.457 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.457 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:10.457 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.561022+0000", 2024-04-26T03:39:10.457 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.4679999999999992e-06 2024-04-26T03:39:10.457 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.458 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.458 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.458 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.561119+0000", 2024-04-26T03:39:10.458 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.7088000000000003e-05 2024-04-26T03:39:10.458 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.458 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.458 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:10.458 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.561127+0000", 2024-04-26T03:39:10.458 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7579999999999998e-06 2024-04-26T03:39:10.458 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.458 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.458 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:10.458 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.572031+0000", 2024-04-26T03:39:10.458 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.010903691 2024-04-26T03:39:10.459 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.459 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.459 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:10.459 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.572069+0000", 2024-04-26T03:39:10.459 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8111999999999997e-05 2024-04-26T03:39:10.459 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.459 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.459 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.459 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.459 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.459 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.4s4 40/32 ECSubWrite(tid=22, reqid=client.4556.0:262, at_version=40'14, trim_to=0'0, roll_forward_to=35'12))", 2024-04-26T03:39:10.459 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:56.521692+0000", 2024-04-26T03:39:10.459 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 13.902933318000001, 2024-04-26T03:39:10.459 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.014776724, 2024-04-26T03:39:10.460 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:10.460 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:10.460 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:10.460 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:10.460 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.460 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:10.460 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.521692+0000", 2024-04-26T03:39:10.460 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.460 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.460 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.460 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:10.460 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.521692+0000", 2024-04-26T03:39:10.460 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.460 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.461 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.461 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:10.461 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.521692+0000", 2024-04-26T03:39:10.461 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.4999999999999998e-07 2024-04-26T03:39:10.461 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.461 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.461 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:10.461 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.522058+0000", 2024-04-26T03:39:10.461 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00036550299999999999 2024-04-26T03:39:10.461 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.461 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.461 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:10.461 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.522068+0000", 2024-04-26T03:39:10.461 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0263e-05 2024-04-26T03:39:10.461 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.462 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.462 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:10.462 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.522074+0000", 2024-04-26T03:39:10.462 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7209999999999999e-06 2024-04-26T03:39:10.462 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.462 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.462 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.462 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.522144+0000", 2024-04-26T03:39:10.462 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.0240999999999994e-05 2024-04-26T03:39:10.462 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.462 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.462 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:10.462 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.522154+0000", 2024-04-26T03:39:10.462 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0457e-05 2024-04-26T03:39:10.463 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.463 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.463 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:10.463 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.536432+0000", 2024-04-26T03:39:10.463 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.014278107999999999 2024-04-26T03:39:10.463 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.463 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.463 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:10.463 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.536469+0000", 2024-04-26T03:39:10.463 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.6081000000000002e-05 2024-04-26T03:39:10.463 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.463 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.463 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.463 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.464 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.464 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:266 3.8s0 3:1b3fe483:::smithi09235267-19:head [append 0~737280] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T03:39:10.464 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:56.527487+0000", 2024-04-26T03:39:10.464 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 13.897137679, 2024-04-26T03:39:10.464 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.015863135, 2024-04-26T03:39:10.464 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:10.464 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:10.464 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:10.464 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:10.464 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:10.464 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:10.464 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 266 2024-04-26T03:39:10.464 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.464 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:10.464 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.465 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:10.465 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.527487+0000", 2024-04-26T03:39:10.465 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.465 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.465 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.465 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:10.465 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.527487+0000", 2024-04-26T03:39:10.465 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.465 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.465 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.465 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:10.465 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.527489+0000", 2024-04-26T03:39:10.465 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5239999999999999e-06 2024-04-26T03:39:10.465 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.465 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.465 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:10.466 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.528331+0000", 2024-04-26T03:39:10.466 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00084205300000000005 2024-04-26T03:39:10.466 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.466 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.466 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:10.466 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.528347+0000", 2024-04-26T03:39:10.466 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5882000000000001e-05 2024-04-26T03:39:10.466 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.466 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.466 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:10.466 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.528353+0000", 2024-04-26T03:39:10.466 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.9800000000000003e-06 2024-04-26T03:39:10.466 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.466 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.467 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.467 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.528421+0000", 2024-04-26T03:39:10.467 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.8608000000000004e-05 2024-04-26T03:39:10.467 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.467 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.467 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:10.467 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.528526+0000", 2024-04-26T03:39:10.467 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00010430200000000001 2024-04-26T03:39:10.467 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.467 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.467 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:10.467 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.533515+0000", 2024-04-26T03:39:10.467 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0049888149999999997 2024-04-26T03:39:10.467 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.467 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.468 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:10.468 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.542563+0000", 2024-04-26T03:39:10.468 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0090485449999999999 2024-04-26T03:39:10.468 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.468 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.468 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:10.468 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.543246+0000", 2024-04-26T03:39:10.468 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00068309800000000004 2024-04-26T03:39:10.468 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.468 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.468 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:10.468 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.543351+0000", 2024-04-26T03:39:10.468 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00010432799999999999 2024-04-26T03:39:10.468 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.469 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.469 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.469 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.469 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.469 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:266 3.8s0 3:1b3fe483:::smithi09235267-19:head [append 0~737280] snapc 4=[2] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T03:39:10.469 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:56.530709+0000", 2024-04-26T03:39:10.469 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 13.893916236000001, 2024-04-26T03:39:10.469 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.012550212999999999, 2024-04-26T03:39:10.469 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:10.469 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:10.469 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "waiting for ondisk", 2024-04-26T03:39:10.469 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:10.469 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:10.469 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:10.469 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 266 2024-04-26T03:39:10.470 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.470 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:10.470 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.470 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:10.470 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.530709+0000", 2024-04-26T03:39:10.470 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.470 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.470 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.470 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:10.470 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.530709+0000", 2024-04-26T03:39:10.470 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.470 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.470 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.470 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:10.471 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.530710+0000", 2024-04-26T03:39:10.471 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5000000000000001e-07 2024-04-26T03:39:10.471 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.471 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.471 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:10.471 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.531500+0000", 2024-04-26T03:39:10.471 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00079064200000000004 2024-04-26T03:39:10.471 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.471 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.471 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:10.471 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.531511+0000", 2024-04-26T03:39:10.471 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0934000000000001e-05 2024-04-26T03:39:10.471 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.471 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.471 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:10.472 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.531519+0000", 2024-04-26T03:39:10.472 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9470000000000003e-06 2024-04-26T03:39:10.472 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.472 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.472 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.472 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.538902+0000", 2024-04-26T03:39:10.472 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0073822979999999998 2024-04-26T03:39:10.472 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.472 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.472 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for ondisk", 2024-04-26T03:39:10.472 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.538974+0000", 2024-04-26T03:39:10.472 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.2317000000000006e-05 2024-04-26T03:39:10.472 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.472 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.472 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:10.473 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.543259+0000", 2024-04-26T03:39:10.473 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0042852250000000001 2024-04-26T03:39:10.473 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.473 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.473 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.473 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.473 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.473 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:267 3.8s0 3:1b3fe483:::smithi09235267-19:head [append 0~376832] snapc 4=[2] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T03:39:10.473 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:56.531532+0000", 2024-04-26T03:39:10.473 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 13.893092719, 2024-04-26T03:39:10.473 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.012723017999999999, 2024-04-26T03:39:10.473 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:10.473 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:10.473 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:10.474 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:10.474 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:10.474 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:10.474 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 267 2024-04-26T03:39:10.474 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.474 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:10.474 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.474 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:10.474 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.531532+0000", 2024-04-26T03:39:10.474 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.474 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.474 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.474 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:10.474 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.531532+0000", 2024-04-26T03:39:10.475 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.475 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.475 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.475 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:10.475 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.531533+0000", 2024-04-26T03:39:10.475 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.8400000000000004e-07 2024-04-26T03:39:10.475 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.475 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.475 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:10.475 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.531836+0000", 2024-04-26T03:39:10.475 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00030317300000000001 2024-04-26T03:39:10.475 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.475 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.475 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:10.475 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.531848+0000", 2024-04-26T03:39:10.476 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2221e-05 2024-04-26T03:39:10.476 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.476 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.476 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:10.476 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.531855+0000", 2024-04-26T03:39:10.476 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.8469999999999998e-06 2024-04-26T03:39:10.476 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.476 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.476 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.476 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.539012+0000", 2024-04-26T03:39:10.476 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0071566980000000004 2024-04-26T03:39:10.476 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.476 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.476 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:10.476 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.539109+0000", 2024-04-26T03:39:10.476 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.7313999999999997e-05 2024-04-26T03:39:10.477 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.477 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.477 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:10.477 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.539555+0000", 2024-04-26T03:39:10.477 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00044609199999999998 2024-04-26T03:39:10.477 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.477 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.477 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:10.477 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.542683+0000", 2024-04-26T03:39:10.477 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0031274670000000001 2024-04-26T03:39:10.477 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.477 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.477 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:10.477 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.544219+0000", 2024-04-26T03:39:10.477 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001535998 2024-04-26T03:39:10.478 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.478 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.478 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:10.478 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.544255+0000", 2024-04-26T03:39:10.478 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.6523999999999998e-05 2024-04-26T03:39:10.478 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.478 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.478 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.478 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.478 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.478 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:268 3.8s0 3:1b3fe483:::smithi09235267-19:head [setxattr _header (56) in=7b] snapc 4=[2] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T03:39:10.478 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:56.531867+0000", 2024-04-26T03:39:10.478 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 13.892758253, 2024-04-26T03:39:10.478 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.014163701000000001, 2024-04-26T03:39:10.479 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:10.479 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:10.479 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:10.479 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:10.479 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:10.479 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:10.479 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 268 2024-04-26T03:39:10.479 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.479 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:10.479 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.479 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:10.479 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.531867+0000", 2024-04-26T03:39:10.479 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.479 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.479 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.480 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:10.480 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.531867+0000", 2024-04-26T03:39:10.480 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.480 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.480 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.480 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:10.480 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.531867+0000", 2024-04-26T03:39:10.480 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.2099999999999996e-07 2024-04-26T03:39:10.480 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.480 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.480 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:10.480 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.531872+0000", 2024-04-26T03:39:10.480 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.578e-06 2024-04-26T03:39:10.480 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.480 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.481 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:10.481 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.531881+0000", 2024-04-26T03:39:10.481 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.9369999999999993e-06 2024-04-26T03:39:10.481 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.481 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.481 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:10.481 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.531887+0000", 2024-04-26T03:39:10.481 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.9150000000000001e-06 2024-04-26T03:39:10.481 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.481 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.481 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.481 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.541782+0000", 2024-04-26T03:39:10.481 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0098949529999999997 2024-04-26T03:39:10.481 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.481 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.482 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:10.482 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.541877+0000", 2024-04-26T03:39:10.482 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.5051999999999997e-05 2024-04-26T03:39:10.482 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.482 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.482 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:10.482 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.542321+0000", 2024-04-26T03:39:10.482 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000443986 2024-04-26T03:39:10.482 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.482 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.482 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:10.482 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.543061+0000", 2024-04-26T03:39:10.482 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00074033599999999997 2024-04-26T03:39:10.482 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.483 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.483 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:10.483 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.545981+0000", 2024-04-26T03:39:10.483 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.002919919 2024-04-26T03:39:10.483 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.483 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.483 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:10.483 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.546031+0000", 2024-04-26T03:39:10.483 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.9404000000000003e-05 2024-04-26T03:39:10.483 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.483 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.483 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.483 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.483 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.483 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:269 3.8s0 3:1b3fe483:::smithi09235267-19:head [read 0~1] snapc 0=[] RETRY=1 ondisk+retry+read+rwordered+known_if_redirected+supports_pool_eio e40)", 2024-04-26T03:39:10.484 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:56.531896+0000", 2024-04-26T03:39:10.484 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 13.892729254000001, 2024-04-26T03:39:10.484 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.017279658999999999, 2024-04-26T03:39:10.484 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:10.484 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:10.484 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:10.484 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:10.484 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:10.484 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:10.484 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 269 2024-04-26T03:39:10.484 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.484 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:10.484 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.484 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:10.485 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.531896+0000", 2024-04-26T03:39:10.485 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.485 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.485 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.485 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:10.485 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.531896+0000", 2024-04-26T03:39:10.485 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.485 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.485 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.485 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:10.485 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.531896+0000", 2024-04-26T03:39:10.485 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7599999999999997e-07 2024-04-26T03:39:10.485 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.485 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.485 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:10.486 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.531899+0000", 2024-04-26T03:39:10.486 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8550000000000002e-06 2024-04-26T03:39:10.486 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.486 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.486 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:10.486 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.531907+0000", 2024-04-26T03:39:10.486 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9349999999999994e-06 2024-04-26T03:39:10.486 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.486 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.486 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:10.486 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.531913+0000", 2024-04-26T03:39:10.486 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7540000000000003e-06 2024-04-26T03:39:10.486 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.486 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.487 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.487 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.542916+0000", 2024-04-26T03:39:10.487 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.01100284 2024-04-26T03:39:10.487 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.487 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.487 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:10.487 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.543008+0000", 2024-04-26T03:39:10.487 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1822000000000003e-05 2024-04-26T03:39:10.487 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.487 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.487 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.487 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.546345+0000", 2024-04-26T03:39:10.487 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0033376539999999998 2024-04-26T03:39:10.487 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.487 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.488 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:10.488 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.546438+0000", 2024-04-26T03:39:10.488 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.2394999999999999e-05 2024-04-26T03:39:10.488 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.488 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.488 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:10.488 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.549176+0000", 2024-04-26T03:39:10.488 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0027378279999999999 2024-04-26T03:39:10.488 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.488 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.488 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.488 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.488 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.488 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:332 3.18s0 3:1b3fe483:::smithi09235267-19:head [append 0~0] snapc 7=[7,6,5] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-26T03:39:10.488 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:01.558550+0000", 2024-04-26T03:39:10.489 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 8.8660746699999997, 2024-04-26T03:39:10.489 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0097538829999999997, 2024-04-26T03:39:10.489 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:10.489 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:10.489 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:10.489 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:10.489 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:10.489 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:10.489 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 332 2024-04-26T03:39:10.489 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.489 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:10.489 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.489 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:10.489 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558550+0000", 2024-04-26T03:39:10.489 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.490 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.490 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.490 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:10.490 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558550+0000", 2024-04-26T03:39:10.490 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.490 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.490 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.490 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:10.490 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558551+0000", 2024-04-26T03:39:10.490 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.1099999999999995e-07 2024-04-26T03:39:10.490 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.490 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.490 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:10.490 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558555+0000", 2024-04-26T03:39:10.490 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8130000000000002e-06 2024-04-26T03:39:10.491 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.491 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.491 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:10.491 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558564+0000", 2024-04-26T03:39:10.491 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7660000000000004e-06 2024-04-26T03:39:10.491 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.491 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.491 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:10.491 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558570+0000", 2024-04-26T03:39:10.491 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7399999999999998e-06 2024-04-26T03:39:10.491 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.491 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.491 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.491 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.561389+0000", 2024-04-26T03:39:10.491 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0028189740000000001 2024-04-26T03:39:10.492 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.492 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.492 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:10.492 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.561491+0000", 2024-04-26T03:39:10.492 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00010136099999999999 2024-04-26T03:39:10.492 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.492 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.492 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:10.492 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.561980+0000", 2024-04-26T03:39:10.492 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00048898699999999997 2024-04-26T03:39:10.492 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.492 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.492 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:10.492 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.564997+0000", 2024-04-26T03:39:10.492 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.003016782 2024-04-26T03:39:10.493 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.493 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.493 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:10.493 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.568223+0000", 2024-04-26T03:39:10.493 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.003226876 2024-04-26T03:39:10.493 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.493 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.493 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:10.493 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.568304+0000", 2024-04-26T03:39:10.493 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.0872999999999996e-05 2024-04-26T03:39:10.493 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.493 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.493 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.493 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.493 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.494 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:443 3.18s0 3:1de5c702:::smithi09235267-2:head [copy-from ver 3] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-04-26T03:39:10.494 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:04.728363+0000", 2024-04-26T03:39:10.494 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 5.6962624750000002, 2024-04-26T03:39:10.494 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.016662854000000001, 2024-04-26T03:39:10.494 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:10.494 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:10.494 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:10.494 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:10.494 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:10.494 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:10.494 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 443 2024-04-26T03:39:10.494 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.494 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:10.494 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.495 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:10.495 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728363+0000", 2024-04-26T03:39:10.495 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.495 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.495 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.495 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:10.495 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728363+0000", 2024-04-26T03:39:10.495 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.495 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.495 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.495 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:10.495 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728364+0000", 2024-04-26T03:39:10.496 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4190000000000001e-06 2024-04-26T03:39:10.496 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.496 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.496 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:10.496 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728370+0000", 2024-04-26T03:39:10.496 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.5400000000000003e-06 2024-04-26T03:39:10.496 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.496 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.496 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:10.496 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728381+0000", 2024-04-26T03:39:10.496 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0998e-05 2024-04-26T03:39:10.496 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.496 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.496 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:10.497 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728387+0000", 2024-04-26T03:39:10.497 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.2430000000000002e-06 2024-04-26T03:39:10.497 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.497 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.497 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.497 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728450+0000", 2024-04-26T03:39:10.497 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.3639000000000005e-05 2024-04-26T03:39:10.497 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.497 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.497 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:10.497 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728581+0000", 2024-04-26T03:39:10.497 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013051100000000001 2024-04-26T03:39:10.497 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.497 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.498 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:10.498 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.734449+0000", 2024-04-26T03:39:10.498 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0058678250000000001 2024-04-26T03:39:10.498 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.498 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.498 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:10.498 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.744550+0000", 2024-04-26T03:39:10.498 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.010101044999999999 2024-04-26T03:39:10.498 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.498 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.498 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:10.498 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.744977+0000", 2024-04-26T03:39:10.498 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00042698699999999998 2024-04-26T03:39:10.498 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.498 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.499 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:10.499 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.745025+0000", 2024-04-26T03:39:10.499 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.8646999999999997e-05 2024-04-26T03:39:10.499 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.499 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.499 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.499 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.499 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.499 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:444 3.18s0 3:1de5c702:::smithi09235267-2:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e49)", 2024-04-26T03:39:10.499 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:04.728401+0000", 2024-04-26T03:39:10.499 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 5.6962243929999996, 2024-04-26T03:39:10.499 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.01449813, 2024-04-26T03:39:10.499 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:10.499 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:10.499 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:10.500 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:10.500 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:10.500 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:10.500 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 444 2024-04-26T03:39:10.500 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.500 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:10.500 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.500 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:10.500 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728401+0000", 2024-04-26T03:39:10.500 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.500 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.500 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.500 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:10.500 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728401+0000", 2024-04-26T03:39:10.501 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.501 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.501 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.501 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:10.501 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728401+0000", 2024-04-26T03:39:10.501 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.5499999999999998e-07 2024-04-26T03:39:10.501 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.501 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.501 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:10.501 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728404+0000", 2024-04-26T03:39:10.501 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4959999999999999e-06 2024-04-26T03:39:10.501 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.501 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.501 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:10.501 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728424+0000", 2024-04-26T03:39:10.502 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0307999999999999e-05 2024-04-26T03:39:10.502 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.502 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.502 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:10.502 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728430+0000", 2024-04-26T03:39:10.502 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.5459999999999999e-06 2024-04-26T03:39:10.502 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.502 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.502 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.502 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728698+0000", 2024-04-26T03:39:10.502 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00026843 2024-04-26T03:39:10.502 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.502 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.502 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for blocked object", 2024-04-26T03:39:10.503 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728723+0000", 2024-04-26T03:39:10.503 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5258000000000001e-05 2024-04-26T03:39:10.503 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.503 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.503 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.503 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.742719+0000", 2024-04-26T03:39:10.503 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.013995254 2024-04-26T03:39:10.503 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.503 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.503 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:10.503 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.742810+0000", 2024-04-26T03:39:10.503 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1774999999999995e-05 2024-04-26T03:39:10.503 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.503 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.503 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:10.503 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.742899+0000", 2024-04-26T03:39:10.504 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.8508e-05 2024-04-26T03:39:10.504 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.504 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.504 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.504 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.504 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.504 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:484 3.18s0 3:1b3fe483:::smithi09235267-19:head [append 0~425984] snapc 9=[9] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e51)", 2024-04-26T03:39:10.504 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:06.700875+0000", 2024-04-26T03:39:10.504 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 3.7237499729999999, 2024-04-26T03:39:10.504 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.016679452000000001, 2024-04-26T03:39:10.504 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:10.504 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:10.504 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:10.504 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:10.504 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:10.505 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:10.505 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 484 2024-04-26T03:39:10.505 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.505 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:10.505 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.505 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:10.505 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.700875+0000", 2024-04-26T03:39:10.505 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.505 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.505 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.505 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:10.505 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.700875+0000", 2024-04-26T03:39:10.505 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.505 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.505 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.506 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:10.506 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.700876+0000", 2024-04-26T03:39:10.506 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.2600000000000001e-07 2024-04-26T03:39:10.506 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.506 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.506 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:10.506 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.701387+0000", 2024-04-26T03:39:10.506 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00051046299999999995 2024-04-26T03:39:10.506 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.506 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.506 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:10.506 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.701400+0000", 2024-04-26T03:39:10.506 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3249e-05 2024-04-26T03:39:10.506 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.507 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.507 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:10.507 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.701419+0000", 2024-04-26T03:39:10.507 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.969e-05 2024-04-26T03:39:10.507 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.507 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.507 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.507 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.701489+0000", 2024-04-26T03:39:10.507 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.9465e-05 2024-04-26T03:39:10.507 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.507 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.507 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op must wait for map", 2024-04-26T03:39:10.507 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.701494+0000", 2024-04-26T03:39:10.507 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.215e-06 2024-04-26T03:39:10.507 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.508 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.508 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.508 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.703613+0000", 2024-04-26T03:39:10.508 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0021187630000000001 2024-04-26T03:39:10.508 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.508 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.508 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:10.508 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.703686+0000", 2024-04-26T03:39:10.508 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.2741e-05 2024-04-26T03:39:10.508 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.508 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.508 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:10.508 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.704604+0000", 2024-04-26T03:39:10.508 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00091841999999999998 2024-04-26T03:39:10.509 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.509 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.509 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:10.509 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.717233+0000", 2024-04-26T03:39:10.509 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.012629067000000001 2024-04-26T03:39:10.509 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.509 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.509 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:10.509 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.717494+0000", 2024-04-26T03:39:10.509 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00026084900000000001 2024-04-26T03:39:10.509 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.509 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.509 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:10.509 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.717555+0000", 2024-04-26T03:39:10.509 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.0603999999999998e-05 2024-04-26T03:39:10.510 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.510 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.510 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.510 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.510 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.510 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:485 3.18s0 3:1b3fe483:::smithi09235267-19:head [append 0~786432] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-26T03:39:10.510 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:06.704369+0000", 2024-04-26T03:39:10.510 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 3.7202557990000003, 2024-04-26T03:39:10.510 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.013472728999999999, 2024-04-26T03:39:10.510 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:10.510 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:10.510 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:10.510 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:10.510 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:10.511 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:10.511 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 485 2024-04-26T03:39:10.511 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.511 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:10.511 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.511 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:10.511 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.704369+0000", 2024-04-26T03:39:10.511 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.511 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.511 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.511 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:10.511 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.704369+0000", 2024-04-26T03:39:10.511 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.511 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.512 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.512 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:10.512 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.704371+0000", 2024-04-26T03:39:10.512 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3e-06 2024-04-26T03:39:10.512 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.512 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.512 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:10.512 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.705189+0000", 2024-04-26T03:39:10.512 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00081839399999999996 2024-04-26T03:39:10.512 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.512 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.512 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:10.512 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.705209+0000", 2024-04-26T03:39:10.512 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0224000000000002e-05 2024-04-26T03:39:10.512 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.513 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.513 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:10.513 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.705221+0000", 2024-04-26T03:39:10.513 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1766000000000001e-05 2024-04-26T03:39:10.513 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.513 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.513 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.513 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.708313+0000", 2024-04-26T03:39:10.513 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0030923360000000002 2024-04-26T03:39:10.513 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.513 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.513 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:10.513 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.708466+0000", 2024-04-26T03:39:10.513 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00015266699999999999 2024-04-26T03:39:10.513 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.514 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.514 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:10.514 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.709499+0000", 2024-04-26T03:39:10.514 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001032977 2024-04-26T03:39:10.514 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.514 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.514 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:10.514 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.717345+0000", 2024-04-26T03:39:10.514 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0078458989999999999 2024-04-26T03:39:10.514 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.514 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.514 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:10.514 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.717805+0000", 2024-04-26T03:39:10.514 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00045974200000000002 2024-04-26T03:39:10.515 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.515 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.515 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:10.515 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.717842+0000", 2024-04-26T03:39:10.515 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7423999999999999e-05 2024-04-26T03:39:10.515 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.515 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.515 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.515 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.515 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.515 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:486 3.18s0 3:1b3fe483:::smithi09235267-19:head [append 0~376832] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-26T03:39:10.515 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:06.706864+0000", 2024-04-26T03:39:10.515 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 3.7177614650000002, 2024-04-26T03:39:10.515 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.013205438999999999, 2024-04-26T03:39:10.515 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:10.516 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:10.516 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:10.516 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:10.516 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:10.516 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:10.516 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 486 2024-04-26T03:39:10.516 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.516 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:10.516 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.516 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:10.516 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.706864+0000", 2024-04-26T03:39:10.516 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.516 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.516 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.516 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:10.517 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.706864+0000", 2024-04-26T03:39:10.517 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.517 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.517 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.517 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:10.517 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.706865+0000", 2024-04-26T03:39:10.517 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3999999999999999e-06 2024-04-26T03:39:10.517 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.517 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.517 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:10.517 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.707260+0000", 2024-04-26T03:39:10.517 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00039510700000000001 2024-04-26T03:39:10.517 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.517 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.517 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:10.518 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.707274+0000", 2024-04-26T03:39:10.518 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4173e-05 2024-04-26T03:39:10.518 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.518 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.518 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:10.518 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.707284+0000", 2024-04-26T03:39:10.518 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.2900000000000008e-06 2024-04-26T03:39:10.518 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.518 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.518 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.518 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.714104+0000", 2024-04-26T03:39:10.518 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0068203270000000002 2024-04-26T03:39:10.518 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.518 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.518 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:10.519 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.714210+0000", 2024-04-26T03:39:10.519 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00010627 2024-04-26T03:39:10.519 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.519 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.519 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:10.519 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.714867+0000", 2024-04-26T03:39:10.519 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00065648900000000001 2024-04-26T03:39:10.519 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.519 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.519 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:10.519 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.719569+0000", 2024-04-26T03:39:10.519 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0047021399999999996 2024-04-26T03:39:10.519 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.519 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.519 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:10.520 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.720007+0000", 2024-04-26T03:39:10.520 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00043843499999999999 2024-04-26T03:39:10.520 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.520 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.520 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:10.520 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.720069+0000", 2024-04-26T03:39:10.520 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.1808000000000002e-05 2024-04-26T03:39:10.520 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.520 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.520 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.520 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.520 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.520 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:487 3.18s0 3:1b3fe483:::smithi09235267-19:head [setxattr _header (56) in=7b] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-26T03:39:10.520 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:06.707300+0000", 2024-04-26T03:39:10.520 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 3.7173248619999999, 2024-04-26T03:39:10.521 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.013837118000000001, 2024-04-26T03:39:10.521 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:10.521 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:10.521 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:10.521 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:10.521 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:10.521 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:10.521 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 487 2024-04-26T03:39:10.521 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.522 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:10.522 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.522 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:10.522 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.707300+0000", 2024-04-26T03:39:10.522 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.522 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.522 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.522 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:10.522 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.707300+0000", 2024-04-26T03:39:10.522 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.522 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.522 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.522 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:10.522 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.707301+0000", 2024-04-26T03:39:10.522 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.0800000000000003e-07 2024-04-26T03:39:10.523 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.523 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.523 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:10.523 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.707306+0000", 2024-04-26T03:39:10.523 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.2610000000000001e-06 2024-04-26T03:39:10.523 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.523 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.523 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:10.523 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.707317+0000", 2024-04-26T03:39:10.523 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0825999999999999e-05 2024-04-26T03:39:10.523 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.523 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.523 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:10.523 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.707325+0000", 2024-04-26T03:39:10.523 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.2830000000000004e-06 2024-04-26T03:39:10.524 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.524 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.524 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.524 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.717877+0000", 2024-04-26T03:39:10.524 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.010552404 2024-04-26T03:39:10.524 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.524 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.524 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:10.524 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.717938+0000", 2024-04-26T03:39:10.524 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.0908e-05 2024-04-26T03:39:10.524 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.524 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.524 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:10.524 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.718210+0000", 2024-04-26T03:39:10.524 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00027242199999999997 2024-04-26T03:39:10.525 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.525 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.525 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:10.525 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.719865+0000", 2024-04-26T03:39:10.525 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001654555 2024-04-26T03:39:10.525 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.525 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.525 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:10.525 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.721044+0000", 2024-04-26T03:39:10.525 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0011796039999999999 2024-04-26T03:39:10.525 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.525 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.525 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:10.525 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.721137+0000", 2024-04-26T03:39:10.526 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.2947000000000003e-05 2024-04-26T03:39:10.526 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.526 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.526 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.526 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.526 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.526 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:488 3.18s0 3:1b3fe483:::smithi09235267-19:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e51)", 2024-04-26T03:39:10.526 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:06.707338+0000", 2024-04-26T03:39:10.526 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 3.7172872950000002, 2024-04-26T03:39:10.526 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.015400540000000001, 2024-04-26T03:39:10.526 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:10.526 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:10.526 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:10.526 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:10.526 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:10.527 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:10.527 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 488 2024-04-26T03:39:10.527 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.527 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:10.527 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.527 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:10.527 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.707338+0000", 2024-04-26T03:39:10.527 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.527 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.527 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.527 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:10.527 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.707338+0000", 2024-04-26T03:39:10.527 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.527 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.527 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.528 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:10.528 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.707338+0000", 2024-04-26T03:39:10.528 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5099999999999999e-07 2024-04-26T03:39:10.528 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.528 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.528 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:10.528 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.707342+0000", 2024-04-26T03:39:10.528 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8879999999999999e-06 2024-04-26T03:39:10.528 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.528 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.528 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:10.528 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.707352+0000", 2024-04-26T03:39:10.528 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.4490000000000005e-06 2024-04-26T03:39:10.528 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.528 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.529 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:10.529 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.707360+0000", 2024-04-26T03:39:10.529 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.3229999999999998e-06 2024-04-26T03:39:10.529 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.529 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.529 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.529 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.718657+0000", 2024-04-26T03:39:10.529 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.011297019 2024-04-26T03:39:10.529 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.529 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.529 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:10.529 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.718712+0000", 2024-04-26T03:39:10.529 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.5241000000000002e-05 2024-04-26T03:39:10.529 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.529 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.530 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.530 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.721424+0000", 2024-04-26T03:39:10.530 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0027115529999999998 2024-04-26T03:39:10.530 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.530 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.530 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:10.530 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.721472+0000", 2024-04-26T03:39:10.530 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.8174999999999999e-05 2024-04-26T03:39:10.530 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.530 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.530 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:10.530 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.722738+0000", 2024-04-26T03:39:10.530 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001266241 2024-04-26T03:39:10.530 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.531 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.531 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.531 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.531 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.531 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:39:10.532 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-04-26T03:39:10.533 INFO:tasks.ceph.osd.10.smithi066.stderr:2024-04-26T03:39:10.531+0000 7f130fcc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 31763) UID: 0 2024-04-26T03:39:10.633 INFO:tasks.ceph.osd.15.smithi092.stderr:2024-04-26T03:39:10.631+0000 7f6d12dec640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 31824) UID: 0 2024-04-26T03:39:10.682 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:39:10.682 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-04-26T03:39:10.683 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 0 2024-04-26T03:39:10.683 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:39:10.690 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-04-26T03:39:10.705 INFO:teuthology.orchestra.run.smithi028.stderr:set pool 3 pg_num to 16 2024-04-26T03:39:10.709 INFO:tasks.rados.rados.0.smithi092.stdout:230: done (0 left) 2024-04-26T03:39:10.709 INFO:tasks.rados.rados.0.smithi092.stdout:231: read oid 2 snap -1 2024-04-26T03:39:10.709 INFO:tasks.rados.rados.0.smithi092.stdout:231: expect (ObjNum 110 snap 8 seq_num 110) 2024-04-26T03:39:10.709 INFO:tasks.rados.rados.0.smithi092.stdout:232: read oid 34 snap -1 2024-04-26T03:39:10.709 INFO:tasks.rados.rados.0.smithi092.stdout:232: expect deleted 2024-04-26T03:39:10.710 INFO:tasks.rados.rados.0.smithi092.stdout:233: snap_create 2024-04-26T03:39:10.712 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-26T03:39:10.724 INFO:tasks.rados.rados.0.smithi092.stdout:231: expect (ObjNum 110 snap 8 seq_num 110) 2024-04-26T03:39:10.766 INFO:tasks.ceph.osd.4.smithi028.stderr:2024-04-26T03:39:10.765+0000 7fd9511b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31963) UID: 0 2024-04-26T03:39:10.800 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:39:10.801 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-04-26T03:39:10.801 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-04-26T03:39:10.801 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-04-26T03:39:10.801 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:39:10.809 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-04-26T03:39:10.853 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-04-26T03:39:10.852+0000 7fee562fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31759) UID: 0 2024-04-26T03:39:10.918 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:39:10.918 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-04-26T03:39:10.918 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-04-26T03:39:10.918 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-04-26T03:39:10.918 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.918 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:17 3.3s0 3:c944ba3d:::smithi09235267-6:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:10.918 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.485815+0000", 2024-04-26T03:39:10.918 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 19.429376607999998, 2024-04-26T03:39:10.918 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.013104035999999999, 2024-04-26T03:39:10.918 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:10.918 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:10.918 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:10.919 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:10.919 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:10.919 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:10.919 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 17 2024-04-26T03:39:10.919 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.919 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:10.919 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.919 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:10.919 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.485815+0000", 2024-04-26T03:39:10.919 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.919 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.919 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.919 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:10.919 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.485815+0000", 2024-04-26T03:39:10.920 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.920 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.920 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.920 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:10.920 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.485816+0000", 2024-04-26T03:39:10.920 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.0599999999999999e-07 2024-04-26T03:39:10.920 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.920 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.920 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:10.920 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.485821+0000", 2024-04-26T03:39:10.920 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.2700000000000004e-06 2024-04-26T03:39:10.920 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.920 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.921 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:10.921 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.486544+0000", 2024-04-26T03:39:10.921 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00072360100000000004 2024-04-26T03:39:10.921 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.921 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.921 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:10.921 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.486558+0000", 2024-04-26T03:39:10.921 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3127e-05 2024-04-26T03:39:10.921 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.921 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.921 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.921 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.490100+0000", 2024-04-26T03:39:10.921 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.003542084 2024-04-26T03:39:10.921 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.922 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.922 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:10.922 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.490196+0000", 2024-04-26T03:39:10.922 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.5990999999999996e-05 2024-04-26T03:39:10.922 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.922 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.922 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:10.922 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.490783+0000", 2024-04-26T03:39:10.922 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00058718699999999997 2024-04-26T03:39:10.922 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.922 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.922 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:10.922 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.494389+0000", 2024-04-26T03:39:10.922 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0036066980000000002 2024-04-26T03:39:10.923 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.923 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.923 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:10.923 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.498804+0000", 2024-04-26T03:39:10.923 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0044143150000000003 2024-04-26T03:39:10.923 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.923 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.923 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:10.923 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.498919+0000", 2024-04-26T03:39:10.923 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000114957 2024-04-26T03:39:10.923 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.923 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.923 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.923 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.924 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.924 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:18 3.3s0 3:c944ba3d:::smithi09235267-6:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:10.924 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.486578+0000", 2024-04-26T03:39:10.924 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 19.428613664, 2024-04-26T03:39:10.924 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.024780098, 2024-04-26T03:39:10.924 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:10.924 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:10.924 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:10.924 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:10.924 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:10.924 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:10.924 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 18 2024-04-26T03:39:10.924 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.924 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:10.925 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.925 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:10.925 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.486578+0000", 2024-04-26T03:39:10.925 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.925 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.925 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.925 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:10.925 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.486578+0000", 2024-04-26T03:39:10.925 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.925 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.925 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.925 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:10.925 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.486579+0000", 2024-04-26T03:39:10.925 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0529999999999999e-06 2024-04-26T03:39:10.925 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.926 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.926 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:10.926 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.486589+0000", 2024-04-26T03:39:10.926 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.9359999999999994e-06 2024-04-26T03:39:10.926 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.926 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.926 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:10.926 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.486601+0000", 2024-04-26T03:39:10.926 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2309e-05 2024-04-26T03:39:10.926 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.926 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.926 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:10.926 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.486607+0000", 2024-04-26T03:39:10.926 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5209999999999999e-06 2024-04-26T03:39:10.927 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.927 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.927 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.927 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.491290+0000", 2024-04-26T03:39:10.927 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0046821939999999998 2024-04-26T03:39:10.927 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.927 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.927 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:10.927 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.491369+0000", 2024-04-26T03:39:10.927 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9204999999999998e-05 2024-04-26T03:39:10.927 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.927 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.927 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.927 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.499348+0000", 2024-04-26T03:39:10.928 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0079792289999999991 2024-04-26T03:39:10.928 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.928 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.928 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:10.928 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.499429+0000", 2024-04-26T03:39:10.928 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.1051999999999996e-05 2024-04-26T03:39:10.928 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.928 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.928 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:10.928 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.511358+0000", 2024-04-26T03:39:10.928 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.011928599 2024-04-26T03:39:10.928 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.928 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.928 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.928 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.929 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.929 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:88 3.3s0 3:c8a9e28b:::smithi09235267-30:head [append 0~491520,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:10.929 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.543490+0000", 2024-04-26T03:39:10.929 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 19.371700897, 2024-04-26T03:39:10.929 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.01534287, 2024-04-26T03:39:10.929 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:10.929 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:10.929 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:10.929 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:10.929 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:10.929 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:10.929 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 88 2024-04-26T03:39:10.929 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.929 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:10.929 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.930 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:10.930 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.543490+0000", 2024-04-26T03:39:10.930 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.930 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.930 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.930 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:10.930 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.543490+0000", 2024-04-26T03:39:10.930 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.930 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.930 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.930 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:10.930 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.543492+0000", 2024-04-26T03:39:10.930 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5349999999999999e-06 2024-04-26T03:39:10.930 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.931 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.931 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:10.931 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.543954+0000", 2024-04-26T03:39:10.931 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00046193100000000001 2024-04-26T03:39:10.931 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.931 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.931 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:10.931 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.543967+0000", 2024-04-26T03:39:10.931 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3191e-05 2024-04-26T03:39:10.931 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.931 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.931 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:10.931 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.543972+0000", 2024-04-26T03:39:10.931 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.2950000000000001e-06 2024-04-26T03:39:10.932 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.932 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.932 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.932 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.544031+0000", 2024-04-26T03:39:10.932 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.8153999999999999e-05 2024-04-26T03:39:10.932 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.932 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.932 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:10.932 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.544176+0000", 2024-04-26T03:39:10.932 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014557900000000001 2024-04-26T03:39:10.932 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.932 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.932 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:10.932 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.545222+0000", 2024-04-26T03:39:10.933 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0010454590000000001 2024-04-26T03:39:10.933 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.933 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.933 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:10.933 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.551331+0000", 2024-04-26T03:39:10.933 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0061097180000000001 2024-04-26T03:39:10.933 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.933 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.933 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:10.933 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.558761+0000", 2024-04-26T03:39:10.933 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0074299800000000001 2024-04-26T03:39:10.933 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.933 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.933 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:10.934 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.558833+0000", 2024-04-26T03:39:10.934 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.2027999999999995e-05 2024-04-26T03:39:10.934 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.934 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.934 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.934 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.934 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.934 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:89 3.3s0 3:c8a9e28b:::smithi09235267-30:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:10.934 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.543986+0000", 2024-04-26T03:39:10.934 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 19.371204821999999, 2024-04-26T03:39:10.934 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.015002372, 2024-04-26T03:39:10.934 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:10.934 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:10.935 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:10.935 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:10.935 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:10.935 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:10.935 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 89 2024-04-26T03:39:10.935 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.935 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:10.935 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.935 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:10.935 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.543986+0000", 2024-04-26T03:39:10.935 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.935 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.935 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.935 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:10.935 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.543986+0000", 2024-04-26T03:39:10.936 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.936 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.936 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.936 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:10.936 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.543987+0000", 2024-04-26T03:39:10.936 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5899999999999996e-07 2024-04-26T03:39:10.936 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.936 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.936 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:10.936 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.543991+0000", 2024-04-26T03:39:10.936 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.58e-06 2024-04-26T03:39:10.936 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.936 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.937 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:10.937 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.543998+0000", 2024-04-26T03:39:10.937 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3119999999999996e-06 2024-04-26T03:39:10.937 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.937 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.937 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:10.937 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.544001+0000", 2024-04-26T03:39:10.937 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.399e-06 2024-04-26T03:39:10.937 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.937 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.937 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.937 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.550243+0000", 2024-04-26T03:39:10.937 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0062414180000000003 2024-04-26T03:39:10.937 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.937 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.938 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:10.938 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.550330+0000", 2024-04-26T03:39:10.938 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.6897999999999994e-05 2024-04-26T03:39:10.938 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.938 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.938 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:10.938 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.550794+0000", 2024-04-26T03:39:10.938 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00046398799999999998 2024-04-26T03:39:10.938 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.938 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.938 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:10.938 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.557987+0000", 2024-04-26T03:39:10.938 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0071936109999999999 2024-04-26T03:39:10.938 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.939 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.939 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:10.939 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.558928+0000", 2024-04-26T03:39:10.939 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000940153 2024-04-26T03:39:10.939 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.939 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.939 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:10.939 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.558989+0000", 2024-04-26T03:39:10.939 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.1353999999999996e-05 2024-04-26T03:39:10.939 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.939 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.939 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.939 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.940 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.940 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:90 3.3s0 3:c8a9e28b:::smithi09235267-30:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:10.940 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.544026+0000", 2024-04-26T03:39:10.940 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 19.371164930999999, 2024-04-26T03:39:10.940 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.017169266999999998, 2024-04-26T03:39:10.940 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:10.940 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:10.940 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:10.940 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:10.940 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:10.940 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:10.940 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 90 2024-04-26T03:39:10.940 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.940 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:10.941 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.941 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:10.941 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.544026+0000", 2024-04-26T03:39:10.941 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.941 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.941 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.941 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:10.941 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.544026+0000", 2024-04-26T03:39:10.941 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.941 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.941 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.941 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:10.941 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.544027+0000", 2024-04-26T03:39:10.941 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.0999999999999998e-07 2024-04-26T03:39:10.941 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.942 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.942 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:10.942 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.544030+0000", 2024-04-26T03:39:10.942 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6410000000000002e-06 2024-04-26T03:39:10.942 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.942 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.942 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:10.942 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.544037+0000", 2024-04-26T03:39:10.942 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.2459999999999997e-06 2024-04-26T03:39:10.942 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.942 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.942 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:10.942 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.544042+0000", 2024-04-26T03:39:10.942 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.9919999999999998e-06 2024-04-26T03:39:10.943 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.943 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.943 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.943 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.551230+0000", 2024-04-26T03:39:10.943 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0071875619999999998 2024-04-26T03:39:10.943 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.943 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.943 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:10.943 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.551311+0000", 2024-04-26T03:39:10.943 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.1182999999999998e-05 2024-04-26T03:39:10.943 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.943 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.943 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.943 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.559039+0000", 2024-04-26T03:39:10.944 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0077281479999999998 2024-04-26T03:39:10.944 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.944 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.944 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:10.944 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.559125+0000", 2024-04-26T03:39:10.944 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5678000000000002e-05 2024-04-26T03:39:10.944 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.944 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.944 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:10.944 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.561196+0000", 2024-04-26T03:39:10.944 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.002071107 2024-04-26T03:39:10.944 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.944 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.944 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.944 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.945 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.945 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:111 3.3s0 3:cad73b03:::smithi09235267-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:10.945 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.570167+0000", 2024-04-26T03:39:10.945 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 19.345024155000001, 2024-04-26T03:39:10.945 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.014356136, 2024-04-26T03:39:10.945 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:10.945 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:10.945 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:10.945 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:10.945 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:10.945 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:10.945 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 111 2024-04-26T03:39:10.945 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.945 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:10.946 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.946 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:10.946 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.570167+0000", 2024-04-26T03:39:10.946 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.946 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.946 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.946 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:10.946 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.570167+0000", 2024-04-26T03:39:10.946 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.946 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.946 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.946 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:10.946 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.570168+0000", 2024-04-26T03:39:10.946 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.0699999999999997e-07 2024-04-26T03:39:10.947 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.947 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.947 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:10.947 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.570170+0000", 2024-04-26T03:39:10.947 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6089999999999999e-06 2024-04-26T03:39:10.947 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.947 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.947 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:10.947 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.570178+0000", 2024-04-26T03:39:10.947 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.4449999999999999e-06 2024-04-26T03:39:10.947 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.947 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.947 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:10.947 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.570185+0000", 2024-04-26T03:39:10.947 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.9800000000000001e-06 2024-04-26T03:39:10.948 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.948 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.948 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.948 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.575666+0000", 2024-04-26T03:39:10.948 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0054812819999999996 2024-04-26T03:39:10.948 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.948 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.948 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:10.948 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.575751+0000", 2024-04-26T03:39:10.948 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5247999999999994e-05 2024-04-26T03:39:10.948 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.948 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.948 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.948 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.580678+0000", 2024-04-26T03:39:10.949 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0049265330000000003 2024-04-26T03:39:10.949 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.949 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.949 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:10.949 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.580798+0000", 2024-04-26T03:39:10.949 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000120365 2024-04-26T03:39:10.949 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.949 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.949 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:10.949 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.584523+0000", 2024-04-26T03:39:10.949 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0037251670000000001 2024-04-26T03:39:10.949 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.949 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.949 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.949 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.950 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.950 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.as3 35/32 ECSubWrite(tid=24, reqid=client.4556.0:150, at_version=35'13, trim_to=0'0, roll_forward_to=35'12))", 2024-04-26T03:39:10.950 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.636550+0000", 2024-04-26T03:39:10.950 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 19.278640994, 2024-04-26T03:39:10.950 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.013229930000000001, 2024-04-26T03:39:10.950 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:10.950 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:10.950 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:10.950 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:10.950 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.950 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:10.950 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.636550+0000", 2024-04-26T03:39:10.950 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.950 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.951 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.951 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:10.951 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.636550+0000", 2024-04-26T03:39:10.951 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.951 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.951 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.951 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:10.951 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.636551+0000", 2024-04-26T03:39:10.951 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4499999999999998e-07 2024-04-26T03:39:10.951 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.951 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.951 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:10.951 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.636656+0000", 2024-04-26T03:39:10.951 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00010504 2024-04-26T03:39:10.952 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.952 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.952 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:10.952 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.636667+0000", 2024-04-26T03:39:10.952 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1474000000000001e-05 2024-04-26T03:39:10.952 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.952 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.952 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:10.952 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.636683+0000", 2024-04-26T03:39:10.952 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6022999999999999e-05 2024-04-26T03:39:10.952 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.952 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.952 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.952 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.636857+0000", 2024-04-26T03:39:10.952 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00017374600000000001 2024-04-26T03:39:10.953 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.953 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.953 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:10.953 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.636865+0000", 2024-04-26T03:39:10.953 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.1000000000000004e-06 2024-04-26T03:39:10.953 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.953 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.953 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:10.953 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.649757+0000", 2024-04-26T03:39:10.953 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.01289242 2024-04-26T03:39:10.953 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.953 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.953 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:10.953 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.649780+0000", 2024-04-26T03:39:10.954 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2881999999999999e-05 2024-04-26T03:39:10.954 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.954 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.954 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.954 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.954 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.954 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.4s5 40/32 ECSubWrite(tid=22, reqid=client.4556.0:262, at_version=40'14, trim_to=0'0, roll_forward_to=35'12))", 2024-04-26T03:39:10.954 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:56.521062+0000", 2024-04-26T03:39:10.954 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 14.394129413, 2024-04-26T03:39:10.954 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.014879939, 2024-04-26T03:39:10.954 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:10.954 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:10.954 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:10.954 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:10.954 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.955 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:10.955 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.521062+0000", 2024-04-26T03:39:10.955 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.955 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.955 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.955 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:10.955 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.521062+0000", 2024-04-26T03:39:10.955 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.955 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.955 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.955 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:10.955 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.521062+0000", 2024-04-26T03:39:10.955 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4900000000000002e-07 2024-04-26T03:39:10.955 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.956 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.956 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:10.956 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.521346+0000", 2024-04-26T03:39:10.956 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00028391499999999998 2024-04-26T03:39:10.956 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.956 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.956 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:10.956 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.521358+0000", 2024-04-26T03:39:10.956 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1567e-05 2024-04-26T03:39:10.956 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.956 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.956 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:10.956 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.521365+0000", 2024-04-26T03:39:10.956 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.7849999999999992e-06 2024-04-26T03:39:10.956 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.957 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.957 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.957 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.521432+0000", 2024-04-26T03:39:10.957 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.6963000000000002e-05 2024-04-26T03:39:10.957 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.957 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.957 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:10.957 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.521442+0000", 2024-04-26T03:39:10.957 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0098000000000001e-05 2024-04-26T03:39:10.957 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.957 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.958 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:10.958 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.535904+0000", 2024-04-26T03:39:10.958 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.014461821999999999 2024-04-26T03:39:10.958 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.958 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.958 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:10.958 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.535942+0000", 2024-04-26T03:39:10.958 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7540000000000003e-05 2024-04-26T03:39:10.958 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.958 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.958 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.958 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.958 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.958 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:312 3.3s0 3:c8a9e28b:::smithi09235267-30:head [append 0~475136] snapc 6=[6,5] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-26T03:39:10.959 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:59.547999+0000", 2024-04-26T03:39:10.959 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 11.367192567, 2024-04-26T03:39:10.959 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.013877782, 2024-04-26T03:39:10.959 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:10.959 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:10.959 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:10.959 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:10.959 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:10.959 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:10.959 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 312 2024-04-26T03:39:10.959 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.959 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:10.959 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.959 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:10.960 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.547999+0000", 2024-04-26T03:39:10.960 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.960 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.960 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.960 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:10.960 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.547999+0000", 2024-04-26T03:39:10.960 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.960 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.960 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.960 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:10.960 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.548000+0000", 2024-04-26T03:39:10.960 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7370000000000001e-06 2024-04-26T03:39:10.960 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.961 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.961 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:10.961 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.548517+0000", 2024-04-26T03:39:10.961 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00051695699999999996 2024-04-26T03:39:10.961 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.961 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.961 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:10.961 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.548533+0000", 2024-04-26T03:39:10.961 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5985e-05 2024-04-26T03:39:10.961 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.961 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.961 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:10.961 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.548540+0000", 2024-04-26T03:39:10.961 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.0639999999999996e-06 2024-04-26T03:39:10.961 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.962 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.962 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.962 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.548614+0000", 2024-04-26T03:39:10.962 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3194000000000004e-05 2024-04-26T03:39:10.962 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.962 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.962 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:10.962 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.548715+0000", 2024-04-26T03:39:10.962 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000101309 2024-04-26T03:39:10.962 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.962 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.962 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:10.962 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.550003+0000", 2024-04-26T03:39:10.962 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001287722 2024-04-26T03:39:10.963 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.963 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.963 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:10.963 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.561581+0000", 2024-04-26T03:39:10.963 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.011578774999999999 2024-04-26T03:39:10.963 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.963 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.963 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:10.963 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.561791+0000", 2024-04-26T03:39:10.963 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020959900000000001 2024-04-26T03:39:10.963 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.963 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.963 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:10.963 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.561877+0000", 2024-04-26T03:39:10.964 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5439999999999995e-05 2024-04-26T03:39:10.964 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.964 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.964 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.964 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.964 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.964 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:314 3.3s0 3:c8a9e28b:::smithi09235267-30:head [append 0~147456] snapc 6=[6,5] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-26T03:39:10.964 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:59.552775+0000", 2024-04-26T03:39:10.964 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 11.362416291000001, 2024-04-26T03:39:10.964 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.030282514, 2024-04-26T03:39:10.964 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:10.964 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:10.964 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:10.964 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:10.965 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:10.965 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:10.965 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 314 2024-04-26T03:39:10.965 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.965 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:10.965 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.965 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:10.965 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552775+0000", 2024-04-26T03:39:10.965 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.965 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.965 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.965 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:10.965 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552775+0000", 2024-04-26T03:39:10.965 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.966 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.966 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.966 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:10.966 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552776+0000", 2024-04-26T03:39:10.966 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.026e-06 2024-04-26T03:39:10.966 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.966 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.966 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:10.966 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552935+0000", 2024-04-26T03:39:10.966 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00015929799999999999 2024-04-26T03:39:10.966 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.966 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.966 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:10.966 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552948+0000", 2024-04-26T03:39:10.967 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2716e-05 2024-04-26T03:39:10.967 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.967 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.967 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:10.967 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552955+0000", 2024-04-26T03:39:10.967 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.9609999999999999e-06 2024-04-26T03:39:10.967 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.967 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.967 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.967 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.562092+0000", 2024-04-26T03:39:10.967 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0091373979999999997 2024-04-26T03:39:10.967 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.967 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.967 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:10.967 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.562164+0000", 2024-04-26T03:39:10.968 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.1767999999999995e-05 2024-04-26T03:39:10.968 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.968 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.968 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:10.968 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.562591+0000", 2024-04-26T03:39:10.968 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00042723799999999998 2024-04-26T03:39:10.968 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.968 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.968 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:10.968 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.564915+0000", 2024-04-26T03:39:10.968 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0023239340000000002 2024-04-26T03:39:10.968 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.968 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.969 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:10.969 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.583009+0000", 2024-04-26T03:39:10.969 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.018093609 2024-04-26T03:39:10.969 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.969 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.969 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:10.969 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.583058+0000", 2024-04-26T03:39:10.969 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.8566000000000002e-05 2024-04-26T03:39:10.969 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.969 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.969 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.969 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.969 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.969 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:315 3.3s0 3:c8a9e28b:::smithi09235267-30:head [setxattr _header (56) in=7b] snapc 6=[6,5] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-26T03:39:10.969 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:59.552967+0000", 2024-04-26T03:39:10.970 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 11.362224757, 2024-04-26T03:39:10.970 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.030505498999999998, 2024-04-26T03:39:10.970 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:10.970 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:10.970 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:10.970 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:10.970 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:10.970 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:10.970 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 315 2024-04-26T03:39:10.970 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.970 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:10.970 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.970 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:10.970 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552967+0000", 2024-04-26T03:39:10.971 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.971 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.971 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.971 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:10.971 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552967+0000", 2024-04-26T03:39:10.971 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.971 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.971 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.971 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:10.971 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552967+0000", 2024-04-26T03:39:10.971 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.37e-07 2024-04-26T03:39:10.971 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.971 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.971 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:10.971 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552971+0000", 2024-04-26T03:39:10.972 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9580000000000001e-06 2024-04-26T03:39:10.972 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.972 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.972 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:10.972 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552980+0000", 2024-04-26T03:39:10.972 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.4549999999999995e-06 2024-04-26T03:39:10.972 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.972 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.972 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:10.972 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552985+0000", 2024-04-26T03:39:10.972 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.5119999999999999e-06 2024-04-26T03:39:10.972 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.972 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.972 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.973 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.563965+0000", 2024-04-26T03:39:10.973 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.010980177000000001 2024-04-26T03:39:10.973 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.973 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.973 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:10.973 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.564028+0000", 2024-04-26T03:39:10.973 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.2607000000000006e-05 2024-04-26T03:39:10.973 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.973 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.973 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:10.973 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.564435+0000", 2024-04-26T03:39:10.973 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00040658399999999998 2024-04-26T03:39:10.973 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.973 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.974 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:10.974 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.564918+0000", 2024-04-26T03:39:10.974 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00048333499999999999 2024-04-26T03:39:10.974 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.974 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.974 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:10.974 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.583395+0000", 2024-04-26T03:39:10.974 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.018476764999999999 2024-04-26T03:39:10.974 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.974 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.974 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:10.974 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.583472+0000", 2024-04-26T03:39:10.975 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.7368999999999997e-05 2024-04-26T03:39:10.975 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.975 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.975 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.975 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.975 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.975 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:316 3.3s0 3:c8a9e28b:::smithi09235267-30:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e43)", 2024-04-26T03:39:10.975 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:59.552993+0000", 2024-04-26T03:39:10.975 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 11.362198549, 2024-04-26T03:39:10.975 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.032226746000000001, 2024-04-26T03:39:10.976 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:10.976 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:10.976 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:10.976 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:10.976 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:10.976 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:10.976 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 316 2024-04-26T03:39:10.976 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.976 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:10.976 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.976 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:10.976 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552993+0000", 2024-04-26T03:39:10.976 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.976 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.977 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.977 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:10.977 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552993+0000", 2024-04-26T03:39:10.977 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.977 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.977 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.977 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:10.977 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552993+0000", 2024-04-26T03:39:10.977 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7299999999999996e-07 2024-04-26T03:39:10.977 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.977 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.977 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:10.977 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552996+0000", 2024-04-26T03:39:10.977 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.92e-06 2024-04-26T03:39:10.978 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.978 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.978 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:10.978 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.553005+0000", 2024-04-26T03:39:10.978 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.585e-06 2024-04-26T03:39:10.978 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.978 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.978 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:10.978 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.553010+0000", 2024-04-26T03:39:10.978 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.0719999999999999e-06 2024-04-26T03:39:10.978 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.978 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.978 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.978 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.564680+0000", 2024-04-26T03:39:10.979 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.011669736 2024-04-26T03:39:10.979 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.979 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.979 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:10.979 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.564736+0000", 2024-04-26T03:39:10.979 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.6113000000000003e-05 2024-04-26T03:39:10.979 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.979 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.979 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.979 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.583732+0000", 2024-04-26T03:39:10.979 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.018995920999999999 2024-04-26T03:39:10.979 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.979 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.980 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:10.980 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.583823+0000", 2024-04-26T03:39:10.980 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1335999999999994e-05 2024-04-26T03:39:10.980 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.980 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.980 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:10.980 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.585220+0000", 2024-04-26T03:39:10.980 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00139649 2024-04-26T03:39:10.980 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.980 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.980 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.980 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.980 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.980 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.7s5 49/44 ECSubWrite(tid=83, reqid=osd.4.0:83, at_version=49'10, trim_to=0'0, roll_forward_to=48'8))", 2024-04-26T03:39:10.980 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:04.696553+0000", 2024-04-26T03:39:10.981 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.2186386809999998, 2024-04-26T03:39:10.981 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.013590896999999999, 2024-04-26T03:39:10.981 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:10.981 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:10.981 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:10.981 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:10.981 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.981 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:10.981 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.696553+0000", 2024-04-26T03:39:10.981 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.981 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.981 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.981 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:10.982 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.696553+0000", 2024-04-26T03:39:10.982 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.982 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.982 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.982 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:10.982 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.696553+0000", 2024-04-26T03:39:10.982 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.2600000000000002e-07 2024-04-26T03:39:10.982 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.982 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.982 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:10.982 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.696575+0000", 2024-04-26T03:39:10.982 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1441e-05 2024-04-26T03:39:10.982 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.982 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.983 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:10.983 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.696587+0000", 2024-04-26T03:39:10.983 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1892000000000001e-05 2024-04-26T03:39:10.983 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.983 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.983 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:10.983 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.696594+0000", 2024-04-26T03:39:10.983 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8310000000000001e-06 2024-04-26T03:39:10.983 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.983 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.983 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.983 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.696660+0000", 2024-04-26T03:39:10.983 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5550999999999996e-05 2024-04-26T03:39:10.983 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.983 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.985 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:10.985 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.696669+0000", 2024-04-26T03:39:10.985 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1940000000000004e-06 2024-04-26T03:39:10.985 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.985 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.985 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:10.985 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.710095+0000", 2024-04-26T03:39:10.985 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.013425906 2024-04-26T03:39:10.986 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.986 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.986 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:10.986 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.710144+0000", 2024-04-26T03:39:10.986 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.8455999999999998e-05 2024-04-26T03:39:10.986 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.986 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.986 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.986 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.986 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.986 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.2s5 49/44 ECSubWrite(tid=32, reqid=client.4556.0:433, at_version=49'20, trim_to=0'0, roll_forward_to=48'19))", 2024-04-26T03:39:10.986 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:04.700213+0000", 2024-04-26T03:39:10.986 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.2149779330000001, 2024-04-26T03:39:10.986 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.018834018000000001, 2024-04-26T03:39:10.987 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:10.987 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:10.987 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:10.987 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:10.987 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.987 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:10.987 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700213+0000", 2024-04-26T03:39:10.987 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.987 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.987 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.987 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:10.987 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700213+0000", 2024-04-26T03:39:10.987 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.987 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.988 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.988 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:10.988 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700214+0000", 2024-04-26T03:39:10.988 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.48e-07 2024-04-26T03:39:10.988 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.988 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.988 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:10.988 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700972+0000", 2024-04-26T03:39:10.988 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00075845299999999997 2024-04-26T03:39:10.988 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.988 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.988 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:10.988 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700983+0000", 2024-04-26T03:39:10.988 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1105e-05 2024-04-26T03:39:10.989 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.989 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.989 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:10.989 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700991+0000", 2024-04-26T03:39:10.989 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9140000000000008e-06 2024-04-26T03:39:10.989 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.989 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.989 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.989 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.701100+0000", 2024-04-26T03:39:10.989 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000108915 2024-04-26T03:39:10.989 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.989 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.989 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:10.989 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.701120+0000", 2024-04-26T03:39:10.989 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9811000000000001e-05 2024-04-26T03:39:10.990 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.990 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.990 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:10.990 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.718957+0000", 2024-04-26T03:39:10.990 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.017837604 2024-04-26T03:39:10.990 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.990 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.990 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:10.990 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.719047+0000", 2024-04-26T03:39:10.990 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.9968000000000003e-05 2024-04-26T03:39:10.990 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.990 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.990 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.990 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.991 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.991 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.bs2 50/32 ECSubWrite(tid=88, reqid=osd.4.0:88, at_version=50'48, trim_to=0'0, roll_forward_to=49'46))", 2024-04-26T03:39:10.991 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:05.698893+0000", 2024-04-26T03:39:10.991 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 5.2162979930000004, 2024-04-26T03:39:10.991 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.015901432, 2024-04-26T03:39:10.991 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:10.991 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:10.991 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:10.991 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:10.991 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.991 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:10.991 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.698893+0000", 2024-04-26T03:39:10.991 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.991 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.992 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.992 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:10.992 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.698893+0000", 2024-04-26T03:39:10.992 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.992 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.992 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.992 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:10.992 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.698894+0000", 2024-04-26T03:39:10.992 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.1e-07 2024-04-26T03:39:10.992 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.992 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.992 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:10.992 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.698914+0000", 2024-04-26T03:39:10.992 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0613000000000001e-05 2024-04-26T03:39:10.992 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.993 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.993 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:10.993 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.698924+0000", 2024-04-26T03:39:10.993 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.3859999999999996e-06 2024-04-26T03:39:10.993 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.993 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.993 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:10.993 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.698931+0000", 2024-04-26T03:39:10.993 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.0829999999999998e-06 2024-04-26T03:39:10.993 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.993 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.993 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.993 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.699137+0000", 2024-04-26T03:39:10.993 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020594 2024-04-26T03:39:10.994 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.994 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.994 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:10.994 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.699153+0000", 2024-04-26T03:39:10.994 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6218999999999998e-05 2024-04-26T03:39:10.994 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.994 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.994 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:10.994 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.714755+0000", 2024-04-26T03:39:10.994 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.015602517 2024-04-26T03:39:10.994 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.994 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.994 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:10.994 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.714795+0000", 2024-04-26T03:39:10.995 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9363999999999998e-05 2024-04-26T03:39:10.995 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.995 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:10.995 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:10.995 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.995 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.995 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:446 3.13s0 3:c944ba3d:::smithi09235267-6:head [append 0~720896] snapc 8=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:10.995 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:05.700096+0000", 2024-04-26T03:39:10.995 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 5.2150954120000002, 2024-04-26T03:39:10.995 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.019772888999999998, 2024-04-26T03:39:10.995 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:10.995 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:10.995 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:10.995 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:10.995 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:10.996 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:10.996 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 446 2024-04-26T03:39:10.996 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.996 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:10.996 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.996 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:10.996 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.700096+0000", 2024-04-26T03:39:10.996 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.996 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.996 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.996 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:10.996 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.700096+0000", 2024-04-26T03:39:10.996 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:10.997 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.997 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.997 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:10.997 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.700098+0000", 2024-04-26T03:39:10.997 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.464e-06 2024-04-26T03:39:10.997 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.997 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.997 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:10.997 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.700859+0000", 2024-04-26T03:39:10.997 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00076090299999999995 2024-04-26T03:39:10.997 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.997 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.997 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:10.997 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.700875+0000", 2024-04-26T03:39:10.997 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5299e-05 2024-04-26T03:39:10.998 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.998 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.998 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:10.998 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.700881+0000", 2024-04-26T03:39:10.998 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.528e-06 2024-04-26T03:39:10.998 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.998 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.998 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:10.998 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.700945+0000", 2024-04-26T03:39:10.998 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.3744999999999994e-05 2024-04-26T03:39:10.998 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.998 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.998 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:10.998 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.701071+0000", 2024-04-26T03:39:10.999 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012569600000000001 2024-04-26T03:39:10.999 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.999 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.999 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:10.999 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.702209+0000", 2024-04-26T03:39:10.999 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001138266 2024-04-26T03:39:10.999 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.999 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.999 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:10.999 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.714847+0000", 2024-04-26T03:39:10.999 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.012638508 2024-04-26T03:39:10.999 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:10.999 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:10.999 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:11.000 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.719810+0000", 2024-04-26T03:39:11.000 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0049625989999999998 2024-04-26T03:39:11.000 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.000 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.000 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:11.000 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.719869+0000", 2024-04-26T03:39:11.000 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.8881000000000001e-05 2024-04-26T03:39:11.000 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.000 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:11.000 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.000 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.000 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.000 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:447 3.13s0 3:c944ba3d:::smithi09235267-6:head [append 0~442368] snapc 8=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:11.000 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:05.702913+0000", 2024-04-26T03:39:11.000 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 5.2122787109999997, 2024-04-26T03:39:11.001 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.01837453, 2024-04-26T03:39:11.001 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:11.001 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:11.001 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:11.001 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:11.001 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:11.001 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:11.001 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 447 2024-04-26T03:39:11.001 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.001 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:11.001 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.001 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:11.001 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.702913+0000", 2024-04-26T03:39:11.001 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.002 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.002 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.002 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:11.002 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.702913+0000", 2024-04-26T03:39:11.002 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.002 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.002 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.002 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:11.002 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.702915+0000", 2024-04-26T03:39:11.002 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0360000000000001e-06 2024-04-26T03:39:11.002 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.002 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.002 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:11.002 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.703366+0000", 2024-04-26T03:39:11.003 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00045175800000000002 2024-04-26T03:39:11.003 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.003 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.003 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:11.003 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.703382+0000", 2024-04-26T03:39:11.003 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5707999999999999e-05 2024-04-26T03:39:11.003 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.003 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.003 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:11.003 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.703391+0000", 2024-04-26T03:39:11.003 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.3889999999999994e-06 2024-04-26T03:39:11.003 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.003 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.003 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:11.003 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.710509+0000", 2024-04-26T03:39:11.004 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0071172650000000002 2024-04-26T03:39:11.004 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.004 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.004 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:11.004 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.710619+0000", 2024-04-26T03:39:11.004 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00011061 2024-04-26T03:39:11.004 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.004 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.004 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:11.004 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.711319+0000", 2024-04-26T03:39:11.004 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00070012799999999995 2024-04-26T03:39:11.004 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.004 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.004 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:11.005 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.719699+0000", 2024-04-26T03:39:11.005 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0083798540000000008 2024-04-26T03:39:11.005 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.005 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.005 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:11.005 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.721236+0000", 2024-04-26T03:39:11.005 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0015364129999999999 2024-04-26T03:39:11.005 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.005 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.005 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:11.005 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.721287+0000", 2024-04-26T03:39:11.005 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.1369000000000002e-05 2024-04-26T03:39:11.005 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.005 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:11.006 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.006 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.006 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.006 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:448 3.13s0 3:c944ba3d:::smithi09235267-6:head [append 0~327680] snapc 8=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:11.006 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:05.704961+0000", 2024-04-26T03:39:11.006 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 5.2102303719999998, 2024-04-26T03:39:11.006 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.019901015000000001, 2024-04-26T03:39:11.006 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:11.006 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:11.006 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:11.006 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:11.006 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:11.006 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:11.006 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 448 2024-04-26T03:39:11.007 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.007 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:11.007 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.007 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:11.007 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.704961+0000", 2024-04-26T03:39:11.007 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.007 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.007 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.007 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:11.007 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.704961+0000", 2024-04-26T03:39:11.007 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.007 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.007 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.007 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:11.008 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.704963+0000", 2024-04-26T03:39:11.008 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.81e-06 2024-04-26T03:39:11.008 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.008 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.008 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:11.008 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.705283+0000", 2024-04-26T03:39:11.008 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00032049100000000001 2024-04-26T03:39:11.008 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.008 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.008 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:11.008 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.705309+0000", 2024-04-26T03:39:11.008 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5789999999999999e-05 2024-04-26T03:39:11.008 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.008 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.008 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:11.009 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.705352+0000", 2024-04-26T03:39:11.009 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.2865999999999999e-05 2024-04-26T03:39:11.009 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.009 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.009 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:11.009 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.714894+0000", 2024-04-26T03:39:11.009 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.009542109 2024-04-26T03:39:11.009 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.009 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.009 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:11.009 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.715004+0000", 2024-04-26T03:39:11.009 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00010983400000000001 2024-04-26T03:39:11.009 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.009 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.010 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:11.010 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.715894+0000", 2024-04-26T03:39:11.010 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00089006399999999998 2024-04-26T03:39:11.010 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.010 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.010 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:11.010 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.720882+0000", 2024-04-26T03:39:11.010 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0049879499999999997 2024-04-26T03:39:11.010 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.010 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.010 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:11.010 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.724816+0000", 2024-04-26T03:39:11.010 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0039342830000000002 2024-04-26T03:39:11.010 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.011 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.011 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:11.011 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.724862+0000", 2024-04-26T03:39:11.011 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.5818000000000001e-05 2024-04-26T03:39:11.011 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.011 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:11.011 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.011 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.011 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.011 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:449 3.13s0 3:c944ba3d:::smithi09235267-6:head [setxattr _header (55) in=7b] snapc 8=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:11.011 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:05.705415+0000", 2024-04-26T03:39:11.011 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 5.2097761519999999, 2024-04-26T03:39:11.011 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.021062219, 2024-04-26T03:39:11.011 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:11.011 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:11.012 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:11.012 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:11.012 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:11.012 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:11.012 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 449 2024-04-26T03:39:11.012 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.012 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:11.012 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.012 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:11.012 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.705415+0000", 2024-04-26T03:39:11.012 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.012 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.012 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.012 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:11.013 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.705415+0000", 2024-04-26T03:39:11.013 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.013 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.013 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.013 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:11.013 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.705416+0000", 2024-04-26T03:39:11.013 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5099999999999998e-07 2024-04-26T03:39:11.013 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.013 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.013 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:11.013 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.705422+0000", 2024-04-26T03:39:11.013 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.5079999999999996e-06 2024-04-26T03:39:11.013 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.013 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.013 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:11.014 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.705507+0000", 2024-04-26T03:39:11.014 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5282000000000001e-05 2024-04-26T03:39:11.014 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.014 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.014 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:11.014 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.705525+0000", 2024-04-26T03:39:11.014 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8173999999999999e-05 2024-04-26T03:39:11.014 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.014 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.014 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:11.014 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.720102+0000", 2024-04-26T03:39:11.014 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.014577345 2024-04-26T03:39:11.015 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.015 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.015 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:11.015 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.720161+0000", 2024-04-26T03:39:11.015 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.9070000000000002e-05 2024-04-26T03:39:11.015 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.015 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.015 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:11.015 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.720462+0000", 2024-04-26T03:39:11.015 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00030043300000000002 2024-04-26T03:39:11.015 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.015 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.015 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:11.015 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.722295+0000", 2024-04-26T03:39:11.015 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001833339 2024-04-26T03:39:11.016 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.016 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.016 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:11.016 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.726405+0000", 2024-04-26T03:39:11.016 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0041094959999999998 2024-04-26T03:39:11.016 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.016 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.016 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:11.016 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.726477+0000", 2024-04-26T03:39:11.016 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.2720999999999997e-05 2024-04-26T03:39:11.016 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.016 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:11.016 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.016 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.016 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.017 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:450 3.13s0 3:c944ba3d:::smithi09235267-6:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:11.017 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:05.705540+0000", 2024-04-26T03:39:11.017 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 5.2096513839999998, 2024-04-26T03:39:11.017 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.022180959, 2024-04-26T03:39:11.017 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:11.017 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:11.017 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:11.017 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:11.017 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:11.017 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:11.017 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 450 2024-04-26T03:39:11.017 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.017 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:11.017 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.018 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:11.018 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.705540+0000", 2024-04-26T03:39:11.018 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.018 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.018 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.018 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:11.018 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.705540+0000", 2024-04-26T03:39:11.018 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.018 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.018 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.018 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:11.018 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.705541+0000", 2024-04-26T03:39:11.018 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.6000000000000003e-07 2024-04-26T03:39:11.018 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.019 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.019 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:11.019 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.705655+0000", 2024-04-26T03:39:11.019 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000114528 2024-04-26T03:39:11.019 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.019 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.019 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:11.019 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.705668+0000", 2024-04-26T03:39:11.019 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2712e-05 2024-04-26T03:39:11.019 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.019 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.019 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:11.019 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.705676+0000", 2024-04-26T03:39:11.019 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9179999999999994e-06 2024-04-26T03:39:11.019 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.020 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.020 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:11.020 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.720674+0000", 2024-04-26T03:39:11.020 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.014998177 2024-04-26T03:39:11.020 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.020 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.020 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:11.020 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.720807+0000", 2024-04-26T03:39:11.020 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000133053 2024-04-26T03:39:11.020 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.020 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.020 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:11.020 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.726766+0000", 2024-04-26T03:39:11.020 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0059592270000000001 2024-04-26T03:39:11.021 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.021 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.021 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:11.021 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.726868+0000", 2024-04-26T03:39:11.021 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000101931 2024-04-26T03:39:11.021 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.021 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.021 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:11.021 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.727721+0000", 2024-04-26T03:39:11.021 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00085265299999999998 2024-04-26T03:39:11.021 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.021 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:11.021 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.021 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.022 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:11.022 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:39:11.023 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-04-26T03:39:11.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:39:11.026+0000 7f3b6acb9640 0 --2- 172.21.15.28:0/2946896042 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b6414cf90 0x7f3b640c6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-04-26T03:39:11.047 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:39:11.021+0000 7fb78c544640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31956) UID: 0 2024-04-26T03:39:11.124 INFO:tasks.ceph.osd.15.smithi092.stderr:2024-04-26T03:39:11.122+0000 7f6d12dec640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 31824) UID: 0 2024-04-26T03:39:11.171 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:39:11.171 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-04-26T03:39:11.171 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 0 2024-04-26T03:39:11.171 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:39:11.180 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-04-26T03:39:11.224 INFO:tasks.ceph.osd.4.smithi028.stderr:2024-04-26T03:39:11.223+0000 7fd9511b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31963) UID: 0 2024-04-26T03:39:11.287 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:39:11.287 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-04-26T03:39:11.287 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-04-26T03:39:11.287 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-04-26T03:39:11.287 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:39:11.295 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-04-26T03:39:11.343 INFO:tasks.ceph.osd.10.smithi066.stderr:2024-04-26T03:39:11.342+0000 7f130fcc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 31763) UID: 0 2024-04-26T03:39:11.405 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:39:11.405 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-04-26T03:39:11.405 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-04-26T03:39:11.405 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-04-26T03:39:11.405 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.405 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:3 3.2s0 3:42fcabaa:::smithi09235267-1:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:11.405 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.480913+0000", 2024-04-26T03:39:11.405 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 19.921082084999998, 2024-04-26T03:39:11.405 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.012266038999999999, 2024-04-26T03:39:11.406 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:11.406 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:11.406 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:11.406 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:11.406 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:11.406 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:11.407 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 3 2024-04-26T03:39:11.407 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.407 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:11.407 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.407 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:11.407 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.480913+0000", 2024-04-26T03:39:11.407 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.407 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.407 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.407 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:11.408 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.480913+0000", 2024-04-26T03:39:11.408 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.408 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.408 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.408 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:11.408 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.480914+0000", 2024-04-26T03:39:11.408 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.7999999999999996e-07 2024-04-26T03:39:11.408 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.408 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.408 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:11.408 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.480917+0000", 2024-04-26T03:39:11.409 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.366e-06 2024-04-26T03:39:11.409 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.409 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.409 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:11.409 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.480925+0000", 2024-04-26T03:39:11.409 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9759999999999995e-06 2024-04-26T03:39:11.409 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.409 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.409 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:11.409 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.480932+0000", 2024-04-26T03:39:11.409 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.0339999999999999e-06 2024-04-26T03:39:11.410 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.410 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.410 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:11.410 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.483324+0000", 2024-04-26T03:39:11.410 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.002392323 2024-04-26T03:39:11.411 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.411 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.411 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:11.412 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.483402+0000", 2024-04-26T03:39:11.412 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.7745999999999997e-05 2024-04-26T03:39:11.413 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.413 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.413 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:11.413 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.487467+0000", 2024-04-26T03:39:11.413 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0040650310000000002 2024-04-26T03:39:11.413 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.414 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.414 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:11.414 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.487555+0000", 2024-04-26T03:39:11.414 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.8271999999999998e-05 2024-04-26T03:39:11.414 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.414 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.414 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:11.414 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.493179+0000", 2024-04-26T03:39:11.414 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0056238110000000003 2024-04-26T03:39:11.414 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.414 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:11.414 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.415 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.415 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.415 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:79 3.2s0 3:41b9d80c:::smithi09235267-27:head [append 0~442368,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:11.415 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.531542+0000", 2024-04-26T03:39:11.415 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 19.870453032, 2024-04-26T03:39:11.415 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.013543507999999999, 2024-04-26T03:39:11.415 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:11.415 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:11.415 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:11.415 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:11.415 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:11.415 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:11.415 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 79 2024-04-26T03:39:11.416 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.416 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:11.416 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.416 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:11.416 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.531542+0000", 2024-04-26T03:39:11.416 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.416 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.416 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.416 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:11.416 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.531542+0000", 2024-04-26T03:39:11.416 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.416 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.416 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.416 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:11.417 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.531544+0000", 2024-04-26T03:39:11.417 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9719999999999999e-06 2024-04-26T03:39:11.417 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.417 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.417 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:11.417 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.531978+0000", 2024-04-26T03:39:11.417 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00043396899999999998 2024-04-26T03:39:11.417 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.417 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.417 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:11.417 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.531990+0000", 2024-04-26T03:39:11.418 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2150999999999999e-05 2024-04-26T03:39:11.418 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.418 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.418 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:11.418 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.531997+0000", 2024-04-26T03:39:11.418 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.6320000000000002e-06 2024-04-26T03:39:11.418 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.419 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.419 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:11.419 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532044+0000", 2024-04-26T03:39:11.419 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.7095999999999998e-05 2024-04-26T03:39:11.419 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.419 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.419 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:11.419 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532156+0000", 2024-04-26T03:39:11.419 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000112131 2024-04-26T03:39:11.419 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.420 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.420 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:11.420 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532990+0000", 2024-04-26T03:39:11.420 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00083437099999999998 2024-04-26T03:39:11.420 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.420 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.420 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:11.420 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.537471+0000", 2024-04-26T03:39:11.420 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0044802160000000004 2024-04-26T03:39:11.420 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.420 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.421 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:11.421 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.545008+0000", 2024-04-26T03:39:11.421 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0075372089999999996 2024-04-26T03:39:11.421 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.421 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.421 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:11.421 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.545086+0000", 2024-04-26T03:39:11.422 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.7761000000000003e-05 2024-04-26T03:39:11.422 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.422 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:11.422 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.422 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.422 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.422 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:80 3.2s0 3:41b9d80c:::smithi09235267-27:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:11.422 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.532011+0000", 2024-04-26T03:39:11.422 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 19.869983703999999, 2024-04-26T03:39:11.422 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.013370922, 2024-04-26T03:39:11.423 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:11.423 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:11.423 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:11.423 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:11.423 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:11.423 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:11.423 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 80 2024-04-26T03:39:11.423 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.423 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:11.423 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.423 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:11.423 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532011+0000", 2024-04-26T03:39:11.423 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.424 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.424 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.424 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:11.424 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532011+0000", 2024-04-26T03:39:11.424 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.424 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.424 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.424 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:11.425 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532012+0000", 2024-04-26T03:39:11.425 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.1900000000000003e-07 2024-04-26T03:39:11.425 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.425 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.425 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:11.425 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532015+0000", 2024-04-26T03:39:11.425 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5919999999999999e-06 2024-04-26T03:39:11.425 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.425 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.425 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:11.425 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532025+0000", 2024-04-26T03:39:11.426 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0132999999999999e-05 2024-04-26T03:39:11.426 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.426 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.426 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:11.426 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532029+0000", 2024-04-26T03:39:11.426 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9820000000000002e-06 2024-04-26T03:39:11.426 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.426 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.426 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:11.426 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.536416+0000", 2024-04-26T03:39:11.426 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0043876480000000001 2024-04-26T03:39:11.427 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.427 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.427 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:11.427 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.536558+0000", 2024-04-26T03:39:11.427 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000141994 2024-04-26T03:39:11.427 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.427 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.427 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:11.427 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.537081+0000", 2024-04-26T03:39:11.427 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00052260200000000005 2024-04-26T03:39:11.427 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.428 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.428 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:11.428 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.537579+0000", 2024-04-26T03:39:11.428 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00049790100000000003 2024-04-26T03:39:11.428 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.428 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.428 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:11.428 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.545304+0000", 2024-04-26T03:39:11.428 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.007724936 2024-04-26T03:39:11.428 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.429 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.429 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:11.429 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.545382+0000", 2024-04-26T03:39:11.429 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8615000000000005e-05 2024-04-26T03:39:11.429 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.429 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:11.429 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.429 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.429 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.429 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:81 3.2s0 3:41b9d80c:::smithi09235267-27:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:11.429 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.532034+0000", 2024-04-26T03:39:11.430 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 19.86996079, 2024-04-26T03:39:11.430 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.015088236, 2024-04-26T03:39:11.430 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:11.430 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:11.430 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:11.430 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:11.430 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:11.430 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:11.430 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 81 2024-04-26T03:39:11.430 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.430 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:11.430 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.431 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:11.431 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532034+0000", 2024-04-26T03:39:11.431 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.431 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.431 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.431 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:11.431 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532034+0000", 2024-04-26T03:39:11.431 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.431 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.431 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.431 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:11.432 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532035+0000", 2024-04-26T03:39:11.432 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.22e-07 2024-04-26T03:39:11.432 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.432 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.433 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:11.433 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532036+0000", 2024-04-26T03:39:11.433 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6279999999999999e-06 2024-04-26T03:39:11.433 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.433 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.433 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:11.433 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532041+0000", 2024-04-26T03:39:11.433 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.391e-06 2024-04-26T03:39:11.433 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.433 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.433 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:11.434 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532046+0000", 2024-04-26T03:39:11.434 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.4090000000000002e-06 2024-04-26T03:39:11.435 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.435 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.435 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:11.435 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.537306+0000", 2024-04-26T03:39:11.435 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0052594319999999996 2024-04-26T03:39:11.435 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.435 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.435 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:11.435 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.537400+0000", 2024-04-26T03:39:11.436 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.4753999999999994e-05 2024-04-26T03:39:11.436 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.436 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.436 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:11.436 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.545720+0000", 2024-04-26T03:39:11.436 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.008319557 2024-04-26T03:39:11.436 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.436 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.436 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:11.436 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.545802+0000", 2024-04-26T03:39:11.437 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.1898999999999996e-05 2024-04-26T03:39:11.437 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.437 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.437 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:11.437 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.547123+0000", 2024-04-26T03:39:11.437 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0013208440000000001 2024-04-26T03:39:11.437 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.437 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:11.437 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.437 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.438 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.438 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:139 3.2s0 3:450a1cc5:::smithi09235267-47:head [append 0~786432,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:11.438 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.619094+0000", 2024-04-26T03:39:11.438 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 19.782900743999999, 2024-04-26T03:39:11.438 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.015918081000000001, 2024-04-26T03:39:11.438 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:11.438 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:11.438 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:11.438 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:11.438 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:11.438 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:11.439 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 139 2024-04-26T03:39:11.439 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.439 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:11.439 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.439 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:11.439 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619094+0000", 2024-04-26T03:39:11.439 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.439 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.439 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.439 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:11.439 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619094+0000", 2024-04-26T03:39:11.439 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.440 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.440 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.440 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:11.440 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619096+0000", 2024-04-26T03:39:11.440 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6559999999999999e-06 2024-04-26T03:39:11.440 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.440 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.440 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:11.440 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619809+0000", 2024-04-26T03:39:11.440 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00071288600000000003 2024-04-26T03:39:11.440 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.440 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.440 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:11.441 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619821+0000", 2024-04-26T03:39:11.441 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1586000000000001e-05 2024-04-26T03:39:11.441 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.441 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.441 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:11.441 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619826+0000", 2024-04-26T03:39:11.441 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.3870000000000003e-06 2024-04-26T03:39:11.441 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.441 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.441 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:11.441 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619879+0000", 2024-04-26T03:39:11.441 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.2660000000000001e-05 2024-04-26T03:39:11.441 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.442 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.442 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:11.442 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619966+0000", 2024-04-26T03:39:11.442 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7777999999999999e-05 2024-04-26T03:39:11.442 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.442 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.442 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:11.442 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.620812+0000", 2024-04-26T03:39:11.442 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00084571199999999998 2024-04-26T03:39:11.442 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.442 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.442 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:11.443 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.627482+0000", 2024-04-26T03:39:11.443 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0066701590000000002 2024-04-26T03:39:11.443 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.443 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.443 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:11.443 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.634919+0000", 2024-04-26T03:39:11.443 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.007437056 2024-04-26T03:39:11.443 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.443 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.443 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:11.443 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.635013+0000", 2024-04-26T03:39:11.443 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.3201000000000004e-05 2024-04-26T03:39:11.444 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.444 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:11.444 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.444 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.444 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.444 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:140 3.2s0 3:450a1cc5:::smithi09235267-47:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:11.444 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.619840+0000", 2024-04-26T03:39:11.444 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 19.782155682999999, 2024-04-26T03:39:11.444 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.015356137000000001, 2024-04-26T03:39:11.444 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:11.444 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:11.445 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:11.445 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:11.445 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:11.445 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:11.445 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 140 2024-04-26T03:39:11.445 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.445 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:11.445 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.445 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:11.445 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619840+0000", 2024-04-26T03:39:11.446 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.446 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.446 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.446 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:11.446 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619840+0000", 2024-04-26T03:39:11.446 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.446 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.446 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.446 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:11.446 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619840+0000", 2024-04-26T03:39:11.446 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.8100000000000002e-07 2024-04-26T03:39:11.447 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.447 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.447 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:11.447 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619843+0000", 2024-04-26T03:39:11.447 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.785e-06 2024-04-26T03:39:11.447 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.447 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.448 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:11.448 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619848+0000", 2024-04-26T03:39:11.448 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.2490000000000001e-06 2024-04-26T03:39:11.448 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.448 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.448 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:11.448 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619852+0000", 2024-04-26T03:39:11.448 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.1869999999999999e-06 2024-04-26T03:39:11.448 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.448 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.448 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:11.449 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.626570+0000", 2024-04-26T03:39:11.449 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0067177679999999998 2024-04-26T03:39:11.449 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.449 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.449 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:11.449 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.626669+0000", 2024-04-26T03:39:11.449 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.9177000000000003e-05 2024-04-26T03:39:11.449 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.449 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.449 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:11.449 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.627020+0000", 2024-04-26T03:39:11.450 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00035101199999999999 2024-04-26T03:39:11.450 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.450 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.450 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:11.450 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.627486+0000", 2024-04-26T03:39:11.450 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00046560299999999998 2024-04-26T03:39:11.450 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.450 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.450 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:11.450 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.635101+0000", 2024-04-26T03:39:11.451 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0076154680000000002 2024-04-26T03:39:11.451 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.451 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.451 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:11.451 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.635196+0000", 2024-04-26T03:39:11.451 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.4207000000000001e-05 2024-04-26T03:39:11.451 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.451 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:11.451 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.451 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.451 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.451 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:141 3.2s0 3:450a1cc5:::smithi09235267-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:11.451 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.619858+0000", 2024-04-26T03:39:11.452 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 19.782137350999999, 2024-04-26T03:39:11.452 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.017889577, 2024-04-26T03:39:11.452 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:11.452 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:11.452 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:11.452 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:11.452 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:11.452 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:11.452 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 141 2024-04-26T03:39:11.452 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.452 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:11.452 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.453 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:11.453 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619858+0000", 2024-04-26T03:39:11.453 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.453 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.453 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.453 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:11.453 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619858+0000", 2024-04-26T03:39:11.453 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.453 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.453 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.453 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:11.453 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619858+0000", 2024-04-26T03:39:11.453 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7000000000000001e-07 2024-04-26T03:39:11.454 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.454 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.454 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:11.454 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619860+0000", 2024-04-26T03:39:11.454 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1090000000000001e-06 2024-04-26T03:39:11.454 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.454 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.454 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:11.454 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619864+0000", 2024-04-26T03:39:11.454 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.1330000000000001e-06 2024-04-26T03:39:11.454 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.454 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.455 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:11.455 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619867+0000", 2024-04-26T03:39:11.455 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.9100000000000001e-06 2024-04-26T03:39:11.455 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.455 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.455 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:11.455 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.627233+0000", 2024-04-26T03:39:11.455 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0073655659999999996 2024-04-26T03:39:11.455 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.455 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.455 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:11.456 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.627322+0000", 2024-04-26T03:39:11.456 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.9456999999999994e-05 2024-04-26T03:39:11.456 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.456 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.456 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:11.456 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.635530+0000", 2024-04-26T03:39:11.456 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0082076349999999996 2024-04-26T03:39:11.456 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.456 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.456 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:11.456 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.635609+0000", 2024-04-26T03:39:11.457 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8724e-05 2024-04-26T03:39:11.457 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.457 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.457 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:11.457 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.637747+0000", 2024-04-26T03:39:11.457 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0021387730000000001 2024-04-26T03:39:11.457 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.457 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:11.457 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.457 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.457 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.458 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:222 3.2s0 3:42fcabaa:::smithi09235267-1:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:11.458 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.838869+0000", 2024-04-26T03:39:11.458 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 19.563126249, 2024-04-26T03:39:11.458 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.011165972999999999, 2024-04-26T03:39:11.458 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:11.458 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:11.458 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:11.458 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:11.458 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:11.458 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:11.458 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 222 2024-04-26T03:39:11.459 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.459 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:11.459 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.459 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:11.459 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.838869+0000", 2024-04-26T03:39:11.459 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.459 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.459 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.459 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:11.459 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.838869+0000", 2024-04-26T03:39:11.460 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.460 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.460 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.460 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:11.460 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.838870+0000", 2024-04-26T03:39:11.460 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.75e-07 2024-04-26T03:39:11.460 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.460 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.460 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:11.460 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.838873+0000", 2024-04-26T03:39:11.460 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9659999999999998e-06 2024-04-26T03:39:11.461 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.461 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.461 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:11.461 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.838883+0000", 2024-04-26T03:39:11.461 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.4919999999999999e-06 2024-04-26T03:39:11.461 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.461 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.461 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:11.461 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.838889+0000", 2024-04-26T03:39:11.461 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.5679999999999999e-06 2024-04-26T03:39:11.461 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.461 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.462 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:11.462 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.844188+0000", 2024-04-26T03:39:11.462 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0052990700000000003 2024-04-26T03:39:11.462 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.462 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.462 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:11.462 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.844238+0000", 2024-04-26T03:39:11.462 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.0655999999999997e-05 2024-04-26T03:39:11.462 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.462 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.462 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:11.463 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.844493+0000", 2024-04-26T03:39:11.463 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00025461700000000001 2024-04-26T03:39:11.463 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.463 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.463 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:11.463 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.847196+0000", 2024-04-26T03:39:11.463 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.002703067 2024-04-26T03:39:11.463 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.463 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.463 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:11.463 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.849977+0000", 2024-04-26T03:39:11.463 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0027805820000000002 2024-04-26T03:39:11.463 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.464 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.464 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:11.464 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.850035+0000", 2024-04-26T03:39:11.464 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.838e-05 2024-04-26T03:39:11.464 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.464 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:11.464 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.464 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.464 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.464 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:223 3.2s0 3:42fcabaa:::smithi09235267-1:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:11.464 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.838898+0000", 2024-04-26T03:39:11.465 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 19.563096826999999, 2024-04-26T03:39:11.465 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.012225915, 2024-04-26T03:39:11.465 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:11.465 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:11.465 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:11.465 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:11.465 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:11.465 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:11.465 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 223 2024-04-26T03:39:11.465 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.465 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:11.465 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.465 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:11.466 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.838898+0000", 2024-04-26T03:39:11.466 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.466 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.466 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.466 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:11.466 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.838898+0000", 2024-04-26T03:39:11.466 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.466 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.466 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.466 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:11.466 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.838899+0000", 2024-04-26T03:39:11.467 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.7e-07 2024-04-26T03:39:11.467 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.467 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.467 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:11.467 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.838901+0000", 2024-04-26T03:39:11.467 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5799999999999999e-06 2024-04-26T03:39:11.467 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.467 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.467 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:11.467 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.838910+0000", 2024-04-26T03:39:11.467 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.1189999999999997e-06 2024-04-26T03:39:11.467 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.468 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.468 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:11.468 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.838916+0000", 2024-04-26T03:39:11.468 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.8630000000000002e-06 2024-04-26T03:39:11.468 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.468 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.468 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:11.468 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.845205+0000", 2024-04-26T03:39:11.468 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0062883449999999999 2024-04-26T03:39:11.468 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.468 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.468 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:11.468 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.845254+0000", 2024-04-26T03:39:11.469 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.9101000000000003e-05 2024-04-26T03:39:11.469 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.469 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.469 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:11.469 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.850264+0000", 2024-04-26T03:39:11.469 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0050106079999999997 2024-04-26T03:39:11.469 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.469 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.469 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:11.469 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.850341+0000", 2024-04-26T03:39:11.469 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.6654000000000002e-05 2024-04-26T03:39:11.469 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.469 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.470 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:11.470 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.851124+0000", 2024-04-26T03:39:11.470 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000783175 2024-04-26T03:39:11.470 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.470 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:11.470 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.470 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.470 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.470 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.es1 43/32 ECSubWrite(tid=48, reqid=client.4556.0:301, at_version=43'16, trim_to=0'0, roll_forward_to=35'14))", 2024-04-26T03:39:11.470 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:59.531315+0000", 2024-04-26T03:39:11.470 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 11.870679876000001, 2024-04-26T03:39:11.470 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.014140454, 2024-04-26T03:39:11.471 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:11.471 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:11.471 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:11.471 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:11.471 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.471 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:11.471 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.531315+0000", 2024-04-26T03:39:11.471 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.471 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.471 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.472 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:11.472 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.531315+0000", 2024-04-26T03:39:11.472 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.472 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.472 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.472 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:11.472 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.531316+0000", 2024-04-26T03:39:11.472 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1799999999999999e-07 2024-04-26T03:39:11.472 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.472 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.472 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:11.473 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.531691+0000", 2024-04-26T03:39:11.473 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00037520700000000001 2024-04-26T03:39:11.473 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.473 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.473 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:11.473 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.531705+0000", 2024-04-26T03:39:11.473 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.395e-05 2024-04-26T03:39:11.473 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.473 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.473 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:11.473 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.531713+0000", 2024-04-26T03:39:11.473 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.0150000000000003e-06 2024-04-26T03:39:11.473 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.474 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.474 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:11.474 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.531798+0000", 2024-04-26T03:39:11.474 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5714e-05 2024-04-26T03:39:11.474 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.474 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.474 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:11.474 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.531811+0000", 2024-04-26T03:39:11.474 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2782000000000001e-05 2024-04-26T03:39:11.474 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.475 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.475 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:11.475 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.545405+0000", 2024-04-26T03:39:11.475 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.013593888 2024-04-26T03:39:11.475 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.475 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.475 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:11.475 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.545456+0000", 2024-04-26T03:39:11.475 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.0680000000000003e-05 2024-04-26T03:39:11.475 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.475 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:11.475 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.476 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.476 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.476 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.fs3 45/32 ECSubWrite(tid=49, reqid=client.4556.0:325, at_version=45'17, trim_to=0'0, roll_forward_to=42'15))", 2024-04-26T03:39:11.476 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:01.554312+0000", 2024-04-26T03:39:11.476 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 9.8476827809999996, 2024-04-26T03:39:11.476 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.012529861, 2024-04-26T03:39:11.476 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:11.476 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:11.476 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:11.476 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:11.476 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.476 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:11.476 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.554312+0000", 2024-04-26T03:39:11.477 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.477 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.477 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.477 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:11.477 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.554312+0000", 2024-04-26T03:39:11.477 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.477 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.477 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.477 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:11.477 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.554313+0000", 2024-04-26T03:39:11.477 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.9999999999999996e-07 2024-04-26T03:39:11.477 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.478 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.478 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:11.478 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.554865+0000", 2024-04-26T03:39:11.478 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00055152399999999996 2024-04-26T03:39:11.478 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.478 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.478 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:11.478 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.554886+0000", 2024-04-26T03:39:11.478 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1418999999999999e-05 2024-04-26T03:39:11.478 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.478 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.478 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:11.478 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.554897+0000", 2024-04-26T03:39:11.479 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0994e-05 2024-04-26T03:39:11.479 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.479 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.479 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:11.479 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.554971+0000", 2024-04-26T03:39:11.479 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3516000000000006e-05 2024-04-26T03:39:11.479 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.479 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.479 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:11.479 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.554983+0000", 2024-04-26T03:39:11.479 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2133999999999999e-05 2024-04-26T03:39:11.479 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.480 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.480 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:11.480 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.566763+0000", 2024-04-26T03:39:11.480 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.011780067 2024-04-26T03:39:11.480 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.480 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.480 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:11.480 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.566842+0000", 2024-04-26T03:39:11.480 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9307000000000005e-05 2024-04-26T03:39:11.480 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.480 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:11.480 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.481 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.481 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.481 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:333 3.2s0 3:450a1cc5:::smithi09235267-47:head [copy-from ver 6] snapc 7=[7,6,5] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-26T03:39:11.481 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:01.558145+0000", 2024-04-26T03:39:11.481 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 9.8438504420000008, 2024-04-26T03:39:11.481 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.017282072999999998, 2024-04-26T03:39:11.481 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:11.481 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:11.481 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:11.481 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:11.481 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:11.481 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:11.482 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 333 2024-04-26T03:39:11.482 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.482 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:11.482 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.482 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:11.482 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558145+0000", 2024-04-26T03:39:11.482 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.482 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.482 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.482 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:11.482 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558145+0000", 2024-04-26T03:39:11.482 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.482 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.482 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.483 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:11.483 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558147+0000", 2024-04-26T03:39:11.483 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4360000000000001e-06 2024-04-26T03:39:11.483 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.483 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.483 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:11.483 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558166+0000", 2024-04-26T03:39:11.483 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8411999999999999e-05 2024-04-26T03:39:11.483 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.483 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.483 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:11.483 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558181+0000", 2024-04-26T03:39:11.483 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4956e-05 2024-04-26T03:39:11.484 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.484 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.484 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:11.484 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558190+0000", 2024-04-26T03:39:11.484 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.7599999999999997e-06 2024-04-26T03:39:11.484 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.484 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.484 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:11.484 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558309+0000", 2024-04-26T03:39:11.484 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000118893 2024-04-26T03:39:11.484 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.484 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.484 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:11.485 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558491+0000", 2024-04-26T03:39:11.485 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00018157000000000001 2024-04-26T03:39:11.485 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.485 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.485 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:11.485 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.566556+0000", 2024-04-26T03:39:11.485 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0080647549999999998 2024-04-26T03:39:11.485 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.485 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.485 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:11.485 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.572889+0000", 2024-04-26T03:39:11.485 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0063337100000000002 2024-04-26T03:39:11.486 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.486 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.486 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:11.486 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.575342+0000", 2024-04-26T03:39:11.486 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0024523269999999998 2024-04-26T03:39:11.486 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.486 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.486 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:11.486 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.575427+0000", 2024-04-26T03:39:11.486 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5253999999999994e-05 2024-04-26T03:39:11.486 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.487 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:11.487 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.487 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.487 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.487 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:334 3.2s0 3:450a1cc5:::smithi09235267-47:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e45)", 2024-04-26T03:39:11.487 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:01.558215+0000", 2024-04-26T03:39:11.487 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 9.8437802909999998, 2024-04-26T03:39:11.487 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.013358238999999999, 2024-04-26T03:39:11.487 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:11.487 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:11.487 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:11.488 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:11.488 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:11.488 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:11.488 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 334 2024-04-26T03:39:11.488 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.488 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:11.488 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.488 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:11.488 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558215+0000", 2024-04-26T03:39:11.488 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.488 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.488 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.488 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:11.488 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558215+0000", 2024-04-26T03:39:11.489 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.489 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.489 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.489 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:11.489 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558216+0000", 2024-04-26T03:39:11.489 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.0800000000000004e-07 2024-04-26T03:39:11.489 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.489 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.489 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:11.489 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558220+0000", 2024-04-26T03:39:11.489 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8410000000000002e-06 2024-04-26T03:39:11.489 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.490 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.490 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:11.490 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558233+0000", 2024-04-26T03:39:11.490 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3556e-05 2024-04-26T03:39:11.490 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.490 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.490 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:11.490 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558241+0000", 2024-04-26T03:39:11.490 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.4080000000000001e-06 2024-04-26T03:39:11.490 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.490 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.491 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:11.491 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558731+0000", 2024-04-26T03:39:11.491 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00049039899999999998 2024-04-26T03:39:11.491 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.491 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.491 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for blocked object", 2024-04-26T03:39:11.491 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558777+0000", 2024-04-26T03:39:11.491 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6301000000000003e-05 2024-04-26T03:39:11.491 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.491 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.491 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:11.491 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.571433+0000", 2024-04-26T03:39:11.492 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.012656137 2024-04-26T03:39:11.492 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.492 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.492 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:11.492 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.571501+0000", 2024-04-26T03:39:11.492 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7319999999999999e-05 2024-04-26T03:39:11.492 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.492 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.492 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:11.492 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.571573+0000", 2024-04-26T03:39:11.492 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.2469e-05 2024-04-26T03:39:11.492 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.493 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:11.493 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.493 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.493 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.493 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:433 3.2s0 3:42fcabaa:::smithi09235267-1:head [append 0~671744] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-04-26T03:39:11.493 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:04.692691+0000", 2024-04-26T03:39:11.493 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.7093037999999998, 2024-04-26T03:39:11.493 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.026947377000000002, 2024-04-26T03:39:11.493 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:11.493 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:11.493 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:11.493 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:11.494 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:11.494 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:11.494 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 433 2024-04-26T03:39:11.494 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.494 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:11.494 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.494 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:11.494 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.692691+0000", 2024-04-26T03:39:11.494 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.494 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.494 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.494 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:11.495 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.692691+0000", 2024-04-26T03:39:11.495 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.495 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.495 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.495 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:11.495 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.692694+0000", 2024-04-26T03:39:11.495 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.216e-06 2024-04-26T03:39:11.495 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.495 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.495 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:11.495 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.693610+0000", 2024-04-26T03:39:11.495 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00091655899999999995 2024-04-26T03:39:11.496 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.496 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.496 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:11.496 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.693622+0000", 2024-04-26T03:39:11.496 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1976e-05 2024-04-26T03:39:11.496 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.496 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.496 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:11.496 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.693629+0000", 2024-04-26T03:39:11.496 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5749999999999997e-06 2024-04-26T03:39:11.496 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.497 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.497 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:11.497 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.693698+0000", 2024-04-26T03:39:11.497 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.9529000000000005e-05 2024-04-26T03:39:11.497 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.497 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.497 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op must wait for map", 2024-04-26T03:39:11.497 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.693708+0000", 2024-04-26T03:39:11.497 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.6649999999999998e-06 2024-04-26T03:39:11.497 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.497 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.497 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:11.498 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.698905+0000", 2024-04-26T03:39:11.498 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.005197279 2024-04-26T03:39:11.498 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.498 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.498 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:11.498 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.698993+0000", 2024-04-26T03:39:11.498 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.8029999999999996e-05 2024-04-26T03:39:11.498 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.498 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.498 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:11.498 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.699957+0000", 2024-04-26T03:39:11.499 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00096368199999999999 2024-04-26T03:39:11.499 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.499 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.499 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:11.499 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.718295+0000", 2024-04-26T03:39:11.499 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.018338255000000001 2024-04-26T03:39:11.499 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.499 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.499 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:11.499 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.719556+0000", 2024-04-26T03:39:11.499 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001261086 2024-04-26T03:39:11.499 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.500 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.500 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:11.500 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.719639+0000", 2024-04-26T03:39:11.500 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2525e-05 2024-04-26T03:39:11.500 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.500 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:11.500 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.500 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.500 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.500 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:434 3.2s0 3:42fcabaa:::smithi09235267-1:head [append 0~770048] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-04-26T03:39:11.500 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:04.699291+0000", 2024-04-26T03:39:11.501 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.7027041030000003, 2024-04-26T03:39:11.501 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.029702725999999999, 2024-04-26T03:39:11.501 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:11.501 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:11.501 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:11.501 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:11.501 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:11.501 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:11.501 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 434 2024-04-26T03:39:11.501 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.501 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:11.501 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.502 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:11.502 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.699291+0000", 2024-04-26T03:39:11.502 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.502 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.502 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.502 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:11.502 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.699291+0000", 2024-04-26T03:39:11.502 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.502 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.502 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.502 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:11.502 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.699292+0000", 2024-04-26T03:39:11.502 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.192e-06 2024-04-26T03:39:11.503 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.503 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.503 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:11.503 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700019+0000", 2024-04-26T03:39:11.503 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00072667100000000002 2024-04-26T03:39:11.503 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.503 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.503 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:11.503 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700032+0000", 2024-04-26T03:39:11.503 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3064999999999999e-05 2024-04-26T03:39:11.503 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.503 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.504 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:11.504 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700040+0000", 2024-04-26T03:39:11.504 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5299999999999999e-06 2024-04-26T03:39:11.504 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.504 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.504 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:11.504 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.711078+0000", 2024-04-26T03:39:11.504 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.011038536 2024-04-26T03:39:11.504 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.504 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.504 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:11.504 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.711181+0000", 2024-04-26T03:39:11.505 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000102833 2024-04-26T03:39:11.505 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.505 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.505 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:11.505 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.712518+0000", 2024-04-26T03:39:11.505 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0013371769999999999 2024-04-26T03:39:11.505 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.505 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.505 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:11.505 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.719306+0000", 2024-04-26T03:39:11.505 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0067879009999999998 2024-04-26T03:39:11.505 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.506 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.506 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:11.506 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728932+0000", 2024-04-26T03:39:11.506 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0096258230000000004 2024-04-26T03:39:11.506 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.506 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.506 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:11.506 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728994+0000", 2024-04-26T03:39:11.506 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.1997999999999998e-05 2024-04-26T03:39:11.506 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.506 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:11.506 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.507 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.507 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.507 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:435 3.2s0 3:42fcabaa:::smithi09235267-1:head [setxattr _header (55) in=7b] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-04-26T03:39:11.507 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:04.700054+0000", 2024-04-26T03:39:11.507 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.7019408460000003, 2024-04-26T03:39:11.507 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.029111279, 2024-04-26T03:39:11.507 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:11.507 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:11.507 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:11.507 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:11.507 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:11.507 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:11.508 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 435 2024-04-26T03:39:11.508 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.508 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:11.508 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.508 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:11.508 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700054+0000", 2024-04-26T03:39:11.508 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.508 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.508 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.508 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:11.508 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700054+0000", 2024-04-26T03:39:11.509 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.509 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.509 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.509 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:11.509 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700055+0000", 2024-04-26T03:39:11.509 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0470000000000001e-06 2024-04-26T03:39:11.509 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.509 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.509 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:11.509 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700059+0000", 2024-04-26T03:39:11.509 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8909999999999997e-06 2024-04-26T03:39:11.509 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.509 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.510 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:11.510 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700067+0000", 2024-04-26T03:39:11.510 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.447e-06 2024-04-26T03:39:11.510 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.510 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.510 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:11.510 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700072+0000", 2024-04-26T03:39:11.510 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7529999999999998e-06 2024-04-26T03:39:11.510 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.510 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.510 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:11.511 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.718645+0000", 2024-04-26T03:39:11.511 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.018572564 2024-04-26T03:39:11.511 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.511 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.511 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:11.511 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.718711+0000", 2024-04-26T03:39:11.511 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5743999999999999e-05 2024-04-26T03:39:11.511 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.511 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.511 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:11.512 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.718985+0000", 2024-04-26T03:39:11.512 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00027395400000000003 2024-04-26T03:39:11.512 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.512 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.512 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:11.512 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.719895+0000", 2024-04-26T03:39:11.512 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00091011499999999997 2024-04-26T03:39:11.512 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.512 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.512 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:11.512 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.729090+0000", 2024-04-26T03:39:11.512 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0091950920000000002 2024-04-26T03:39:11.513 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.513 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.513 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:11.513 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.729166+0000", 2024-04-26T03:39:11.513 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5672000000000003e-05 2024-04-26T03:39:11.513 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.514 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:11.514 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.514 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.514 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.514 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:436 3.2s0 3:42fcabaa:::smithi09235267-1:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e49)", 2024-04-26T03:39:11.514 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:04.700082+0000", 2024-04-26T03:39:11.514 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.701913201, 2024-04-26T03:39:11.514 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.032317973, 2024-04-26T03:39:11.515 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:11.515 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:11.515 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:11.515 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:11.515 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:11.515 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:11.515 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 436 2024-04-26T03:39:11.515 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.515 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:11.515 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.515 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:11.515 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700082+0000", 2024-04-26T03:39:11.516 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.516 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.516 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.516 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:11.516 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700082+0000", 2024-04-26T03:39:11.516 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.516 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.516 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.516 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:11.516 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700082+0000", 2024-04-26T03:39:11.516 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.3000000000000001e-07 2024-04-26T03:39:11.516 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.517 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.517 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:11.517 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700085+0000", 2024-04-26T03:39:11.517 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4739999999999999e-06 2024-04-26T03:39:11.517 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.517 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.517 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:11.517 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700092+0000", 2024-04-26T03:39:11.517 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3000000000000004e-06 2024-04-26T03:39:11.517 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.517 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.517 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:11.518 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700097+0000", 2024-04-26T03:39:11.518 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.2290000000000002e-06 2024-04-26T03:39:11.518 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.518 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.518 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:11.518 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.719230+0000", 2024-04-26T03:39:11.518 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.019132564000000001 2024-04-26T03:39:11.518 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.518 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.518 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:11.519 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.719296+0000", 2024-04-26T03:39:11.519 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.6370000000000003e-05 2024-04-26T03:39:11.519 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.519 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.519 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:11.519 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.729620+0000", 2024-04-26T03:39:11.519 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.010323183999999999 2024-04-26T03:39:11.519 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.519 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.519 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:11.519 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.729695+0000", 2024-04-26T03:39:11.519 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5500000000000006e-05 2024-04-26T03:39:11.520 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.520 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.520 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:11.520 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.732400+0000", 2024-04-26T03:39:11.520 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0027049219999999998 2024-04-26T03:39:11.520 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.520 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:11.520 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.520 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.520 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.520 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:478 3.2s0 3:450a1cc5:::smithi09235267-47:head [append 0~802816] snapc 8=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:11.521 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:05.910200+0000", 2024-04-26T03:39:11.521 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 5.491795518, 2024-04-26T03:39:11.521 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.013459199999999999, 2024-04-26T03:39:11.521 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:11.521 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:11.521 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:11.521 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:11.521 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:11.521 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:11.521 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 478 2024-04-26T03:39:11.521 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.521 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:11.522 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.522 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:11.522 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.910200+0000", 2024-04-26T03:39:11.522 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.522 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.522 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.522 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:11.522 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.910200+0000", 2024-04-26T03:39:11.522 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.522 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.523 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.523 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:11.523 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.910202+0000", 2024-04-26T03:39:11.523 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.683e-06 2024-04-26T03:39:11.523 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.523 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.523 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:11.523 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.910944+0000", 2024-04-26T03:39:11.523 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00074142299999999995 2024-04-26T03:39:11.523 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.523 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.524 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:11.524 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.910959+0000", 2024-04-26T03:39:11.524 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4909000000000001e-05 2024-04-26T03:39:11.524 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.524 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.524 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:11.524 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.910966+0000", 2024-04-26T03:39:11.524 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.827e-06 2024-04-26T03:39:11.524 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.524 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.524 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:11.524 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.911019+0000", 2024-04-26T03:39:11.525 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.3817999999999999e-05 2024-04-26T03:39:11.525 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.525 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.525 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:11.525 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.911094+0000", 2024-04-26T03:39:11.525 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5059000000000001e-05 2024-04-26T03:39:11.525 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.525 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.525 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:11.526 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.911822+0000", 2024-04-26T03:39:11.526 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00072724499999999995 2024-04-26T03:39:11.526 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.526 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.526 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:11.526 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.923328+0000", 2024-04-26T03:39:11.526 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.01150674 2024-04-26T03:39:11.526 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.526 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.526 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:11.526 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.923574+0000", 2024-04-26T03:39:11.527 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00024563200000000002 2024-04-26T03:39:11.527 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.527 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.527 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:11.527 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.923659+0000", 2024-04-26T03:39:11.527 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.4864000000000006e-05 2024-04-26T03:39:11.527 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.527 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:11.527 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.527 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.527 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.527 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:481 3.2s0 3:450a1cc5:::smithi09235267-47:head [setxattr _header (56) in=7b] snapc 8=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:11.527 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:05.914415+0000", 2024-04-26T03:39:11.528 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 5.4875805289999997, 2024-04-26T03:39:11.528 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.011349236, 2024-04-26T03:39:11.528 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:11.528 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:11.528 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:11.528 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:11.528 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:11.528 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:11.528 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 481 2024-04-26T03:39:11.528 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.528 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:11.528 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.529 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:11.529 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.914415+0000", 2024-04-26T03:39:11.529 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.529 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.529 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.529 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:11.529 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.914415+0000", 2024-04-26T03:39:11.529 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.529 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.529 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.529 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:11.529 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.914415+0000", 2024-04-26T03:39:11.530 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.8800000000000003e-07 2024-04-26T03:39:11.530 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.530 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.530 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:11.530 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.914422+0000", 2024-04-26T03:39:11.530 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.9199999999999998e-06 2024-04-26T03:39:11.530 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.530 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.530 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:11.530 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.914431+0000", 2024-04-26T03:39:11.530 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.8559999999999996e-06 2024-04-26T03:39:11.531 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.531 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.531 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:11.531 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.914438+0000", 2024-04-26T03:39:11.531 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.9199999999999998e-06 2024-04-26T03:39:11.531 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.531 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.531 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:11.531 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.924215+0000", 2024-04-26T03:39:11.531 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0097767129999999994 2024-04-26T03:39:11.531 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.531 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.532 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:11.532 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.924272+0000", 2024-04-26T03:39:11.532 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7003000000000003e-05 2024-04-26T03:39:11.532 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.532 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.532 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:11.532 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.924552+0000", 2024-04-26T03:39:11.532 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00028005100000000002 2024-04-26T03:39:11.532 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.532 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.532 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:11.532 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.925009+0000", 2024-04-26T03:39:11.532 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00045688899999999998 2024-04-26T03:39:11.533 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.533 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.533 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:11.533 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.925717+0000", 2024-04-26T03:39:11.533 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000708339 2024-04-26T03:39:11.533 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.533 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.533 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:11.533 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.925764+0000", 2024-04-26T03:39:11.533 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.7057e-05 2024-04-26T03:39:11.533 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.533 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:11.533 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.534 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.534 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.534 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:482 3.2s0 3:450a1cc5:::smithi09235267-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:11.534 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:05.914446+0000", 2024-04-26T03:39:11.534 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 5.487548951, 2024-04-26T03:39:11.534 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.01384406, 2024-04-26T03:39:11.534 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:11.534 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:11.534 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:11.534 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:11.534 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:11.535 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:11.535 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 482 2024-04-26T03:39:11.535 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.535 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:11.535 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.535 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:11.535 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.914446+0000", 2024-04-26T03:39:11.535 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.535 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.535 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.535 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:11.535 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.914446+0000", 2024-04-26T03:39:11.535 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:11.536 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.536 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.536 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:11.536 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.914447+0000", 2024-04-26T03:39:11.536 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.1600000000000001e-07 2024-04-26T03:39:11.536 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.536 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.536 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:11.536 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.914449+0000", 2024-04-26T03:39:11.536 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4229999999999999e-06 2024-04-26T03:39:11.536 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.536 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.536 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:11.537 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.914458+0000", 2024-04-26T03:39:11.537 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.9299999999999992e-06 2024-04-26T03:39:11.537 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.537 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.537 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:11.537 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.914465+0000", 2024-04-26T03:39:11.537 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.6599999999999998e-06 2024-04-26T03:39:11.537 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.537 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.537 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:11.537 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.924755+0000", 2024-04-26T03:39:11.537 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.010289777 2024-04-26T03:39:11.537 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.538 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.538 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:11.538 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.924801+0000", 2024-04-26T03:39:11.538 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6109000000000002e-05 2024-04-26T03:39:11.538 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.538 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.538 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:11.538 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.926051+0000", 2024-04-26T03:39:11.538 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001250393 2024-04-26T03:39:11.538 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.538 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.538 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:11.539 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.926140+0000", 2024-04-26T03:39:11.539 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.9227999999999993e-05 2024-04-26T03:39:11.539 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:11.539 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:11.539 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:11.539 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.928290+0000", 2024-04-26T03:39:11.539 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0021500239999999999 2024-04-26T03:39:11.539 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.539 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:11.539 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.539 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:11.539 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:11.539 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:39:11.541 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-26T03:39:11.542 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-04-26T03:39:11.540+0000 7fe9223c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31954) UID: 0 2024-04-26T03:39:11.641 INFO:tasks.ceph.osd.10.smithi066.stderr:2024-04-26T03:39:11.640+0000 7f130fcc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 31763) UID: 0 2024-04-26T03:39:11.660 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:11.661 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-26T03:39:11.661 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-26T03:39:11.661 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:11.670 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-26T03:39:11.707 INFO:teuthology.orchestra.run.smithi028.stderr:noscrub is set 2024-04-26T03:39:11.710 INFO:tasks.rados.rados.0.smithi092.stdout:231: done (2 left) 2024-04-26T03:39:11.710 INFO:tasks.rados.rados.0.smithi092.stdout:232: done (1 left) 2024-04-26T03:39:11.710 INFO:tasks.rados.rados.0.smithi092.stdout:233: done (0 left) 2024-04-26T03:39:11.711 INFO:tasks.rados.rados.0.smithi092.stdout:234: read oid 9 snap -1 2024-04-26T03:39:11.711 INFO:tasks.rados.rados.0.smithi092.stdout:234: expect deleted 2024-04-26T03:39:11.711 INFO:tasks.rados.rados.0.smithi092.stdout:235: copy_from oid 37 from oid 10 current snap is 12 2024-04-26T03:39:11.711 INFO:tasks.rados.rados.0.smithi092.stdout:236: delete oid 1 current snap is 12 2024-04-26T03:39:11.717 INFO:tasks.rados.rados.0.smithi092.stdout:234: done (2 left) 2024-04-26T03:39:11.717 INFO:tasks.rados.rados.0.smithi092.stdout:236: done (1 left) 2024-04-26T03:39:11.717 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 15 current snap is 12 2024-04-26T03:39:11.717 INFO:tasks.rados.rados.0.smithi092.stdout:237: seq_num 113 ranges {245760=737280,983040=557056,1540096=606208} 2024-04-26T03:39:11.723 INFO:tasks.rados.rados.0.smithi092.stdout:237: writing smithi09235267-15 from 245760 to 983040 tid 1 2024-04-26T03:39:11.726 INFO:tasks.rados.rados.0.smithi092.stdout:237: writing smithi09235267-15 from 983040 to 1540096 tid 2 2024-04-26T03:39:11.731 INFO:tasks.rados.rados.0.smithi092.stdout:237: writing smithi09235267-15 from 1540096 to 2146304 tid 3 2024-04-26T03:39:11.732 INFO:tasks.rados.rados.0.smithi092.stdout:238: read oid 2 snap -1 2024-04-26T03:39:11.732 INFO:tasks.rados.rados.0.smithi092.stdout:238: expect (ObjNum 110 snap 8 seq_num 110) 2024-04-26T03:39:11.732 INFO:tasks.rados.rados.0.smithi092.stdout:239: setattr oid 23 current snap is 12 2024-04-26T03:39:11.733 INFO:tasks.rados.rados.0.smithi092.stdout:240: rollback oid 5 current snap is 12 2024-04-26T03:39:11.733 INFO:tasks.rados.rados.0.smithi092.stdout:rollback oid 5 to 9 2024-04-26T03:39:11.733 INFO:tasks.rados.rados.0.smithi092.stdout:241: read oid 6 snap 8 2024-04-26T03:39:11.733 INFO:tasks.rados.rados.0.smithi092.stdout:241: expect (ObjNum 102 snap 8 seq_num 102) 2024-04-26T03:39:11.733 INFO:tasks.rados.rados.0.smithi092.stdout:242: snap_create 2024-04-26T03:39:11.742 INFO:tasks.rados.rados.0.smithi092.stdout:237: finishing write tid 1 to smithi09235267-15 2024-04-26T03:39:11.742 INFO:tasks.ceph.osd.9.smithi042.stderr:2024-04-26T03:39:11.741+0000 7fdac63b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 31960) UID: 0 2024-04-26T03:39:11.743 INFO:tasks.rados.rados.0.smithi092.stdout:240: finishing rollback tid 0 to smithi09235267-5 2024-04-26T03:39:11.746 INFO:tasks.rados.rados.0.smithi092.stdout:237: finishing write tid 2 to smithi09235267-15 2024-04-26T03:39:11.746 INFO:tasks.rados.rados.0.smithi092.stdout:237: finishing write tid 3 to smithi09235267-15 2024-04-26T03:39:11.749 INFO:tasks.rados.rados.0.smithi092.stdout:240: finishing rollback tid 1 to smithi09235267-5 2024-04-26T03:39:11.749 INFO:tasks.rados.rados.0.smithi092.stdout:240: finishing rollback tid 2 to smithi09235267-5 2024-04-26T03:39:11.749 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 5 v 14 (ObjNum 129 snap 0 seq_num 3583361784) dirty exists 2024-04-26T03:39:11.749 INFO:tasks.rados.rados.0.smithi092.stdout:237: finishing write tid 4 to smithi09235267-15 2024-04-26T03:39:11.749 INFO:tasks.rados.rados.0.smithi092.stdout:237: finishing write tid 5 to smithi09235267-15 2024-04-26T03:39:11.749 INFO:tasks.rados.rados.0.smithi092.stdout:237: oid 15 updating version 0 to 26 2024-04-26T03:39:11.749 INFO:tasks.rados.rados.0.smithi092.stdout:237: oid 15 updating version 26 to 27 2024-04-26T03:39:11.750 INFO:tasks.rados.rados.0.smithi092.stdout:237: oid 15 version 27 is already newer than 25 2024-04-26T03:39:11.750 INFO:tasks.rados.rados.0.smithi092.stdout:237: oid 15 updating version 27 to 28 2024-04-26T03:39:11.750 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 15 v 28 (ObjNum 113 snap 12 seq_num 113) dirty exists 2024-04-26T03:39:11.750 INFO:tasks.rados.rados.0.smithi092.stdout:237: left oid 15 (ObjNum 113 snap 12 seq_num 113) 2024-04-26T03:39:11.750 INFO:tasks.rados.rados.0.smithi092.stdout:238: expect (ObjNum 110 snap 8 seq_num 110) 2024-04-26T03:39:11.802 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-26T03:39:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-26T03:39:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-26T03:39:11.803 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:11.813 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-26T03:39:11.843 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-04-26T03:39:11.842+0000 7fe9223c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31954) UID: 0 2024-04-26T03:39:11.844 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 23 v 57 (ObjNum 103 snap 8 seq_num 103) dirty exists 2024-04-26T03:39:11.844 INFO:tasks.rados.rados.0.smithi092.stdout:235: finishing copy_from racing read to smithi09235267-37 2024-04-26T03:39:11.844 INFO:tasks.rados.rados.0.smithi092.stdout:235: finishing copy_from to smithi09235267-37 2024-04-26T03:39:11.844 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 37 v 50 (ObjNum 90 snap 7 seq_num 90) dirty exists 2024-04-26T03:39:11.862 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-26T03:39:11.870 INFO:tasks.rados.rados.0.smithi092.stdout:241: expect (ObjNum 102 snap 8 seq_num 102) 2024-04-26T03:39:11.944 INFO:tasks.ceph.osd.14.smithi066.stderr:2024-04-26T03:39:11.943+0000 7f5395f08640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 31765) UID: 0 2024-04-26T03:39:11.956 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:11.956 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-26T03:39:11.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-26T03:39:11.956 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-26T03:39:11.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.956 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:20 3.as0 3:5f4e6bd0:::smithi09235267-7:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:11.956 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.485002+0000", 2024-04-26T03:39:11.956 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.468470564, 2024-04-26T03:39:11.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017259429, 2024-04-26T03:39:11.956 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:11.956 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:11.956 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:11.956 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:11.957 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:11.957 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:11.957 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 20 2024-04-26T03:39:11.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.957 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:11.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:11.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485002+0000", 2024-04-26T03:39:11.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:11.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:11.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485002+0000", 2024-04-26T03:39:11.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:11.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:11.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485003+0000", 2024-04-26T03:39:11.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6400000000000002e-07 2024-04-26T03:39:11.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:11.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485007+0000", 2024-04-26T03:39:11.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4390000000000002e-06 2024-04-26T03:39:11.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:11.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485015+0000", 2024-04-26T03:39:11.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9389999999999997e-06 2024-04-26T03:39:11.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:11.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485243+0000", 2024-04-26T03:39:11.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000227345 2024-04-26T03:39:11.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:11.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.487060+0000", 2024-04-26T03:39:11.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001817117 2024-04-26T03:39:11.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:11.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.487148+0000", 2024-04-26T03:39:11.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8712e-05 2024-04-26T03:39:11.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:11.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.487611+0000", 2024-04-26T03:39:11.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000462223 2024-04-26T03:39:11.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:11.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.491670+0000", 2024-04-26T03:39:11.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004058842 2024-04-26T03:39:11.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:11.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.502181+0000", 2024-04-26T03:39:11.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010511360000000001 2024-04-26T03:39:11.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:11.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.502262+0000", 2024-04-26T03:39:11.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0888000000000002e-05 2024-04-26T03:39:11.962 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:11.962 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:11.962 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:11.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.962 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:21 3.as0 3:5f4e6bd0:::smithi09235267-7:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:11.962 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.485256+0000", 2024-04-26T03:39:11.962 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.468216927, 2024-04-26T03:39:11.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026302736, 2024-04-26T03:39:11.962 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:11.962 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:11.962 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:11.962 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:11.963 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:11.963 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:11.963 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 21 2024-04-26T03:39:11.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.963 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:11.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:11.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485256+0000", 2024-04-26T03:39:11.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:11.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:11.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485256+0000", 2024-04-26T03:39:11.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:11.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:11.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485257+0000", 2024-04-26T03:39:11.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4000000000000001e-07 2024-04-26T03:39:11.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:11.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485261+0000", 2024-04-26T03:39:11.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2910000000000001e-06 2024-04-26T03:39:11.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:11.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485269+0000", 2024-04-26T03:39:11.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1780000000000003e-06 2024-04-26T03:39:11.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:11.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485275+0000", 2024-04-26T03:39:11.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2419999999999997e-06 2024-04-26T03:39:11.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:11.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.487806+0000", 2024-04-26T03:39:11.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025310369999999999 2024-04-26T03:39:11.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:11.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.488019+0000", 2024-04-26T03:39:11.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021297800000000001 2024-04-26T03:39:11.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:11.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.502612+0000", 2024-04-26T03:39:11.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014592327 2024-04-26T03:39:11.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:11.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.502711+0000", 2024-04-26T03:39:11.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9432999999999995e-05 2024-04-26T03:39:11.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:11.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.511559+0000", 2024-04-26T03:39:11.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0088476100000000005 2024-04-26T03:39:11.967 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:11.967 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:11.967 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:11.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.967 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:75 3.as0 3:50cd80b4:::smithi09235267-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:11.967 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.524874+0000", 2024-04-26T03:39:11.967 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.428599084999998, 2024-04-26T03:39:11.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017219323000000002, 2024-04-26T03:39:11.967 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:11.967 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:11.967 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:11.967 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:11.968 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:11.968 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:11.968 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 75 2024-04-26T03:39:11.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.968 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:11.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:11.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.524874+0000", 2024-04-26T03:39:11.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:11.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:11.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.524874+0000", 2024-04-26T03:39:11.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:11.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:11.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.524874+0000", 2024-04-26T03:39:11.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9200000000000001e-07 2024-04-26T03:39:11.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:11.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.524877+0000", 2024-04-26T03:39:11.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6079999999999998e-06 2024-04-26T03:39:11.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:11.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.524883+0000", 2024-04-26T03:39:11.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0039999999999996e-06 2024-04-26T03:39:11.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:11.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.524888+0000", 2024-04-26T03:39:11.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3070000000000002e-06 2024-04-26T03:39:11.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:11.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.530308+0000", 2024-04-26T03:39:11.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054201850000000001 2024-04-26T03:39:11.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:11.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.530412+0000", 2024-04-26T03:39:11.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010306699999999999 2024-04-26T03:39:11.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:11.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.534668+0000", 2024-04-26T03:39:11.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042560949999999997 2024-04-26T03:39:11.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:11.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.534740+0000", 2024-04-26T03:39:11.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2082000000000005e-05 2024-04-26T03:39:11.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:11.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.542093+0000", 2024-04-26T03:39:11.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0073534830000000001 2024-04-26T03:39:11.972 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:11.972 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:11.972 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:11.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.972 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:105 3.as0 3:527f9dfd:::smithi09235267-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:11.972 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.562626+0000", 2024-04-26T03:39:11.972 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.390847246, 2024-04-26T03:39:11.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.023769061000000001, 2024-04-26T03:39:11.973 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:11.973 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:11.973 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:11.973 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:11.973 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:11.973 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:11.973 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 105 2024-04-26T03:39:11.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.973 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:11.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:11.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.562626+0000", 2024-04-26T03:39:11.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:11.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:11.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.562626+0000", 2024-04-26T03:39:11.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:11.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:11.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.562626+0000", 2024-04-26T03:39:11.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7800000000000002e-07 2024-04-26T03:39:11.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:11.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.562629+0000", 2024-04-26T03:39:11.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7779999999999999e-06 2024-04-26T03:39:11.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:11.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.562636+0000", 2024-04-26T03:39:11.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3520000000000001e-06 2024-04-26T03:39:11.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:11.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.562641+0000", 2024-04-26T03:39:11.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4789999999999998e-06 2024-04-26T03:39:11.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:11.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.568071+0000", 2024-04-26T03:39:11.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054305719999999998 2024-04-26T03:39:11.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:11.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.568158+0000", 2024-04-26T03:39:11.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6726999999999998e-05 2024-04-26T03:39:11.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:11.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.575494+0000", 2024-04-26T03:39:11.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0073362180000000003 2024-04-26T03:39:11.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:11.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.575571+0000", 2024-04-26T03:39:11.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7134999999999999e-05 2024-04-26T03:39:11.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:11.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.586395+0000", 2024-04-26T03:39:11.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010823322 2024-04-26T03:39:11.977 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:11.977 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:11.977 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:11.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.978 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:117 3.as0 3:58923ed4:::smithi09235267-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:11.978 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.578829+0000", 2024-04-26T03:39:11.978 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.374644128, 2024-04-26T03:39:11.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017648347000000002, 2024-04-26T03:39:11.978 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:11.978 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:11.978 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:11.978 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:11.978 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:11.978 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:11.978 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 117 2024-04-26T03:39:11.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.978 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:11.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:11.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.578829+0000", 2024-04-26T03:39:11.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:11.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:11.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.578829+0000", 2024-04-26T03:39:11.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:11.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:11.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.578829+0000", 2024-04-26T03:39:11.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4900000000000001e-07 2024-04-26T03:39:11.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:11.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.578831+0000", 2024-04-26T03:39:11.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0700000000000001e-06 2024-04-26T03:39:11.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:11.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.578838+0000", 2024-04-26T03:39:11.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2689999999999996e-06 2024-04-26T03:39:11.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:11.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.578843+0000", 2024-04-26T03:39:11.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5300000000000004e-06 2024-04-26T03:39:11.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:11.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.586870+0000", 2024-04-26T03:39:11.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0080264919999999997 2024-04-26T03:39:11.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:11.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.586955+0000", 2024-04-26T03:39:11.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5753000000000004e-05 2024-04-26T03:39:11.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:11.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.589337+0000", 2024-04-26T03:39:11.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023813010000000002 2024-04-26T03:39:11.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:11.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.589419+0000", 2024-04-26T03:39:11.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2078999999999998e-05 2024-04-26T03:39:11.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:11.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.596477+0000", 2024-04-26T03:39:11.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0070584039999999999 2024-04-26T03:39:11.983 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:11.983 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:11.983 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:11.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.983 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:150 3.as0 3:5930de0f:::smithi09235267-50:head [append 0~458752,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:11.983 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.634461+0000", 2024-04-26T03:39:11.983 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.319011638999999, 2024-04-26T03:39:11.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017493549000000001, 2024-04-26T03:39:11.983 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:11.983 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:11.984 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:11.984 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:11.984 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:11.984 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:11.984 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 150 2024-04-26T03:39:11.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.984 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:11.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:11.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.634461+0000", 2024-04-26T03:39:11.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:11.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:11.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.634461+0000", 2024-04-26T03:39:11.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:11.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:11.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.634463+0000", 2024-04-26T03:39:11.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.618e-06 2024-04-26T03:39:11.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:11.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.634611+0000", 2024-04-26T03:39:11.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000148214 2024-04-26T03:39:11.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:11.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.634628+0000", 2024-04-26T03:39:11.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7105999999999999e-05 2024-04-26T03:39:11.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:11.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.634636+0000", 2024-04-26T03:39:11.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7829999999999999e-06 2024-04-26T03:39:11.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:11.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.634700+0000", 2024-04-26T03:39:11.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4286000000000001e-05 2024-04-26T03:39:11.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:11.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.634839+0000", 2024-04-26T03:39:11.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013907200000000001 2024-04-26T03:39:11.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:11.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.635607+0000", 2024-04-26T03:39:11.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00076726300000000002 2024-04-26T03:39:11.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:11.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.647963+0000", 2024-04-26T03:39:11.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012356525 2024-04-26T03:39:11.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:11.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.651885+0000", 2024-04-26T03:39:11.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039220180000000002 2024-04-26T03:39:11.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:11.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.651955+0000", 2024-04-26T03:39:11.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9664000000000003e-05 2024-04-26T03:39:11.989 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:11.989 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:11.989 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:11.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.990 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:151 3.as0 3:5930de0f:::smithi09235267-50:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:11.990 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.637279+0000", 2024-04-26T03:39:11.990 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.316193797, 2024-04-26T03:39:11.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016253165999999999, 2024-04-26T03:39:11.990 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:11.990 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:11.990 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:11.990 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:11.990 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:11.990 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:11.990 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 151 2024-04-26T03:39:11.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.991 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:11.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:11.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637279+0000", 2024-04-26T03:39:11.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:11.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:11.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637279+0000", 2024-04-26T03:39:11.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:11.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:11.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637281+0000", 2024-04-26T03:39:11.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5319999999999999e-06 2024-04-26T03:39:11.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:11.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637373+0000", 2024-04-26T03:39:11.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2198000000000001e-05 2024-04-26T03:39:11.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:11.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637384+0000", 2024-04-26T03:39:11.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1198e-05 2024-04-26T03:39:11.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:11.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637393+0000", 2024-04-26T03:39:11.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0280000000000006e-06 2024-04-26T03:39:11.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:11.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.639806+0000", 2024-04-26T03:39:11.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024132929999999999 2024-04-26T03:39:11.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:11.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.639879+0000", 2024-04-26T03:39:11.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2470000000000002e-05 2024-04-26T03:39:11.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:11.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.640572+0000", 2024-04-26T03:39:11.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00069311299999999998 2024-04-26T03:39:11.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:11.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.647964+0000", 2024-04-26T03:39:11.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0073922019999999996 2024-04-26T03:39:11.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:11.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.653490+0000", 2024-04-26T03:39:11.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055261850000000003 2024-04-26T03:39:11.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:11.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.653532+0000", 2024-04-26T03:39:11.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1946999999999997e-05 2024-04-26T03:39:11.996 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:11.996 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:11.996 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:11.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.996 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:152 3.as0 3:5930de0f:::smithi09235267-50:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:11.996 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.637409+0000", 2024-04-26T03:39:11.996 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.316064353000002, 2024-04-26T03:39:11.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016318039999999999, 2024-04-26T03:39:11.996 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:11.996 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:11.997 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:11.997 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:11.997 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:11.997 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:11.997 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 152 2024-04-26T03:39:11.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.997 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:11.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:11.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637409+0000", 2024-04-26T03:39:11.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:11.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:11.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637409+0000", 2024-04-26T03:39:11.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:11.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:11.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637409+0000", 2024-04-26T03:39:11.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5499999999999998e-07 2024-04-26T03:39:11.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:11.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637412+0000", 2024-04-26T03:39:11.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2959999999999999e-06 2024-04-26T03:39:11.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:11.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:11.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:11.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637419+0000", 2024-04-26T03:39:11.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4670000000000001e-06 2024-04-26T03:39:11.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:12.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637424+0000", 2024-04-26T03:39:12.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8720000000000001e-06 2024-04-26T03:39:12.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:12.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.643035+0000", 2024-04-26T03:39:12.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056115899999999996 2024-04-26T03:39:12.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:12.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.643126+0000", 2024-04-26T03:39:12.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0525999999999994e-05 2024-04-26T03:39:12.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:12.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.643545+0000", 2024-04-26T03:39:12.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00041870200000000002 2024-04-26T03:39:12.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:12.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.647965+0000", 2024-04-26T03:39:12.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0044201730000000003 2024-04-26T03:39:12.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:12.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.653647+0000", 2024-04-26T03:39:12.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056822670000000004 2024-04-26T03:39:12.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:12.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.653727+0000", 2024-04-26T03:39:12.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9592000000000006e-05 2024-04-26T03:39:12.003 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:12.003 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:12.003 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:12.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.003 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:153 3.as0 3:5930de0f:::smithi09235267-50:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:12.003 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.637431+0000", 2024-04-26T03:39:12.003 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.316042077999999, 2024-04-26T03:39:12.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019190048000000001, 2024-04-26T03:39:12.003 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:12.003 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:12.004 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:12.004 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:12.004 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:12.004 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:12.004 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 153 2024-04-26T03:39:12.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.004 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:12.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:12.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637431+0000", 2024-04-26T03:39:12.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:12.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:12.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637431+0000", 2024-04-26T03:39:12.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:12.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:12.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637431+0000", 2024-04-26T03:39:12.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2024-04-26T03:39:12.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:12.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637434+0000", 2024-04-26T03:39:12.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7690000000000001e-06 2024-04-26T03:39:12.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:12.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637439+0000", 2024-04-26T03:39:12.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4190000000000001e-06 2024-04-26T03:39:12.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:12.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637444+0000", 2024-04-26T03:39:12.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6800000000000001e-06 2024-04-26T03:39:12.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:12.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.643745+0000", 2024-04-26T03:39:12.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0063006570000000003 2024-04-26T03:39:12.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:12.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.643818+0000", 2024-04-26T03:39:12.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3558000000000003e-05 2024-04-26T03:39:12.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:12.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.654106+0000", 2024-04-26T03:39:12.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010287436 2024-04-26T03:39:12.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:12.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.654189+0000", 2024-04-26T03:39:12.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2719999999999994e-05 2024-04-26T03:39:12.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:12.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.656621+0000", 2024-04-26T03:39:12.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024323690000000002 2024-04-26T03:39:12.008 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:12.009 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:12.009 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:12.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.009 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:159 3.as0 3:50cd80b4:::smithi09235267-25:head [read 0~409600,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:12.009 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.637451+0000", 2024-04-26T03:39:12.009 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.316021729999999, 2024-04-26T03:39:12.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015016775, 2024-04-26T03:39:12.009 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:12.009 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:12.009 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:12.009 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:12.010 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:12.010 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:12.010 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 159 2024-04-26T03:39:12.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.010 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:12.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:12.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637451+0000", 2024-04-26T03:39:12.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:12.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:12.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637451+0000", 2024-04-26T03:39:12.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:12.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:12.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637452+0000", 2024-04-26T03:39:12.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0499999999999999e-07 2024-04-26T03:39:12.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:12.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637454+0000", 2024-04-26T03:39:12.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.853e-06 2024-04-26T03:39:12.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:12.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637460+0000", 2024-04-26T03:39:12.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5960000000000003e-06 2024-04-26T03:39:12.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:12.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637464+0000", 2024-04-26T03:39:12.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.155e-06 2024-04-26T03:39:12.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:12.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.643851+0000", 2024-04-26T03:39:12.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0063864560000000004 2024-04-26T03:39:12.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:12.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.643919+0000", 2024-04-26T03:39:12.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7834000000000001e-05 2024-04-26T03:39:12.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:12.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.652468+0000", 2024-04-26T03:39:12.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0085494760000000003 2024-04-26T03:39:12.015 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:12.015 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:12.015 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:12.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.015 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:160 3.as0 3:50cd80b4:::smithi09235267-25:head [read 0~409600,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:12.016 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.637472+0000", 2024-04-26T03:39:12.016 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.316001223000001, 2024-04-26T03:39:12.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015503981, 2024-04-26T03:39:12.016 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:12.016 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:12.016 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:12.016 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:12.016 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:12.016 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:12.017 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 160 2024-04-26T03:39:12.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.017 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:12.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:12.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637472+0000", 2024-04-26T03:39:12.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:12.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:12.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637472+0000", 2024-04-26T03:39:12.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:12.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:12.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637472+0000", 2024-04-26T03:39:12.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2800000000000002e-07 2024-04-26T03:39:12.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:12.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637475+0000", 2024-04-26T03:39:12.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.65e-06 2024-04-26T03:39:12.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:12.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637481+0000", 2024-04-26T03:39:12.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9309999999999996e-06 2024-04-26T03:39:12.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:12.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637487+0000", 2024-04-26T03:39:12.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6830000000000002e-06 2024-04-26T03:39:12.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:12.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.644230+0000", 2024-04-26T03:39:12.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067428519999999997 2024-04-26T03:39:12.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:12.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.644300+0000", 2024-04-26T03:39:12.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9443999999999994e-05 2024-04-26T03:39:12.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:12.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.652976+0000", 2024-04-26T03:39:12.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0086759929999999999 2024-04-26T03:39:12.021 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:12.021 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:12.021 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:12.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.021 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:161 3.as0 3:50cd80b4:::smithi09235267-25:head [read 0~409600,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:12.021 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.637494+0000", 2024-04-26T03:39:12.021 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.315978829999999, 2024-04-26T03:39:12.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015912572999999999, 2024-04-26T03:39:12.021 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:12.021 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:12.021 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:12.022 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:12.022 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:12.022 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:12.022 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 161 2024-04-26T03:39:12.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.022 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:12.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:12.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637494+0000", 2024-04-26T03:39:12.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:12.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:12.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637494+0000", 2024-04-26T03:39:12.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:12.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:12.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637494+0000", 2024-04-26T03:39:12.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0400000000000002e-07 2024-04-26T03:39:12.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:12.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637496+0000", 2024-04-26T03:39:12.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8729999999999999e-06 2024-04-26T03:39:12.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:12.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637500+0000", 2024-04-26T03:39:12.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4769999999999999e-06 2024-04-26T03:39:12.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:12.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637502+0000", 2024-04-26T03:39:12.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5100000000000001e-06 2024-04-26T03:39:12.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:12.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.648250+0000", 2024-04-26T03:39:12.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010747283999999999 2024-04-26T03:39:12.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:12.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.648336+0000", 2024-04-26T03:39:12.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6910999999999995e-05 2024-04-26T03:39:12.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:12.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.653407+0000", 2024-04-26T03:39:12.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005070214 2024-04-26T03:39:12.026 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:12.026 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:12.026 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:12.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.026 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:310 3.as0 3:50cd80b4:::smithi09235267-25:head [copy-from ver 3] snapc 6=[6,5] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-26T03:39:12.026 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:59.536617+0000", 2024-04-26T03:39:12.026 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.416855485999999, 2024-04-26T03:39:12.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022665805000000001, 2024-04-26T03:39:12.027 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:12.027 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:12.027 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:12.027 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:12.027 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:12.027 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:12.027 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 310 2024-04-26T03:39:12.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.027 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:12.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:12.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536617+0000", 2024-04-26T03:39:12.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:12.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:12.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536617+0000", 2024-04-26T03:39:12.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:12.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:12.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536620+0000", 2024-04-26T03:39:12.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0540000000000002e-06 2024-04-26T03:39:12.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:12.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536636+0000", 2024-04-26T03:39:12.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6963e-05 2024-04-26T03:39:12.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:12.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536710+0000", 2024-04-26T03:39:12.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3711e-05 2024-04-26T03:39:12.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:12.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536719+0000", 2024-04-26T03:39:12.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9889999999999998e-06 2024-04-26T03:39:12.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:12.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536789+0000", 2024-04-26T03:39:12.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9884999999999999e-05 2024-04-26T03:39:12.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-04-26T03:39:12.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536797+0000", 2024-04-26T03:39:12.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5909999999999998e-06 2024-04-26T03:39:12.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:12.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.538587+0000", 2024-04-26T03:39:12.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017899249999999999 2024-04-26T03:39:12.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:12.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.538669+0000", 2024-04-26T03:39:12.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2045000000000005e-05 2024-04-26T03:39:12.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:12.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.549606+0000", 2024-04-26T03:39:12.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010937448000000001 2024-04-26T03:39:12.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:12.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.558484+0000", 2024-04-26T03:39:12.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00887808 2024-04-26T03:39:12.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:12.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.559195+0000", 2024-04-26T03:39:12.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00071067799999999996 2024-04-26T03:39:12.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:12.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.559283+0000", 2024-04-26T03:39:12.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8436000000000005e-05 2024-04-26T03:39:12.033 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:12.033 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:12.033 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:12.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.033 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:311 3.as0 3:50cd80b4:::smithi09235267-25:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e43)", 2024-04-26T03:39:12.033 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:59.536742+0000", 2024-04-26T03:39:12.034 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.416730594000001, 2024-04-26T03:39:12.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021335742000000001, 2024-04-26T03:39:12.034 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:12.034 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:12.034 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:12.034 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:12.034 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:12.034 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:12.034 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 311 2024-04-26T03:39:12.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.034 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:12.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:12.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536742+0000", 2024-04-26T03:39:12.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:12.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:12.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536742+0000", 2024-04-26T03:39:12.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:12.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:12.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536743+0000", 2024-04-26T03:39:12.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5000000000000003e-07 2024-04-26T03:39:12.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:12.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536746+0000", 2024-04-26T03:39:12.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8720000000000002e-06 2024-04-26T03:39:12.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:12.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536771+0000", 2024-04-26T03:39:12.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4949000000000001e-05 2024-04-26T03:39:12.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:12.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536776+0000", 2024-04-26T03:39:12.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1089999999999997e-06 2024-04-26T03:39:12.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:12.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536853+0000", 2024-04-26T03:39:12.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7348000000000006e-05 2024-04-26T03:39:12.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-04-26T03:39:12.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536859+0000", 2024-04-26T03:39:12.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.784e-06 2024-04-26T03:39:12.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:12.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.538806+0000", 2024-04-26T03:39:12.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019471029999999999 2024-04-26T03:39:12.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for blocked object", 2024-04-26T03:39:12.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.538849+0000", 2024-04-26T03:39:12.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2901000000000001e-05 2024-04-26T03:39:12.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:12.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.557853+0000", 2024-04-26T03:39:12.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019003955999999999 2024-04-26T03:39:12.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:12.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.557954+0000", 2024-04-26T03:39:12.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010153399999999999 2024-04-26T03:39:12.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:12.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.558078+0000", 2024-04-26T03:39:12.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012363599999999999 2024-04-26T03:39:12.040 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:12.040 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:12.040 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:12.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.041 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.2s2 49/44 ECSubWrite(tid=34, reqid=client.4556.0:434, at_version=49'23, trim_to=0'0, roll_forward_to=48'19))", 2024-04-26T03:39:12.041 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:04.712444+0000", 2024-04-26T03:39:12.041 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.2410286709999996, 2024-04-26T03:39:12.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016297222, 2024-04-26T03:39:12.041 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:12.041 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:12.041 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:12.041 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:12.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:12.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.712444+0000", 2024-04-26T03:39:12.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:12.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:12.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.712444+0000", 2024-04-26T03:39:12.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:12.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:12.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.712445+0000", 2024-04-26T03:39:12.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4399999999999999e-07 2024-04-26T03:39:12.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:12.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.712821+0000", 2024-04-26T03:39:12.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000376194 2024-04-26T03:39:12.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:12.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.712838+0000", 2024-04-26T03:39:12.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7076999999999999e-05 2024-04-26T03:39:12.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:12.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.712854+0000", 2024-04-26T03:39:12.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5852999999999999e-05 2024-04-26T03:39:12.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:12.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.712940+0000", 2024-04-26T03:39:12.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6061000000000002e-05 2024-04-26T03:39:12.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:12.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.712952+0000", 2024-04-26T03:39:12.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1810999999999999e-05 2024-04-26T03:39:12.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:12.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.728691+0000", 2024-04-26T03:39:12.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015739306000000002 2024-04-26T03:39:12.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:12.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.728741+0000", 2024-04-26T03:39:12.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0076e-05 2024-04-26T03:39:12.046 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:12.046 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:12.046 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:12.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.046 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:454 3.as0 3:5930de0f:::smithi09235267-50:head [append 0~737280] snapc 8=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:12.046 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:05.710804+0000", 2024-04-26T03:39:12.046 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.2426693269999998, 2024-04-26T03:39:12.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017146651999999998, 2024-04-26T03:39:12.046 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:12.046 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:12.047 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:12.047 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:12.047 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:12.047 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:12.047 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 454 2024-04-26T03:39:12.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.047 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:12.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:12.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.710804+0000", 2024-04-26T03:39:12.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:12.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:12.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.710804+0000", 2024-04-26T03:39:12.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:12.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:12.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.710805+0000", 2024-04-26T03:39:12.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8509999999999999e-06 2024-04-26T03:39:12.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:12.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.711242+0000", 2024-04-26T03:39:12.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000436091 2024-04-26T03:39:12.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:12.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.711259+0000", 2024-04-26T03:39:12.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7533e-05 2024-04-26T03:39:12.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:12.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.711269+0000", 2024-04-26T03:39:12.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6760000000000002e-06 2024-04-26T03:39:12.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:12.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.711382+0000", 2024-04-26T03:39:12.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011296 2024-04-26T03:39:12.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:12.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.711483+0000", 2024-04-26T03:39:12.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101048 2024-04-26T03:39:12.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:12.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.713492+0000", 2024-04-26T03:39:12.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020094900000000001 2024-04-26T03:39:12.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:12.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.727613+0000", 2024-04-26T03:39:12.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014120819 2024-04-26T03:39:12.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:12.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.727890+0000", 2024-04-26T03:39:12.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027681299999999999 2024-04-26T03:39:12.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:12.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.727950+0000", 2024-04-26T03:39:12.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0371000000000002e-05 2024-04-26T03:39:12.052 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:12.052 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:12.052 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:12.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.053 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:455 3.as0 3:5930de0f:::smithi09235267-50:head [append 0~573440] snapc 8=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:12.053 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:05.716238+0000", 2024-04-26T03:39:12.053 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.2372346939999996, 2024-04-26T03:39:12.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015776366999999999, 2024-04-26T03:39:12.053 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:12.053 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:12.053 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:12.053 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:12.053 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:12.053 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:12.053 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 455 2024-04-26T03:39:12.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.054 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:12.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:12.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716238+0000", 2024-04-26T03:39:12.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:12.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:12.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716238+0000", 2024-04-26T03:39:12.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:12.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:12.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716240+0000", 2024-04-26T03:39:12.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1019999999999999e-06 2024-04-26T03:39:12.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:12.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716566+0000", 2024-04-26T03:39:12.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00032576600000000001 2024-04-26T03:39:12.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:12.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716581+0000", 2024-04-26T03:39:12.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4812e-05 2024-04-26T03:39:12.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:12.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716590+0000", 2024-04-26T03:39:12.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6330000000000002e-06 2024-04-26T03:39:12.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:12.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.720824+0000", 2024-04-26T03:39:12.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042345960000000002 2024-04-26T03:39:12.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:12.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.720922+0000", 2024-04-26T03:39:12.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7833999999999999e-05 2024-04-26T03:39:12.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:12.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.721938+0000", 2024-04-26T03:39:12.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010160620000000001 2024-04-26T03:39:12.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:12.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.727994+0000", 2024-04-26T03:39:12.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060558449999999998 2024-04-26T03:39:12.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:12.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.731928+0000", 2024-04-26T03:39:12.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039341430000000002 2024-04-26T03:39:12.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:12.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.732015+0000", 2024-04-26T03:39:12.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6574000000000002e-05 2024-04-26T03:39:12.059 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:12.059 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:12.059 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:12.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.059 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:456 3.as0 3:5930de0f:::smithi09235267-50:head [append 0~196608] snapc 8=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:12.059 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:05.716614+0000", 2024-04-26T03:39:12.059 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.2368585029999997, 2024-04-26T03:39:12.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015695205, 2024-04-26T03:39:12.060 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:12.060 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:12.060 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:12.060 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:12.060 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:12.060 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:12.060 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 456 2024-04-26T03:39:12.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.060 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:12.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:12.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716614+0000", 2024-04-26T03:39:12.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:12.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:12.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716614+0000", 2024-04-26T03:39:12.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:12.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:12.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716615+0000", 2024-04-26T03:39:12.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2799999999999995e-07 2024-04-26T03:39:12.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:12.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716737+0000", 2024-04-26T03:39:12.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121588 2024-04-26T03:39:12.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:12.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716746+0000", 2024-04-26T03:39:12.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5480000000000007e-06 2024-04-26T03:39:12.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:12.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716754+0000", 2024-04-26T03:39:12.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6040000000000004e-06 2024-04-26T03:39:12.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:12.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.728074+0000", 2024-04-26T03:39:12.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011320227 2024-04-26T03:39:12.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:12.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.728133+0000", 2024-04-26T03:39:12.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9344e-05 2024-04-26T03:39:12.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:12.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.728554+0000", 2024-04-26T03:39:12.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000420419 2024-04-26T03:39:12.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:12.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.731780+0000", 2024-04-26T03:39:12.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0032263539999999999 2024-04-26T03:39:12.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:12.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.732242+0000", 2024-04-26T03:39:12.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00046198399999999997 2024-04-26T03:39:12.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:12.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.732310+0000", 2024-04-26T03:39:12.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7409000000000005e-05 2024-04-26T03:39:12.065 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:12.065 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:12.065 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:12.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.066 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:457 3.as0 3:5930de0f:::smithi09235267-50:head [setxattr _header (56) in=7b] snapc 8=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:12.066 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:05.716766+0000", 2024-04-26T03:39:12.066 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.2367071300000001, 2024-04-26T03:39:12.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017528595000000001, 2024-04-26T03:39:12.066 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:12.066 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:12.066 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:12.066 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:12.066 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:12.066 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:12.066 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 457 2024-04-26T03:39:12.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.067 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:12.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:12.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716766+0000", 2024-04-26T03:39:12.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:12.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:12.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716766+0000", 2024-04-26T03:39:12.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:12.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:12.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716766+0000", 2024-04-26T03:39:12.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6300000000000005e-07 2024-04-26T03:39:12.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:12.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716770+0000", 2024-04-26T03:39:12.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.377e-06 2024-04-26T03:39:12.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:12.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716776+0000", 2024-04-26T03:39:12.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6850000000000004e-06 2024-04-26T03:39:12.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:12.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716782+0000", 2024-04-26T03:39:12.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4779999999999999e-06 2024-04-26T03:39:12.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:12.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.730442+0000", 2024-04-26T03:39:12.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013660284 2024-04-26T03:39:12.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:12.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.730571+0000", 2024-04-26T03:39:12.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012850700000000001 2024-04-26T03:39:12.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:12.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.731245+0000", 2024-04-26T03:39:12.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00067431000000000001 2024-04-26T03:39:12.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:12.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.732606+0000", 2024-04-26T03:39:12.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013605850000000001 2024-04-26T03:39:12.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:12.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.734232+0000", 2024-04-26T03:39:12.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0016260809999999999 2024-04-26T03:39:12.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:12.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.734294+0000", 2024-04-26T03:39:12.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2725000000000007e-05 2024-04-26T03:39:12.072 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:12.072 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:12.072 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:12.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.073 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:458 3.as0 3:5930de0f:::smithi09235267-50:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:12.073 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:05.716790+0000", 2024-04-26T03:39:12.073 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.2366833760000002, 2024-04-26T03:39:12.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019815810999999999, 2024-04-26T03:39:12.073 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:12.073 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:12.073 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:12.074 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:12.074 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:12.074 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:12.074 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 458 2024-04-26T03:39:12.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.074 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:12.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:12.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716790+0000", 2024-04-26T03:39:12.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:12.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:12.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716790+0000", 2024-04-26T03:39:12.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:12.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:12.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716790+0000", 2024-04-26T03:39:12.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-04-26T03:39:12.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:12.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716792+0000", 2024-04-26T03:39:12.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3379999999999999e-06 2024-04-26T03:39:12.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:12.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716798+0000", 2024-04-26T03:39:12.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.066e-06 2024-04-26T03:39:12.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:12.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716803+0000", 2024-04-26T03:39:12.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.673e-06 2024-04-26T03:39:12.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:12.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.732363+0000", 2024-04-26T03:39:12.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015560218000000001 2024-04-26T03:39:12.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:12.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.732421+0000", 2024-04-26T03:39:12.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7649000000000003e-05 2024-04-26T03:39:12.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:12.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.734770+0000", 2024-04-26T03:39:12.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002348761 2024-04-26T03:39:12.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:12.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.734838+0000", 2024-04-26T03:39:12.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8471000000000002e-05 2024-04-26T03:39:12.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:12.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:12.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:12.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.736605+0000", 2024-04-26T03:39:12.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017672250000000001 2024-04-26T03:39:12.079 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:12.079 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:12.080 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:12.080 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:12.080 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:12.080 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:12.081 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0: {} 2024-04-26T03:39:12.082 INFO:teuthology.orchestra.run.smithi028.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T03:39:12.082 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1: {} 2024-04-26T03:39:12.083 INFO:teuthology.orchestra.run.smithi028.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T03:39:12.083 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-26T03:39:12.083 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2: {} 2024-04-26T03:39:12.084 INFO:teuthology.orchestra.run.smithi028.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T03:39:12.084 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3: {} 2024-04-26T03:39:12.084 INFO:teuthology.orchestra.run.smithi028.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T03:39:12.085 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4: {} 2024-04-26T03:39:12.085 INFO:teuthology.orchestra.run.smithi028.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T03:39:12.085 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5: {} 2024-04-26T03:39:12.086 INFO:teuthology.orchestra.run.smithi028.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T03:39:12.086 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-04-26T03:39:12.080+0000 7fe9223c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31954) UID: 0 2024-04-26T03:39:12.094 INFO:teuthology.orchestra.run.smithi028.stdout:osd.6: {} 2024-04-26T03:39:12.095 INFO:teuthology.orchestra.run.smithi028.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T03:39:12.112 INFO:teuthology.orchestra.run.smithi028.stdout:osd.7: {} 2024-04-26T03:39:12.112 INFO:teuthology.orchestra.run.smithi028.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T03:39:12.129 INFO:teuthology.orchestra.run.smithi028.stdout:osd.8: {} 2024-04-26T03:39:12.129 INFO:teuthology.orchestra.run.smithi028.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T03:39:12.147 INFO:teuthology.orchestra.run.smithi028.stdout:osd.9: {} 2024-04-26T03:39:12.147 INFO:teuthology.orchestra.run.smithi028.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T03:39:12.165 INFO:teuthology.orchestra.run.smithi028.stdout:osd.10: {} 2024-04-26T03:39:12.166 INFO:teuthology.orchestra.run.smithi028.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T03:39:12.182 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:39:12.181+0000 7f60d3c4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31757) UID: 0 2024-04-26T03:39:12.183 INFO:teuthology.orchestra.run.smithi028.stdout:osd.11: {} 2024-04-26T03:39:12.183 INFO:teuthology.orchestra.run.smithi028.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T03:39:12.200 INFO:teuthology.orchestra.run.smithi028.stdout:osd.12: {} 2024-04-26T03:39:12.200 INFO:teuthology.orchestra.run.smithi028.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T03:39:12.218 INFO:teuthology.orchestra.run.smithi028.stdout:osd.13: {} 2024-04-26T03:39:12.218 INFO:teuthology.orchestra.run.smithi028.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T03:39:12.224 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:12.224 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:12.233 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-26T03:39:12.236 INFO:teuthology.orchestra.run.smithi028.stdout:osd.14: {} 2024-04-26T03:39:12.236 INFO:teuthology.orchestra.run.smithi028.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T03:39:12.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:39:12.235+0000 7fc889025640 0 -- 172.21.15.28:0/4241233552 >> [v2:172.21.15.92:6826/3982634323,v1:172.21.15.92:6827/3982634323] conn(0x7fc884222b20 msgr2=0x7fc884224f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-26T03:39:12.283 INFO:tasks.ceph.osd.15.smithi092.stderr:2024-04-26T03:39:12.281+0000 7f6d12dec640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 31824) UID: 0 2024-04-26T03:39:12.335 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:12.335 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:12.344 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-26T03:39:12.384 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:39:12.383+0000 7fba0d2ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31961) UID: 0 2024-04-26T03:39:12.455 INFO:teuthology.orchestra.run.smithi028.stdout:osd.15: {} 2024-04-26T03:39:12.456 INFO:teuthology.orchestra.run.smithi028.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T03:39:12.485 INFO:tasks.ceph.osd.10.smithi066.stderr:2024-04-26T03:39:12.484+0000 7f130fcc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 31763) UID: 0 2024-04-26T03:39:12.486 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:12.487 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:12.495 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-04-26T03:39:12.586 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-04-26T03:39:12.585+0000 7fee562fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31759) UID: 0 2024-04-26T03:39:12.598 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:12.598 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:12.607 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-04-26T03:39:12.687 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-26T03:39:12.686+0000 7f610b3a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31816) UID: 0 2024-04-26T03:39:12.707 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:12.708 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:12.716 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-04-26T03:39:12.720 INFO:tasks.rados.rados.0.smithi092.stdout:235: done (6 left) 2024-04-26T03:39:12.720 INFO:tasks.rados.rados.0.smithi092.stdout:237: done (5 left) 2024-04-26T03:39:12.720 INFO:tasks.rados.rados.0.smithi092.stdout:238: done (4 left) 2024-04-26T03:39:12.720 INFO:tasks.rados.rados.0.smithi092.stdout:239: done (3 left) 2024-04-26T03:39:12.720 INFO:tasks.rados.rados.0.smithi092.stdout:240: done (2 left) 2024-04-26T03:39:12.720 INFO:tasks.rados.rados.0.smithi092.stdout:241: done (1 left) 2024-04-26T03:39:12.720 INFO:tasks.rados.rados.0.smithi092.stdout:242: done (0 left) 2024-04-26T03:39:12.720 INFO:tasks.rados.rados.0.smithi092.stdout:243: copy_from oid 23 from oid 2 current snap is 13 2024-04-26T03:39:12.720 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 44 current snap is 13 2024-04-26T03:39:12.720 INFO:tasks.rados.rados.0.smithi092.stdout:244: seq_num 114 ranges {2293760=753664,3047424=688128,3735552=475136} 2024-04-26T03:39:12.738 INFO:tasks.rados.rados.0.smithi092.stdout:244: writing smithi09235267-44 from 2293760 to 3047424 tid 1 2024-04-26T03:39:12.740 INFO:tasks.rados.rados.0.smithi092.stdout:244: writing smithi09235267-44 from 3047424 to 3735552 tid 2 2024-04-26T03:39:12.744 INFO:tasks.rados.rados.0.smithi092.stdout:244: writing smithi09235267-44 from 3735552 to 4210688 tid 3 2024-04-26T03:39:12.744 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 3 current snap is 13 2024-04-26T03:39:12.749 INFO:tasks.rados.rados.0.smithi092.stdout:245: seq_num 115 ranges {1867776=770048,2637824=704512,3342336=458752} 2024-04-26T03:39:12.760 INFO:tasks.rados.rados.0.smithi092.stdout:245: writing smithi09235267-3 from 1867776 to 2637824 tid 1 2024-04-26T03:39:12.765 INFO:tasks.rados.rados.0.smithi092.stdout:245: writing smithi09235267-3 from 2637824 to 3342336 tid 2 2024-04-26T03:39:12.769 INFO:tasks.rados.rados.0.smithi092.stdout:245: writing smithi09235267-3 from 3342336 to 3801088 tid 3 2024-04-26T03:39:12.769 INFO:tasks.rados.rados.0.smithi092.stdout:246: delete oid 35 current snap is 13 2024-04-26T03:39:12.769 INFO:tasks.rados.rados.0.smithi092.stdout:243: finishing copy_from racing read to smithi09235267-23 2024-04-26T03:39:12.769 INFO:tasks.rados.rados.0.smithi092.stdout:243: finishing copy_from to smithi09235267-23 2024-04-26T03:39:12.769 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 23 v 55 (ObjNum 110 snap 8 seq_num 110) dirty exists 2024-04-26T03:39:12.770 INFO:tasks.rados.rados.0.smithi092.stdout:244: finishing write tid 1 to smithi09235267-44 2024-04-26T03:39:12.770 INFO:tasks.rados.rados.0.smithi092.stdout:244: finishing write tid 2 to smithi09235267-44 2024-04-26T03:39:12.770 INFO:tasks.rados.rados.0.smithi092.stdout:244: finishing write tid 3 to smithi09235267-44 2024-04-26T03:39:12.770 INFO:tasks.rados.rados.0.smithi092.stdout:244: finishing write tid 4 to smithi09235267-44 2024-04-26T03:39:12.770 INFO:tasks.rados.rados.0.smithi092.stdout:244: finishing write tid 5 to smithi09235267-44 2024-04-26T03:39:12.770 INFO:tasks.rados.rados.0.smithi092.stdout:244: oid 44 updating version 0 to 49 2024-04-26T03:39:12.770 INFO:tasks.rados.rados.0.smithi092.stdout:244: oid 44 version 49 is already newer than 46 2024-04-26T03:39:12.770 INFO:tasks.rados.rados.0.smithi092.stdout:244: oid 44 version 49 is already newer than 47 2024-04-26T03:39:12.770 INFO:tasks.rados.rados.0.smithi092.stdout:244: oid 44 version 49 is already newer than 48 2024-04-26T03:39:12.770 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 44 v 49 (ObjNum 114 snap 13 seq_num 114) dirty exists 2024-04-26T03:39:12.770 INFO:tasks.rados.rados.0.smithi092.stdout:244: left oid 44 (ObjNum 114 snap 13 seq_num 114) 2024-04-26T03:39:12.771 INFO:tasks.rados.rados.0.smithi092.stdout:243: done (3 left) 2024-04-26T03:39:12.771 INFO:tasks.rados.rados.0.smithi092.stdout:244: done (2 left) 2024-04-26T03:39:12.771 INFO:tasks.rados.rados.0.smithi092.stdout:246: done (1 left) 2024-04-26T03:39:12.771 INFO:tasks.rados.rados.0.smithi092.stdout:247: delete oid 38 current snap is 13 2024-04-26T03:39:12.773 INFO:tasks.rados.rados.0.smithi092.stdout:247: done (1 left) 2024-04-26T03:39:12.773 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 19 current snap is 13 2024-04-26T03:39:12.773 INFO:tasks.rados.rados.0.smithi092.stdout:248: seq_num 118 ranges {3014656=442368,3457024=753664,4210688=655360,4866048=147456} 2024-04-26T03:39:12.788 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:39:12.787+0000 7fb78c544640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31956) UID: 0 2024-04-26T03:39:12.791 INFO:tasks.rados.rados.0.smithi092.stdout:248: writing smithi09235267-19 from 3014656 to 3457024 tid 1 2024-04-26T03:39:12.796 INFO:tasks.rados.rados.0.smithi092.stdout:248: writing smithi09235267-19 from 3457024 to 4210688 tid 2 2024-04-26T03:39:12.799 INFO:tasks.rados.rados.0.smithi092.stdout:248: writing smithi09235267-19 from 4210688 to 4866048 tid 3 2024-04-26T03:39:12.801 INFO:tasks.rados.rados.0.smithi092.stdout:248: writing smithi09235267-19 from 4866048 to 5013504 tid 4 2024-04-26T03:39:12.801 INFO:tasks.rados.rados.0.smithi092.stdout:249: read oid 44 snap 12 2024-04-26T03:39:12.801 INFO:tasks.rados.rados.0.smithi092.stdout:249: expect (ObjNum 92 snap 7 seq_num 92) 2024-04-26T03:39:12.801 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 2 current snap is 13 2024-04-26T03:39:12.801 INFO:tasks.rados.rados.0.smithi092.stdout:245: finishing write tid 1 to smithi09235267-3 2024-04-26T03:39:12.801 INFO:tasks.rados.rados.0.smithi092.stdout:250: seq_num 119 ranges {2670592=458752,3129344=507904,3637248=507904,4145152=458752,4603904=81920} 2024-04-26T03:39:12.818 INFO:tasks.rados.rados.0.smithi092.stdout:250: writing smithi09235267-2 from 2670592 to 3129344 tid 1 2024-04-26T03:39:12.821 INFO:tasks.rados.rados.0.smithi092.stdout:250: writing smithi09235267-2 from 3129344 to 3637248 tid 2 2024-04-26T03:39:12.824 INFO:tasks.rados.rados.0.smithi092.stdout:250: writing smithi09235267-2 from 3637248 to 4145152 tid 3 2024-04-26T03:39:12.827 INFO:tasks.rados.rados.0.smithi092.stdout:250: writing smithi09235267-2 from 4145152 to 4603904 tid 4 2024-04-26T03:39:12.830 INFO:tasks.rados.rados.0.smithi092.stdout:250: writing smithi09235267-2 from 4603904 to 4685824 tid 5 2024-04-26T03:39:12.830 INFO:tasks.rados.rados.0.smithi092.stdout:251: snap_create 2024-04-26T03:39:12.830 INFO:tasks.rados.rados.0.smithi092.stdout:245: finishing write tid 2 to smithi09235267-3 2024-04-26T03:39:12.830 INFO:tasks.rados.rados.0.smithi092.stdout:245: finishing write tid 3 to smithi09235267-3 2024-04-26T03:39:12.830 INFO:tasks.rados.rados.0.smithi092.stdout:245: finishing write tid 4 to smithi09235267-3 2024-04-26T03:39:12.830 INFO:tasks.rados.rados.0.smithi092.stdout:245: finishing write tid 5 to smithi09235267-3 2024-04-26T03:39:12.830 INFO:tasks.rados.rados.0.smithi092.stdout:245: oid 3 updating version 0 to 61 2024-04-26T03:39:12.830 INFO:tasks.rados.rados.0.smithi092.stdout:245: oid 3 version 61 is already newer than 60 2024-04-26T03:39:12.830 INFO:tasks.rados.rados.0.smithi092.stdout:245: oid 3 updating version 61 to 62 2024-04-26T03:39:12.830 INFO:tasks.rados.rados.0.smithi092.stdout:245: oid 3 updating version 62 to 63 2024-04-26T03:39:12.830 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 3 v 63 (ObjNum 115 snap 13 seq_num 115) dirty exists 2024-04-26T03:39:12.830 INFO:tasks.rados.rados.0.smithi092.stdout:245: left oid 3 (ObjNum 115 snap 13 seq_num 115) 2024-04-26T03:39:12.831 INFO:tasks.rados.rados.0.smithi092.stdout:248: finishing write tid 1 to smithi09235267-19 2024-04-26T03:39:12.831 INFO:tasks.rados.rados.0.smithi092.stdout:249: expect (ObjNum 92 snap 7 seq_num 92) 2024-04-26T03:39:12.848 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:12.848 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:12.857 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-04-26T03:39:12.899 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-04-26T03:39:12.897+0000 7f745118d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 31821) UID: 0 2024-04-26T03:39:12.901 INFO:tasks.rados.rados.0.smithi092.stdout:248: finishing write tid 2 to smithi09235267-19 2024-04-26T03:39:12.901 INFO:tasks.rados.rados.0.smithi092.stdout:248: finishing write tid 3 to smithi09235267-19 2024-04-26T03:39:12.901 INFO:tasks.rados.rados.0.smithi092.stdout:248: finishing write tid 4 to smithi09235267-19 2024-04-26T03:39:12.901 INFO:tasks.rados.rados.0.smithi092.stdout:248: finishing write tid 5 to smithi09235267-19 2024-04-26T03:39:12.901 INFO:tasks.rados.rados.0.smithi092.stdout:248: finishing write tid 6 to smithi09235267-19 2024-04-26T03:39:12.901 INFO:tasks.rados.rados.0.smithi092.stdout:248: oid 19 updating version 0 to 58 2024-04-26T03:39:12.901 INFO:tasks.rados.rados.0.smithi092.stdout:248: oid 19 updating version 58 to 59 2024-04-26T03:39:12.901 INFO:tasks.rados.rados.0.smithi092.stdout:248: oid 19 updating version 59 to 61 2024-04-26T03:39:12.901 INFO:tasks.rados.rados.0.smithi092.stdout:248: oid 19 version 61 is already newer than 57 2024-04-26T03:39:12.901 INFO:tasks.rados.rados.0.smithi092.stdout:248: oid 19 version 61 is already newer than 60 2024-04-26T03:39:12.901 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 19 v 61 (ObjNum 118 snap 13 seq_num 118) dirty exists 2024-04-26T03:39:12.901 INFO:tasks.rados.rados.0.smithi092.stdout:248: left oid 19 (ObjNum 118 snap 13 seq_num 118) 2024-04-26T03:39:12.901 INFO:tasks.rados.rados.0.smithi092.stdout:250: finishing write tid 1 to smithi09235267-2 2024-04-26T03:39:12.902 INFO:tasks.rados.rados.0.smithi092.stdout:250: finishing write tid 2 to smithi09235267-2 2024-04-26T03:39:12.902 INFO:tasks.rados.rados.0.smithi092.stdout:250: finishing write tid 3 to smithi09235267-2 2024-04-26T03:39:12.902 INFO:tasks.rados.rados.0.smithi092.stdout:250: finishing write tid 4 to smithi09235267-2 2024-04-26T03:39:12.902 INFO:tasks.rados.rados.0.smithi092.stdout:250: finishing write tid 5 to smithi09235267-2 2024-04-26T03:39:12.902 INFO:tasks.rados.rados.0.smithi092.stdout:250: finishing write tid 6 to smithi09235267-2 2024-04-26T03:39:12.902 INFO:tasks.rados.rados.0.smithi092.stdout:250: finishing write tid 7 to smithi09235267-2 2024-04-26T03:39:12.902 INFO:tasks.rados.rados.0.smithi092.stdout:250: oid 2 updating version 0 to 68 2024-04-26T03:39:12.902 INFO:tasks.rados.rados.0.smithi092.stdout:250: oid 2 version 68 is already newer than 63 2024-04-26T03:39:12.902 INFO:tasks.rados.rados.0.smithi092.stdout:250: oid 2 version 68 is already newer than 67 2024-04-26T03:39:12.902 INFO:tasks.rados.rados.0.smithi092.stdout:250: oid 2 version 68 is already newer than 65 2024-04-26T03:39:12.903 INFO:tasks.rados.rados.0.smithi092.stdout:250: oid 2 version 68 is already newer than 64 2024-04-26T03:39:12.903 INFO:tasks.rados.rados.0.smithi092.stdout:250: oid 2 version 68 is already newer than 66 2024-04-26T03:39:12.903 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 2 v 68 (ObjNum 119 snap 13 seq_num 119) dirty exists 2024-04-26T03:39:12.903 INFO:tasks.rados.rados.0.smithi092.stdout:250: left oid 2 (ObjNum 119 snap 13 seq_num 119) 2024-04-26T03:39:12.991 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-04-26T03:39:12.989+0000 7fdc2d21c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31819) UID: 0 2024-04-26T03:39:13.000 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:13.000 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:13.009 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-04-26T03:39:13.092 INFO:tasks.ceph.osd.9.smithi042.stderr:2024-04-26T03:39:13.091+0000 7fdac63b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 31960) UID: 0 2024-04-26T03:39:13.152 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:13.152 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:13.160 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-04-26T03:39:13.193 INFO:tasks.ceph.osd.8.smithi028.stderr:2024-04-26T03:39:13.192+0000 7f5bb75d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 31968) UID: 0 2024-04-26T03:39:13.263 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:13.263 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:13.271 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-26T03:39:13.294 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-04-26T03:39:13.293+0000 7f745118d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 31821) UID: 0 2024-04-26T03:39:13.391 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:13.391 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:13.395 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:39:13.394+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:39:13.400 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-26T03:39:13.496 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:39:13.495+0000 7fba0d2ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31961) UID: 0 2024-04-26T03:39:13.507 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:13.507 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:13.516 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-26T03:39:13.597 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-04-26T03:39:13.596+0000 7fee562fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31759) UID: 0 2024-04-26T03:39:13.663 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:13.663 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:13.672 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-26T03:39:13.698 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:39:13.697+0000 7f60d3c4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31757) UID: 0 2024-04-26T03:39:13.721 INFO:tasks.rados.rados.0.smithi092.stdout:245: done (4 left) 2024-04-26T03:39:13.721 INFO:tasks.rados.rados.0.smithi092.stdout:248: done (3 left) 2024-04-26T03:39:13.721 INFO:tasks.rados.rados.0.smithi092.stdout:249: done (2 left) 2024-04-26T03:39:13.721 INFO:tasks.rados.rados.0.smithi092.stdout:250: done (1 left) 2024-04-26T03:39:13.721 INFO:tasks.rados.rados.0.smithi092.stdout:251: done (0 left) 2024-04-26T03:39:13.721 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 46 current snap is 14 2024-04-26T03:39:13.721 INFO:tasks.rados.rados.0.smithi092.stdout:252: seq_num 120 ranges {1720320=475136,2195456=655360,2850816=622592,3473408=278528} 2024-04-26T03:39:13.724 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-26T03:39:13.732 INFO:tasks.rados.rados.0.smithi092.stdout:252: writing smithi09235267-46 from 1720320 to 2195456 tid 1 2024-04-26T03:39:13.737 INFO:tasks.rados.rados.0.smithi092.stdout:252: writing smithi09235267-46 from 2195456 to 2850816 tid 2 2024-04-26T03:39:13.741 INFO:tasks.rados.rados.0.smithi092.stdout:252: writing smithi09235267-46 from 2850816 to 3473408 tid 3 2024-04-26T03:39:13.743 INFO:tasks.rados.rados.0.smithi092.stdout:252: writing smithi09235267-46 from 3473408 to 3751936 tid 4 2024-04-26T03:39:13.743 INFO:tasks.rados.rados.0.smithi092.stdout:253: read oid 19 snap -1 2024-04-26T03:39:13.744 INFO:tasks.rados.rados.0.smithi092.stdout:253: expect (ObjNum 118 snap 13 seq_num 118) 2024-04-26T03:39:13.744 INFO:tasks.rados.rados.0.smithi092.stdout:254: copy_from oid 22 from oid 45 current snap is 14 2024-04-26T03:39:13.744 INFO:tasks.rados.rados.0.smithi092.stdout:255: setattr oid 40 current snap is 14 2024-04-26T03:39:13.745 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 13 current snap is 14 2024-04-26T03:39:13.745 INFO:tasks.rados.rados.0.smithi092.stdout:256: seq_num 121 ranges {0=491520,491520=802816,1294336=720896,2015232=32768} 2024-04-26T03:39:13.747 INFO:tasks.rados.rados.0.smithi092.stdout:256: writing smithi09235267-13 from 0 to 491520 tid 1 2024-04-26T03:39:13.753 INFO:tasks.rados.rados.0.smithi092.stdout:256: writing smithi09235267-13 from 491520 to 1294336 tid 2 2024-04-26T03:39:13.758 INFO:tasks.rados.rados.0.smithi092.stdout:256: writing smithi09235267-13 from 1294336 to 2015232 tid 3 2024-04-26T03:39:13.758 INFO:tasks.rados.rados.0.smithi092.stdout:256: writing smithi09235267-13 from 2015232 to 2048000 tid 4 2024-04-26T03:39:13.758 INFO:tasks.rados.rados.0.smithi092.stdout:257: setattr oid 50 current snap is 14 2024-04-26T03:39:13.759 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 40 v 15 (ObjNum 60 snap 0 seq_num 60) dirty exists 2024-04-26T03:39:13.759 INFO:tasks.rados.rados.0.smithi092.stdout:252: finishing write tid 1 to smithi09235267-46 2024-04-26T03:39:13.759 INFO:tasks.rados.rados.0.smithi092.stdout:254: finishing copy_from racing read to smithi09235267-22 2024-04-26T03:39:13.759 INFO:tasks.rados.rados.0.smithi092.stdout:252: finishing write tid 2 to smithi09235267-46 2024-04-26T03:39:13.760 INFO:tasks.rados.rados.0.smithi092.stdout:255: done (5 left) 2024-04-26T03:39:13.760 INFO:tasks.rados.rados.0.smithi092.stdout:258: read oid 29 snap -1 2024-04-26T03:39:13.760 INFO:tasks.rados.rados.0.smithi092.stdout:258: expect (ObjNum 28 snap 0 seq_num 28) 2024-04-26T03:39:13.760 INFO:tasks.rados.rados.0.smithi092.stdout:259: snap_remove snap 8 2024-04-26T03:39:13.799 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:13.799 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:13.799 INFO:tasks.ceph.osd.10.smithi066.stderr:2024-04-26T03:39:13.798+0000 7f130fcc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 31763) UID: 0 2024-04-26T03:39:13.808 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-26T03:39:13.900 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-04-26T03:39:13.899+0000 7fee562fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31759) UID: 0 2024-04-26T03:39:13.966 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:13.966 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:13.978 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-26T03:39:14.022 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:39:14.021+0000 7fba0d2ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31961) UID: 0 2024-04-26T03:39:14.085 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:14.086 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:14.094 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-04-26T03:39:14.102 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:39:14.100+0000 7fb78c544640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31956) UID: 0 2024-04-26T03:39:14.202 INFO:tasks.ceph.osd.9.smithi042.stderr:2024-04-26T03:39:14.201+0000 7fdac63b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 31960) UID: 0 2024-04-26T03:39:14.242 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:14.242 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:14.251 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-04-26T03:39:14.304 INFO:tasks.ceph.osd.4.smithi028.stderr:2024-04-26T03:39:14.303+0000 7fd9511b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31963) UID: 0 2024-04-26T03:39:14.359 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:14.359 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:14.368 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-04-26T03:39:14.404 INFO:tasks.ceph.osd.9.smithi042.stderr:2024-04-26T03:39:14.403+0000 7fdac63b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 31960) UID: 0 2024-04-26T03:39:14.468 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-26T03:39:14.505 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:39:14.504+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:39:14.516 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:14.516 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:14.525 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-04-26T03:39:14.594 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0: {} 2024-04-26T03:39:14.594 INFO:teuthology.orchestra.run.smithi028.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T03:39:14.606 INFO:tasks.ceph.osd.4.smithi028.stderr:2024-04-26T03:39:14.605+0000 7fd9511b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31963) UID: 0 2024-04-26T03:39:14.613 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1: {} 2024-04-26T03:39:14.613 INFO:teuthology.orchestra.run.smithi028.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T03:39:14.632 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2: {} 2024-04-26T03:39:14.632 INFO:teuthology.orchestra.run.smithi028.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T03:39:14.634 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:14.634 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:14.643 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-04-26T03:39:14.649 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3: {} 2024-04-26T03:39:14.650 INFO:teuthology.orchestra.run.smithi028.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T03:39:14.666 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4: {} 2024-04-26T03:39:14.667 INFO:teuthology.orchestra.run.smithi028.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T03:39:14.683 INFO:teuthology.orchestra.run.smithi028.stderr:nodeep-scrub is set 2024-04-26T03:39:14.688 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5: {} 2024-04-26T03:39:14.688 INFO:teuthology.orchestra.run.smithi028.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T03:39:14.688 INFO:tasks.rados.rados.0.smithi092.stdout:259: done (6 left) 2024-04-26T03:39:14.689 INFO:tasks.rados.rados.0.smithi092.stdout:260: snap_remove snap 9 2024-04-26T03:39:14.707 INFO:tasks.ceph.osd.14.smithi066.stderr:2024-04-26T03:39:14.706+0000 7f5395f08640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 31765) UID: 0 2024-04-26T03:39:14.717 INFO:teuthology.orchestra.run.smithi028.stdout:osd.6: {} 2024-04-26T03:39:14.717 INFO:teuthology.orchestra.run.smithi028.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T03:39:14.735 INFO:teuthology.orchestra.run.smithi028.stdout:osd.7: {} 2024-04-26T03:39:14.736 INFO:teuthology.orchestra.run.smithi028.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T03:39:14.753 INFO:teuthology.orchestra.run.smithi028.stdout:osd.8: {} 2024-04-26T03:39:14.753 INFO:teuthology.orchestra.run.smithi028.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T03:39:14.768 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:14.768 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:14.771 INFO:teuthology.orchestra.run.smithi028.stdout:osd.9: {} 2024-04-26T03:39:14.771 INFO:teuthology.orchestra.run.smithi028.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T03:39:14.777 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-04-26T03:39:14.788 INFO:teuthology.orchestra.run.smithi028.stdout:osd.10: {} 2024-04-26T03:39:14.789 INFO:teuthology.orchestra.run.smithi028.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T03:39:14.806 INFO:teuthology.orchestra.run.smithi028.stdout:osd.11: {} 2024-04-26T03:39:14.806 INFO:teuthology.orchestra.run.smithi028.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T03:39:14.808 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-04-26T03:39:14.807+0000 7f745118d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 31821) UID: 0 2024-04-26T03:39:14.823 INFO:teuthology.orchestra.run.smithi028.stdout:osd.12: {} 2024-04-26T03:39:14.823 INFO:teuthology.orchestra.run.smithi028.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T03:39:14.841 INFO:teuthology.orchestra.run.smithi028.stdout:osd.13: {} 2024-04-26T03:39:14.841 INFO:teuthology.orchestra.run.smithi028.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T03:39:14.859 INFO:teuthology.orchestra.run.smithi028.stdout:osd.14: {} 2024-04-26T03:39:14.860 INFO:teuthology.orchestra.run.smithi028.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T03:39:14.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:39:14.859+0000 7fb60ad9d640 0 -- 172.21.15.28:0/3084424832 >> [v2:172.21.15.92:6826/3982634323,v1:172.21.15.92:6827/3982634323] conn(0x7fb60c222740 msgr2=0x7fb60c224b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-26T03:39:14.897 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:39:14.897 INFO:teuthology.orchestra.run.smithi028.stdout: "size": 20, 2024-04-26T03:39:14.897 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 600, 2024-04-26T03:39:14.897 INFO:teuthology.orchestra.run.smithi028.stdout: "ops": [ 2024-04-26T03:39:14.897 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.897 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.es6 35/32 ECSubWrite(tid=2, reqid=client.4556.0:58, at_version=35'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:39:14.897 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.512937+0000", 2024-04-26T03:39:14.897 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 23.381769236, 2024-04-26T03:39:14.897 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0046369269999999999, 2024-04-26T03:39:14.898 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:14.898 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:14.898 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:14.898 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:14.898 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.898 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:14.898 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.512937+0000", 2024-04-26T03:39:14.898 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:14.898 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.898 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.898 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:14.898 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.512937+0000", 2024-04-26T03:39:14.898 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:14.899 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.899 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.899 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:14.899 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.512937+0000", 2024-04-26T03:39:14.899 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.8099999999999999e-07 2024-04-26T03:39:14.899 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.899 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.899 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:14.899 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.513175+0000", 2024-04-26T03:39:14.899 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000237851 2024-04-26T03:39:14.899 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.899 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.899 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:14.900 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.513196+0000", 2024-04-26T03:39:14.900 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.0344999999999999e-05 2024-04-26T03:39:14.900 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.900 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.900 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:14.900 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.513207+0000", 2024-04-26T03:39:14.900 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1083999999999999e-05 2024-04-26T03:39:14.900 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.900 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.900 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:14.900 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.513309+0000", 2024-04-26T03:39:14.900 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000102321 2024-04-26T03:39:14.900 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.901 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.901 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:14.901 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.513318+0000", 2024-04-26T03:39:14.901 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.5219999999999996e-06 2024-04-26T03:39:14.901 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.901 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.901 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:14.901 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.517543+0000", 2024-04-26T03:39:14.901 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0042245160000000002 2024-04-26T03:39:14.901 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.901 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.901 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:14.901 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.517574+0000", 2024-04-26T03:39:14.901 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.1007000000000003e-05 2024-04-26T03:39:14.902 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:14.902 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:14.902 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:14.902 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.902 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.902 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.bs3 35/32 ECSubWrite(tid=10, reqid=client.4556.0:61, at_version=35'5, trim_to=0'0, roll_forward_to=35'3))", 2024-04-26T03:39:14.902 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.514935+0000", 2024-04-26T03:39:14.902 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 23.379771501, 2024-04-26T03:39:14.902 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0043992700000000003, 2024-04-26T03:39:14.902 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:14.902 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:14.902 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:14.902 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:14.903 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.903 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:14.903 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.514935+0000", 2024-04-26T03:39:14.903 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:14.903 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.903 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.903 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:14.903 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.514935+0000", 2024-04-26T03:39:14.903 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:14.903 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.903 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.903 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:14.903 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.514936+0000", 2024-04-26T03:39:14.903 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1170000000000001e-06 2024-04-26T03:39:14.903 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.904 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.904 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:14.904 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.515012+0000", 2024-04-26T03:39:14.904 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.5631999999999997e-05 2024-04-26T03:39:14.904 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.904 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.904 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:14.904 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.515030+0000", 2024-04-26T03:39:14.904 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.8881000000000001e-05 2024-04-26T03:39:14.904 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.904 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.904 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:14.904 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.515040+0000", 2024-04-26T03:39:14.904 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.6039999999999999e-06 2024-04-26T03:39:14.905 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.905 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.905 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:14.905 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.515118+0000", 2024-04-26T03:39:14.905 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.8121000000000006e-05 2024-04-26T03:39:14.905 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.905 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.905 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:14.905 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.515129+0000", 2024-04-26T03:39:14.905 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.04e-05 2024-04-26T03:39:14.905 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.905 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.905 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:14.905 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.519303+0000", 2024-04-26T03:39:14.906 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0041743129999999998 2024-04-26T03:39:14.906 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.906 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.906 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:14.906 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.519334+0000", 2024-04-26T03:39:14.906 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.1201999999999997e-05 2024-04-26T03:39:14.906 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:14.906 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:14.906 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:14.906 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.906 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.907 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.es6 35/32 ECSubWrite(tid=8, reqid=client.4556.0:70, at_version=35'3, trim_to=0'0, roll_forward_to=35'2))", 2024-04-26T03:39:14.907 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.523457+0000", 2024-04-26T03:39:14.907 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 23.371249379999998, 2024-04-26T03:39:14.907 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0042674139999999998, 2024-04-26T03:39:14.908 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:14.908 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:14.908 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:14.908 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:14.908 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.908 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:14.908 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.523457+0000", 2024-04-26T03:39:14.908 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:14.908 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.908 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.908 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:14.908 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.523457+0000", 2024-04-26T03:39:14.908 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:14.908 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.909 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.909 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:14.909 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.523458+0000", 2024-04-26T03:39:14.909 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.78e-07 2024-04-26T03:39:14.909 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.909 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.909 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:14.909 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.523548+0000", 2024-04-26T03:39:14.909 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.0366999999999997e-05 2024-04-26T03:39:14.909 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.909 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.910 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:14.910 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.523563+0000", 2024-04-26T03:39:14.910 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.4450999999999999e-05 2024-04-26T03:39:14.910 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.910 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.910 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:14.910 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.523571+0000", 2024-04-26T03:39:14.910 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.4659999999999999e-06 2024-04-26T03:39:14.910 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.910 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.910 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:14.911 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.523664+0000", 2024-04-26T03:39:14.911 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.3312000000000004e-05 2024-04-26T03:39:14.911 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.911 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.911 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:14.912 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.523673+0000", 2024-04-26T03:39:14.912 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.9660000000000002e-06 2024-04-26T03:39:14.912 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.912 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.912 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:14.912 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.527682+0000", 2024-04-26T03:39:14.912 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0040086540000000004 2024-04-26T03:39:14.912 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.912 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.912 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:14.912 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.527724+0000", 2024-04-26T03:39:14.912 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.2320000000000001e-05 2024-04-26T03:39:14.912 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:14.912 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:14.913 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:14.913 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.913 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.913 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.es6 35/32 ECSubWrite(tid=16, reqid=client.4556.0:85, at_version=35'5, trim_to=0'0, roll_forward_to=35'4))", 2024-04-26T03:39:14.913 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.542343+0000", 2024-04-26T03:39:14.913 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 23.352363671999999, 2024-04-26T03:39:14.913 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0051706219999999997, 2024-04-26T03:39:14.913 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:14.913 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:14.913 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:14.913 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:14.913 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.913 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:14.913 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.542343+0000", 2024-04-26T03:39:14.914 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:14.914 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.914 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.914 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:14.914 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.542343+0000", 2024-04-26T03:39:14.914 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:14.914 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.914 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.914 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:14.915 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.542344+0000", 2024-04-26T03:39:14.915 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.488e-06 2024-04-26T03:39:14.915 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.915 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.915 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:14.915 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.542511+0000", 2024-04-26T03:39:14.915 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000166844 2024-04-26T03:39:14.916 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.916 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.917 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:14.917 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.542528+0000", 2024-04-26T03:39:14.917 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.6548000000000001e-05 2024-04-26T03:39:14.917 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.917 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.917 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:14.917 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.542535+0000", 2024-04-26T03:39:14.917 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.7610000000000007e-06 2024-04-26T03:39:14.917 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.917 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.917 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:14.917 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.542610+0000", 2024-04-26T03:39:14.918 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.4632e-05 2024-04-26T03:39:14.918 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.918 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.918 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:14.918 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.542620+0000", 2024-04-26T03:39:14.918 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0108e-05 2024-04-26T03:39:14.918 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.918 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.918 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:14.918 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.547469+0000", 2024-04-26T03:39:14.918 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0048486129999999999 2024-04-26T03:39:14.918 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.918 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.919 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:14.919 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.547513+0000", 2024-04-26T03:39:14.919 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.4628e-05 2024-04-26T03:39:14.919 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:14.919 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:14.919 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:14.919 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.919 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.919 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.es6 35/32 ECSubWrite(tid=24, reqid=client.4556.0:124, at_version=35'7, trim_to=0'0, roll_forward_to=35'6))", 2024-04-26T03:39:14.920 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.596283+0000", 2024-04-26T03:39:14.920 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 23.298423372999999, 2024-04-26T03:39:14.920 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0047651710000000003, 2024-04-26T03:39:14.920 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:14.920 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:14.920 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:14.920 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:14.920 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.920 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:14.920 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.596283+0000", 2024-04-26T03:39:14.920 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:14.920 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.920 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.921 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:14.921 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.596283+0000", 2024-04-26T03:39:14.921 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:14.921 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.921 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.921 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:14.921 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.596284+0000", 2024-04-26T03:39:14.921 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0950000000000001e-06 2024-04-26T03:39:14.921 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.921 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.921 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:14.921 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.596595+0000", 2024-04-26T03:39:14.921 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00031093400000000002 2024-04-26T03:39:14.921 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.922 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.922 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:14.922 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.596610+0000", 2024-04-26T03:39:14.922 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.5264000000000002e-05 2024-04-26T03:39:14.922 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.922 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.922 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:14.922 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.596617+0000", 2024-04-26T03:39:14.922 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.0859999999999996e-06 2024-04-26T03:39:14.922 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.922 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.922 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:14.922 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.596669+0000", 2024-04-26T03:39:14.922 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.1436e-05 2024-04-26T03:39:14.922 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.923 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.923 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:14.923 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.596675+0000", 2024-04-26T03:39:14.923 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.4030000000000004e-06 2024-04-26T03:39:14.923 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.923 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.923 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:14.923 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601010+0000", 2024-04-26T03:39:14.923 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0043352030000000001 2024-04-26T03:39:14.923 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.923 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.923 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:14.923 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601048+0000", 2024-04-26T03:39:14.924 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.7750000000000003e-05 2024-04-26T03:39:14.924 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:14.924 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:14.924 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:14.924 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.924 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.924 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.bs3 35/32 ECSubWrite(tid=20, reqid=client.4556.0:127, at_version=35'7, trim_to=0'0, roll_forward_to=35'6))", 2024-04-26T03:39:14.924 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.601363+0000", 2024-04-26T03:39:14.924 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 23.293343415999999, 2024-04-26T03:39:14.924 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.007091913, 2024-04-26T03:39:14.924 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:14.924 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:14.925 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:14.925 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:14.925 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.925 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:14.925 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601363+0000", 2024-04-26T03:39:14.925 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:14.925 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.925 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.925 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:14.925 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601363+0000", 2024-04-26T03:39:14.925 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:14.925 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.925 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.925 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:14.925 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601364+0000", 2024-04-26T03:39:14.926 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.54e-07 2024-04-26T03:39:14.926 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.926 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.926 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:14.926 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601498+0000", 2024-04-26T03:39:14.926 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00013417600000000001 2024-04-26T03:39:14.926 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.926 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.926 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:14.926 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601514+0000", 2024-04-26T03:39:14.926 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.6371999999999998e-05 2024-04-26T03:39:14.926 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.926 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.927 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:14.927 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601523+0000", 2024-04-26T03:39:14.927 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.0189999999999995e-06 2024-04-26T03:39:14.927 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.927 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.927 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:14.927 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601589+0000", 2024-04-26T03:39:14.927 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.5924000000000001e-05 2024-04-26T03:39:14.927 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.927 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.927 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:14.927 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601598+0000", 2024-04-26T03:39:14.927 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.6940000000000001e-06 2024-04-26T03:39:14.927 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.927 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.928 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:14.928 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.608417+0000", 2024-04-26T03:39:14.928 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.006819121 2024-04-26T03:39:14.928 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.928 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.928 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:14.928 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.608455+0000", 2024-04-26T03:39:14.928 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.7753000000000003e-05 2024-04-26T03:39:14.928 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:14.928 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:14.928 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:14.928 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.928 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.929 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.bs3 35/32 ECSubWrite(tid=22, reqid=client.4556.0:133, at_version=35'9, trim_to=0'0, roll_forward_to=35'7))", 2024-04-26T03:39:14.929 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.610898+0000", 2024-04-26T03:39:14.929 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 23.283808208, 2024-04-26T03:39:14.929 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0048565170000000003, 2024-04-26T03:39:14.929 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:14.929 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:14.929 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:14.929 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:14.929 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.929 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:14.929 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.610898+0000", 2024-04-26T03:39:14.929 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:14.929 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.929 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.929 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:14.930 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.610898+0000", 2024-04-26T03:39:14.930 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:14.930 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.930 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.930 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:14.930 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.610899+0000", 2024-04-26T03:39:14.930 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.9500000000000001e-07 2024-04-26T03:39:14.930 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.930 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.930 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:14.930 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.611030+0000", 2024-04-26T03:39:14.930 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00013066099999999999 2024-04-26T03:39:14.930 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.931 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.931 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:14.931 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.611046+0000", 2024-04-26T03:39:14.931 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.6056e-05 2024-04-26T03:39:14.931 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.931 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.931 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:14.931 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.611053+0000", 2024-04-26T03:39:14.931 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.1999999999999997e-06 2024-04-26T03:39:14.931 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.931 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.931 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:14.931 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.611119+0000", 2024-04-26T03:39:14.931 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.5839000000000004e-05 2024-04-26T03:39:14.932 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.932 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.932 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:14.932 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.611136+0000", 2024-04-26T03:39:14.932 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.7070000000000001e-05 2024-04-26T03:39:14.932 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.932 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.932 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:14.932 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.615722+0000", 2024-04-26T03:39:14.932 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.004586499 2024-04-26T03:39:14.932 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.932 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.932 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:14.933 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.615755+0000", 2024-04-26T03:39:14.933 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.2296999999999999e-05 2024-04-26T03:39:14.933 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:14.933 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:14.933 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:14.933 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.933 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.933 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.es6 35/32 ECSubWrite(tid=34, reqid=client.4556.0:177, at_version=35'9, trim_to=0'0, roll_forward_to=35'8))", 2024-04-26T03:39:14.933 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.731787+0000", 2024-04-26T03:39:14.933 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 23.162918928, 2024-04-26T03:39:14.933 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0075575629999999998, 2024-04-26T03:39:14.933 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:14.933 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:14.933 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:14.934 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:14.934 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.934 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:14.934 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.731787+0000", 2024-04-26T03:39:14.934 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:14.934 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.934 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.934 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:14.934 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.731787+0000", 2024-04-26T03:39:14.934 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:14.934 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.934 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.934 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:14.935 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.731788+0000", 2024-04-26T03:39:14.935 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1200000000000001e-06 2024-04-26T03:39:14.935 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.935 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.935 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:14.935 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.732056+0000", 2024-04-26T03:39:14.935 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00026767099999999998 2024-04-26T03:39:14.935 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.935 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.935 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:14.935 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.732072+0000", 2024-04-26T03:39:14.935 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.6242000000000001e-05 2024-04-26T03:39:14.935 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.936 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.936 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:14.936 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.732083+0000", 2024-04-26T03:39:14.936 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0135e-05 2024-04-26T03:39:14.936 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.936 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.936 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:14.936 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.732152+0000", 2024-04-26T03:39:14.936 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.9325000000000005e-05 2024-04-26T03:39:14.936 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.936 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.936 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:14.936 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.732163+0000", 2024-04-26T03:39:14.937 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0728e-05 2024-04-26T03:39:14.937 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.937 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.937 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:14.937 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.739313+0000", 2024-04-26T03:39:14.937 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0071506599999999997 2024-04-26T03:39:14.937 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.937 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.937 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:14.937 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.739345+0000", 2024-04-26T03:39:14.937 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.1681999999999999e-05 2024-04-26T03:39:14.937 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:14.937 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:14.938 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:14.938 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.938 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.938 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.es6 35/32 ECSubWrite(tid=35, reqid=client.4556.0:178, at_version=35'10, trim_to=0'0, roll_forward_to=35'8))", 2024-04-26T03:39:14.938 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.736069+0000", 2024-04-26T03:39:14.938 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 23.15863731, 2024-04-26T03:39:14.938 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0040055810000000002, 2024-04-26T03:39:14.938 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:14.938 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:14.938 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:14.938 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:14.938 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.939 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:14.939 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.736069+0000", 2024-04-26T03:39:14.939 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:14.939 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.939 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.939 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:14.939 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.736069+0000", 2024-04-26T03:39:14.939 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:14.939 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.939 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.939 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:14.939 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.736070+0000", 2024-04-26T03:39:14.939 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.4799999999999998e-07 2024-04-26T03:39:14.939 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.939 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.940 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:14.940 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.736255+0000", 2024-04-26T03:39:14.940 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00018521999999999999 2024-04-26T03:39:14.940 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.940 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.940 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:14.940 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.736266+0000", 2024-04-26T03:39:14.940 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0886999999999999e-05 2024-04-26T03:39:14.940 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.940 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.940 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:14.940 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.736271+0000", 2024-04-26T03:39:14.940 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.1490000000000002e-06 2024-04-26T03:39:14.940 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.940 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.941 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:14.941 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.736970+0000", 2024-04-26T03:39:14.941 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00069932999999999996 2024-04-26T03:39:14.941 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.941 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.941 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:14.941 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.736982+0000", 2024-04-26T03:39:14.941 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1642e-05 2024-04-26T03:39:14.941 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.941 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.941 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:14.941 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.740045+0000", 2024-04-26T03:39:14.941 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0030632720000000001 2024-04-26T03:39:14.941 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.941 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.942 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:14.942 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.740075+0000", 2024-04-26T03:39:14.942 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.9533e-05 2024-04-26T03:39:14.942 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:14.942 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:14.942 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:14.942 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.942 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.942 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.es6 35/32 ECSubWrite(tid=39, reqid=client.4556.0:207, at_version=35'12, trim_to=0'0, roll_forward_to=35'11))", 2024-04-26T03:39:14.942 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.807524+0000", 2024-04-26T03:39:14.942 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 23.087182502000001, 2024-04-26T03:39:14.942 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0081817109999999995, 2024-04-26T03:39:14.942 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:14.942 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:14.943 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:14.943 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:14.943 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.943 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:14.943 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.807524+0000", 2024-04-26T03:39:14.943 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:14.943 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.943 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.943 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:14.943 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.807524+0000", 2024-04-26T03:39:14.944 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:14.944 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.944 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.944 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:14.944 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.807525+0000", 2024-04-26T03:39:14.944 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.6700000000000002e-07 2024-04-26T03:39:14.944 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.944 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.944 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:14.944 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.808020+0000", 2024-04-26T03:39:14.944 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00049521799999999996 2024-04-26T03:39:14.944 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.944 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.945 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:14.945 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.808031+0000", 2024-04-26T03:39:14.945 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0862999999999999e-05 2024-04-26T03:39:14.945 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.945 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.945 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:14.945 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.808037+0000", 2024-04-26T03:39:14.945 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.438e-06 2024-04-26T03:39:14.945 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.945 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.945 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:14.945 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.808102+0000", 2024-04-26T03:39:14.945 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.4388999999999996e-05 2024-04-26T03:39:14.945 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.946 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.946 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:14.946 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.808114+0000", 2024-04-26T03:39:14.946 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.2211e-05 2024-04-26T03:39:14.946 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.946 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.946 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:14.946 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.815665+0000", 2024-04-26T03:39:14.946 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0075510259999999997 2024-04-26T03:39:14.946 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.946 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.946 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:14.946 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.815706+0000", 2024-04-26T03:39:14.947 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.0698999999999998e-05 2024-04-26T03:39:14.947 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:14.947 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:14.947 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:14.947 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.947 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.947 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.es6 35/32 ECSubWrite(tid=40, reqid=client.4556.0:208, at_version=35'13, trim_to=0'0, roll_forward_to=35'11))", 2024-04-26T03:39:14.947 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.811551+0000", 2024-04-26T03:39:14.947 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 23.083155509000001, 2024-04-26T03:39:14.947 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0045656179999999996, 2024-04-26T03:39:14.947 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:14.947 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:14.947 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:14.947 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:14.947 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.948 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:14.948 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.811551+0000", 2024-04-26T03:39:14.948 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:14.948 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.948 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.948 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:14.948 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.811551+0000", 2024-04-26T03:39:14.948 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:14.948 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.948 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.948 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:14.948 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.811551+0000", 2024-04-26T03:39:14.948 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.3900000000000005e-07 2024-04-26T03:39:14.948 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.948 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.949 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:14.949 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.811779+0000", 2024-04-26T03:39:14.949 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000227541 2024-04-26T03:39:14.949 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.949 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.949 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:14.949 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.811790+0000", 2024-04-26T03:39:14.949 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1076e-05 2024-04-26T03:39:14.949 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.949 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.949 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:14.949 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.811796+0000", 2024-04-26T03:39:14.949 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.643e-06 2024-04-26T03:39:14.949 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.949 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.950 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:14.950 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.813542+0000", 2024-04-26T03:39:14.950 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0017465969999999999 2024-04-26T03:39:14.950 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.950 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.950 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:14.950 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.813549+0000", 2024-04-26T03:39:14.950 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.5359999999999998e-06 2024-04-26T03:39:14.950 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.950 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.950 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:14.950 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.816088+0000", 2024-04-26T03:39:14.950 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0025395750000000001 2024-04-26T03:39:14.950 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.951 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.951 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:14.951 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.816116+0000", 2024-04-26T03:39:14.951 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.8110999999999999e-05 2024-04-26T03:39:14.951 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:14.951 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:14.951 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:14.951 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.951 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.951 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.es6 43/32 ECSubWrite(tid=48, reqid=client.4556.0:301, at_version=43'16, trim_to=0'0, roll_forward_to=35'14))", 2024-04-26T03:39:14.951 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:59.531146+0000", 2024-04-26T03:39:14.951 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 15.363560219, 2024-04-26T03:39:14.951 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.004955175, 2024-04-26T03:39:14.951 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:14.952 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:14.952 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:14.952 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:14.952 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.952 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:14.952 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.531146+0000", 2024-04-26T03:39:14.952 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:14.952 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.952 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.952 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:14.952 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.531146+0000", 2024-04-26T03:39:14.952 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:14.952 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.952 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.952 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:14.953 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.531146+0000", 2024-04-26T03:39:14.953 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.49e-07 2024-04-26T03:39:14.953 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.953 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.953 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:14.953 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.531483+0000", 2024-04-26T03:39:14.953 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00033656099999999998 2024-04-26T03:39:14.953 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.953 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.953 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:14.953 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.531497+0000", 2024-04-26T03:39:14.953 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.378e-05 2024-04-26T03:39:14.953 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.953 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.954 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:14.954 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.531506+0000", 2024-04-26T03:39:14.954 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.2159999999999995e-06 2024-04-26T03:39:14.954 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.954 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.954 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:14.954 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.531587+0000", 2024-04-26T03:39:14.954 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.1180999999999994e-05 2024-04-26T03:39:14.954 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.954 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.954 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:14.954 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.531599+0000", 2024-04-26T03:39:14.954 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.2169e-05 2024-04-26T03:39:14.954 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.955 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.955 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:14.955 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.536067+0000", 2024-04-26T03:39:14.955 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0044680099999999997 2024-04-26T03:39:14.955 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.955 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.955 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:14.955 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.536101+0000", 2024-04-26T03:39:14.955 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.4109000000000002e-05 2024-04-26T03:39:14.955 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:14.955 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:14.955 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:14.955 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.956 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.956 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.es6 43/32 ECSubWrite(tid=49, reqid=client.4556.0:302, at_version=43'17, trim_to=0'0, roll_forward_to=35'14))", 2024-04-26T03:39:14.956 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:59.537150+0000", 2024-04-26T03:39:14.956 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 15.357556519999999, 2024-04-26T03:39:14.956 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0058389130000000003, 2024-04-26T03:39:14.956 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:14.956 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:14.956 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:14.956 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:14.956 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.956 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:14.956 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.537150+0000", 2024-04-26T03:39:14.956 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:14.957 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.957 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.957 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:14.957 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.537150+0000", 2024-04-26T03:39:14.957 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:14.957 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.957 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.957 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:14.957 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.537151+0000", 2024-04-26T03:39:14.957 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.92e-07 2024-04-26T03:39:14.957 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.957 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.957 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:14.957 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.537707+0000", 2024-04-26T03:39:14.958 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00055628199999999996 2024-04-26T03:39:14.958 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.958 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.958 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:14.958 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.537720+0000", 2024-04-26T03:39:14.958 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.3434999999999999e-05 2024-04-26T03:39:14.958 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.958 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.958 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:14.958 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.537728+0000", 2024-04-26T03:39:14.958 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.8290000000000008e-06 2024-04-26T03:39:14.958 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.958 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.959 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:14.959 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.537787+0000", 2024-04-26T03:39:14.959 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.9249000000000001e-05 2024-04-26T03:39:14.959 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.959 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.959 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:14.959 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.537798+0000", 2024-04-26T03:39:14.959 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0997e-05 2024-04-26T03:39:14.959 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.959 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.959 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:14.959 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.542961+0000", 2024-04-26T03:39:14.959 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0051625320000000001 2024-04-26T03:39:14.959 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.960 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.960 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:14.960 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.542989+0000", 2024-04-26T03:39:14.960 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.7797000000000002e-05 2024-04-26T03:39:14.960 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:14.960 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:14.960 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:14.960 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.960 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.960 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.cs5 45/32 ECSubWrite(tid=53, reqid=client.4556.0:338, at_version=45'21, trim_to=0'0, roll_forward_to=42'19))", 2024-04-26T03:39:14.960 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:01.572023+0000", 2024-04-26T03:39:14.960 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 13.322683427999999, 2024-04-26T03:39:14.960 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0057687980000000003, 2024-04-26T03:39:14.961 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:14.961 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:14.961 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:14.961 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:14.961 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.961 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:14.961 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.572023+0000", 2024-04-26T03:39:14.961 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:14.961 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.961 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.961 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:14.961 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.572023+0000", 2024-04-26T03:39:14.961 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:14.961 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.962 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.962 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:14.962 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.572023+0000", 2024-04-26T03:39:14.962 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.8599999999999999e-07 2024-04-26T03:39:14.962 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.962 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.962 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:14.962 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.572451+0000", 2024-04-26T03:39:14.962 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00042800899999999997 2024-04-26T03:39:14.962 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.962 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.962 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:14.962 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.572468+0000", 2024-04-26T03:39:14.963 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.6991999999999999e-05 2024-04-26T03:39:14.963 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.963 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.963 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:14.963 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.572480+0000", 2024-04-26T03:39:14.963 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1338000000000001e-05 2024-04-26T03:39:14.963 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.963 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.963 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:14.963 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.572569+0000", 2024-04-26T03:39:14.963 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.9571e-05 2024-04-26T03:39:14.963 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.963 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.963 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:14.963 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.572582+0000", 2024-04-26T03:39:14.964 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.295e-05 2024-04-26T03:39:14.964 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.964 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.964 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:14.964 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.577759+0000", 2024-04-26T03:39:14.964 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0051768300000000003 2024-04-26T03:39:14.964 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.964 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.964 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:14.964 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.577792+0000", 2024-04-26T03:39:14.964 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.2722000000000003e-05 2024-04-26T03:39:14.964 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:14.964 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:14.964 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:14.965 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.965 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.965 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.cs5 45/32 ECSubWrite(tid=56, reqid=client.4556.0:339, at_version=45'22, trim_to=0'0, roll_forward_to=42'19))", 2024-04-26T03:39:14.965 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:01.578382+0000", 2024-04-26T03:39:14.965 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 13.316324459000001, 2024-04-26T03:39:14.965 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0044307010000000004, 2024-04-26T03:39:14.965 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:14.965 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:14.965 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:14.965 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:14.965 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.965 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:14.965 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.578382+0000", 2024-04-26T03:39:14.965 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:14.965 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.966 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.966 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:14.966 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.578382+0000", 2024-04-26T03:39:14.966 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:14.966 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.966 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.966 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:14.966 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.578383+0000", 2024-04-26T03:39:14.966 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.1299999999999999e-07 2024-04-26T03:39:14.966 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.966 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.966 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:14.966 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.578813+0000", 2024-04-26T03:39:14.967 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00043056000000000002 2024-04-26T03:39:14.967 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.967 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.967 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:14.967 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.578825+0000", 2024-04-26T03:39:14.967 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.167e-05 2024-04-26T03:39:14.967 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.967 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.967 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:14.967 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.578831+0000", 2024-04-26T03:39:14.967 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.088e-06 2024-04-26T03:39:14.967 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.967 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.967 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:14.967 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.578892+0000", 2024-04-26T03:39:14.968 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.1495000000000007e-05 2024-04-26T03:39:14.968 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.968 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.968 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:14.968 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.578899+0000", 2024-04-26T03:39:14.968 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.4019999999999999e-06 2024-04-26T03:39:14.968 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.968 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.968 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:14.968 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.582776+0000", 2024-04-26T03:39:14.968 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0038771360000000002 2024-04-26T03:39:14.968 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.968 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.968 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:14.968 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.582813+0000", 2024-04-26T03:39:14.969 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.6536999999999999e-05 2024-04-26T03:39:14.969 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:14.969 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:14.969 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:14.969 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.969 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.969 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.es6 45/32 ECSubWrite(tid=62, reqid=client.4556.0:349, at_version=45'21, trim_to=0'0, roll_forward_to=43'19))", 2024-04-26T03:39:14.969 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:01.632894+0000", 2024-04-26T03:39:14.969 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 13.261812773999999, 2024-04-26T03:39:14.969 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0051349860000000002, 2024-04-26T03:39:14.969 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:14.969 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:14.969 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:14.969 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:14.970 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.970 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:14.970 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.632894+0000", 2024-04-26T03:39:14.970 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:14.970 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.970 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.970 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:14.970 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.632894+0000", 2024-04-26T03:39:14.970 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:14.970 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.970 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.970 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:14.970 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.632894+0000", 2024-04-26T03:39:14.970 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.6100000000000002e-07 2024-04-26T03:39:14.971 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.971 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.971 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:14.971 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.633352+0000", 2024-04-26T03:39:14.971 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00045790799999999998 2024-04-26T03:39:14.971 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.971 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.971 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:14.971 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.633364+0000", 2024-04-26T03:39:14.971 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.2157999999999999e-05 2024-04-26T03:39:14.971 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.971 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.971 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:14.971 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.633372+0000", 2024-04-26T03:39:14.971 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.9589999999999995e-06 2024-04-26T03:39:14.972 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.972 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.972 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:14.972 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.633431+0000", 2024-04-26T03:39:14.972 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.8690999999999998e-05 2024-04-26T03:39:14.972 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.972 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.972 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:14.972 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.633437+0000", 2024-04-26T03:39:14.972 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.798e-06 2024-04-26T03:39:14.972 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.972 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.972 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:14.972 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.637987+0000", 2024-04-26T03:39:14.973 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0045496130000000001 2024-04-26T03:39:14.973 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.973 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.973 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:14.973 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.638029+0000", 2024-04-26T03:39:14.973 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.1498000000000001e-05 2024-04-26T03:39:14.973 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:14.973 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:14.973 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:14.973 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.973 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.973 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.es6 45/32 ECSubWrite(tid=63, reqid=client.4556.0:350, at_version=45'22, trim_to=0'0, roll_forward_to=43'19))", 2024-04-26T03:39:14.973 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:01.637512+0000", 2024-04-26T03:39:14.973 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 13.257194466, 2024-04-26T03:39:14.974 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0059848239999999997, 2024-04-26T03:39:14.974 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:14.974 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:14.974 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:14.974 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:14.974 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.974 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:14.974 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.637512+0000", 2024-04-26T03:39:14.974 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:14.974 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.974 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.974 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:14.974 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.637512+0000", 2024-04-26T03:39:14.974 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:14.975 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.975 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.975 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:14.975 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.637513+0000", 2024-04-26T03:39:14.975 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.001e-06 2024-04-26T03:39:14.975 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.975 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.975 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:14.975 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.638154+0000", 2024-04-26T03:39:14.975 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00064122099999999998 2024-04-26T03:39:14.975 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.975 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.975 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:14.976 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.638164+0000", 2024-04-26T03:39:14.976 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0325e-05 2024-04-26T03:39:14.976 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.976 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.976 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:14.976 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.638170+0000", 2024-04-26T03:39:14.976 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.6980000000000003e-06 2024-04-26T03:39:14.976 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.976 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.976 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:14.976 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.638223+0000", 2024-04-26T03:39:14.976 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.2667000000000002e-05 2024-04-26T03:39:14.976 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.976 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.977 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:14.977 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.638232+0000", 2024-04-26T03:39:14.977 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.9439999999999994e-06 2024-04-26T03:39:14.977 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.977 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.977 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:14.977 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.643458+0000", 2024-04-26T03:39:14.977 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0052263850000000001 2024-04-26T03:39:14.977 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.977 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.977 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:14.977 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.643497+0000", 2024-04-26T03:39:14.977 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.8583e-05 2024-04-26T03:39:14.977 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:14.977 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:14.978 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:14.978 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.978 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.978 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.bs3 45/32 ECSubWrite(tid=68, reqid=client.4556.0:361, at_version=45'31, trim_to=0'0, roll_forward_to=43'30))", 2024-04-26T03:39:14.978 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:01.715320+0000", 2024-04-26T03:39:14.978 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 13.179385924, 2024-04-26T03:39:14.978 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0047019460000000003, 2024-04-26T03:39:14.978 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:14.978 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:14.978 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:14.978 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:14.978 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.978 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:14.978 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.715320+0000", 2024-04-26T03:39:14.979 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:14.979 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.979 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.979 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:14.979 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.715320+0000", 2024-04-26T03:39:14.979 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:14.979 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.979 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.979 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:14.979 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.715322+0000", 2024-04-26T03:39:14.979 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.248e-06 2024-04-26T03:39:14.979 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.979 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.979 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:14.979 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.715385+0000", 2024-04-26T03:39:14.980 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.3443999999999998e-05 2024-04-26T03:39:14.980 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.980 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.980 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:14.980 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.715406+0000", 2024-04-26T03:39:14.980 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.0573000000000001e-05 2024-04-26T03:39:14.980 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.980 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.980 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:14.980 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.715418+0000", 2024-04-26T03:39:14.980 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1960000000000001e-05 2024-04-26T03:39:14.980 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.980 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.980 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:14.981 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.715525+0000", 2024-04-26T03:39:14.981 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000107694 2024-04-26T03:39:14.981 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.981 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.981 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:14.981 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.715537+0000", 2024-04-26T03:39:14.981 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1749999999999999e-05 2024-04-26T03:39:14.981 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.981 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.981 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:14.981 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.719962+0000", 2024-04-26T03:39:14.981 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0044252349999999996 2024-04-26T03:39:14.981 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.981 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.981 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:14.982 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.720022+0000", 2024-04-26T03:39:14.982 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.0041999999999999e-05 2024-04-26T03:39:14.982 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:14.982 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:14.982 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:14.982 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.982 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.982 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.bs3 45/32 ECSubWrite(tid=69, reqid=client.4556.0:362, at_version=45'32, trim_to=0'0, roll_forward_to=43'30))", 2024-04-26T03:39:14.982 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:01.719989+0000", 2024-04-26T03:39:14.982 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 13.174717422000001, 2024-04-26T03:39:14.982 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0041764519999999998, 2024-04-26T03:39:14.982 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:14.982 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:14.982 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:14.983 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:14.983 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.983 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:14.983 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.719989+0000", 2024-04-26T03:39:14.983 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:14.983 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.983 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.983 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:14.983 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.719989+0000", 2024-04-26T03:39:14.983 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:14.983 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.983 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.983 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:14.983 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.719990+0000", 2024-04-26T03:39:14.983 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.8200000000000008e-07 2024-04-26T03:39:14.984 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.984 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.984 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:14.984 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.720044+0000", 2024-04-26T03:39:14.984 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.4228000000000003e-05 2024-04-26T03:39:14.984 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.984 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.984 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:14.984 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.720057+0000", 2024-04-26T03:39:14.984 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.2717e-05 2024-04-26T03:39:14.984 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.984 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.984 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:14.984 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.720071+0000", 2024-04-26T03:39:14.984 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.4514e-05 2024-04-26T03:39:14.985 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.985 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.985 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:14.985 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.720146+0000", 2024-04-26T03:39:14.985 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.4711999999999999e-05 2024-04-26T03:39:14.985 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.985 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.985 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:14.985 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.720156+0000", 2024-04-26T03:39:14.985 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.6069999999999997e-06 2024-04-26T03:39:14.985 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.985 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.985 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:14.985 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.724134+0000", 2024-04-26T03:39:14.986 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0039785719999999997 2024-04-26T03:39:14.986 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.986 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.986 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:14.986 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.724165+0000", 2024-04-26T03:39:14.986 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.112e-05 2024-04-26T03:39:14.986 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:14.986 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:14.986 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:14.986 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.986 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.986 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.bs3 45/32 ECSubWrite(tid=70, reqid=client.4556.0:363, at_version=45'33, trim_to=0'0, roll_forward_to=43'30))", 2024-04-26T03:39:14.986 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:01.724558+0000", 2024-04-26T03:39:14.986 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 13.170148435, 2024-04-26T03:39:14.987 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0037853320000000002, 2024-04-26T03:39:14.987 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:14.987 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:14.987 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:14.987 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:14.987 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.987 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:14.987 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.724558+0000", 2024-04-26T03:39:14.987 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:14.987 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.987 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.987 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:14.987 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.724558+0000", 2024-04-26T03:39:14.987 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:14.987 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.988 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.988 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:14.988 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.724559+0000", 2024-04-26T03:39:14.988 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.3200000000000003e-07 2024-04-26T03:39:14.988 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.988 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.988 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:14.988 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.724599+0000", 2024-04-26T03:39:14.988 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.9891999999999997e-05 2024-04-26T03:39:14.988 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.988 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.988 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:14.988 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.724614+0000", 2024-04-26T03:39:14.988 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.5786000000000001e-05 2024-04-26T03:39:14.989 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.989 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.989 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:14.989 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.724623+0000", 2024-04-26T03:39:14.989 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.7539999999999995e-06 2024-04-26T03:39:14.989 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.989 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.989 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:14.989 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.724695+0000", 2024-04-26T03:39:14.989 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.1866999999999995e-05 2024-04-26T03:39:14.989 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.989 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.989 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:14.989 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.724704+0000", 2024-04-26T03:39:14.989 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.3449999999999995e-06 2024-04-26T03:39:14.990 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.990 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.990 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:14.990 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.728312+0000", 2024-04-26T03:39:14.990 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0036073469999999999 2024-04-26T03:39:14.990 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:14.990 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:14.990 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:14.990 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.728343+0000", 2024-04-26T03:39:14.990 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.1408999999999997e-05 2024-04-26T03:39:14.990 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:14.990 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:14.990 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:14.990 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:14.991 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:14.991 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:39:14.992 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-26T03:39:14.993 INFO:tasks.ceph.osd.4.smithi028.stderr:2024-04-26T03:39:14.991+0000 7fd9511b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31963) UID: 0 2024-04-26T03:39:15.079 INFO:teuthology.orchestra.run.smithi028.stdout:osd.15: {} 2024-04-26T03:39:15.079 INFO:teuthology.orchestra.run.smithi028.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T03:39:15.093 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:39:15.092+0000 7fba0d2ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31961) UID: 0 2024-04-26T03:39:15.112 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-26T03:39:15.112 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-04-26T03:39:15.112 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-04-26T03:39:15.112 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-26T03:39:15.121 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-26T03:39:15.194 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-04-26T03:39:15.192+0000 7fe9223c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31954) UID: 0 2024-04-26T03:39:15.229 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-26T03:39:15.229 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-04-26T03:39:15.229 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-04-26T03:39:15.230 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-04-26T03:39:15.230 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-26T03:39:15.238 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-26T03:39:15.323 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:39:15.321+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:39:15.387 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-26T03:39:15.387 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-04-26T03:39:15.387 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-04-26T03:39:15.387 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-04-26T03:39:15.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.388 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:13 3.0s0 3:0a8403ef:::smithi09235267-5:head [append 0~81920,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:15.388 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.484308+0000", 2024-04-26T03:39:15.388 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.900446899999999, 2024-04-26T03:39:15.388 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017722486999999999, 2024-04-26T03:39:15.388 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:15.388 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:15.388 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:15.388 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:15.388 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:15.388 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:15.388 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 13 2024-04-26T03:39:15.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.388 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:15.388 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.389 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:15.389 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484308+0000", 2024-04-26T03:39:15.389 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.389 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.389 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:15.389 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484308+0000", 2024-04-26T03:39:15.389 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.389 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.389 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:15.389 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484311+0000", 2024-04-26T03:39:15.389 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8890000000000002e-06 2024-04-26T03:39:15.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.389 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.390 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:15.390 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484792+0000", 2024-04-26T03:39:15.390 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00048166600000000001 2024-04-26T03:39:15.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.390 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:15.390 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484806+0000", 2024-04-26T03:39:15.390 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3709999999999999e-05 2024-04-26T03:39:15.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.390 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:15.390 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484816+0000", 2024-04-26T03:39:15.390 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7000000000000003e-06 2024-04-26T03:39:15.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.391 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:15.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484975+0000", 2024-04-26T03:39:15.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015943199999999999 2024-04-26T03:39:15.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.391 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:15.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.485147+0000", 2024-04-26T03:39:15.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000171888 2024-04-26T03:39:15.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.391 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:15.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.486224+0000", 2024-04-26T03:39:15.392 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010766720000000001 2024-04-26T03:39:15.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.392 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.392 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:15.392 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.489541+0000", 2024-04-26T03:39:15.392 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033176210000000002 2024-04-26T03:39:15.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.392 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.392 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:39:15.392 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.501934+0000", 2024-04-26T03:39:15.392 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012392633 2024-04-26T03:39:15.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.392 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.392 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:15.392 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.502030+0000", 2024-04-26T03:39:15.393 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6275999999999998e-05 2024-04-26T03:39:15.393 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.393 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:15.393 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.393 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.393 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.393 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:14 3.0s0 3:0a8403ef:::smithi09235267-5:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:15.393 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.484901+0000", 2024-04-26T03:39:15.393 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.899853792999998, 2024-04-26T03:39:15.393 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017354086000000001, 2024-04-26T03:39:15.393 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:15.393 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:15.394 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:15.394 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:15.394 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:15.394 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:15.394 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 14 2024-04-26T03:39:15.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.394 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:15.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.394 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:15.394 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484901+0000", 2024-04-26T03:39:15.394 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.394 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:15.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484901+0000", 2024-04-26T03:39:15.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:15.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484902+0000", 2024-04-26T03:39:15.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1069999999999999e-06 2024-04-26T03:39:15.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:15.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484909+0000", 2024-04-26T03:39:15.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.323e-06 2024-04-26T03:39:15.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:15.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484919+0000", 2024-04-26T03:39:15.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0069e-05 2024-04-26T03:39:15.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:15.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484930+0000", 2024-04-26T03:39:15.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1092e-05 2024-04-26T03:39:15.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:15.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.487836+0000", 2024-04-26T03:39:15.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0029053569999999999 2024-04-26T03:39:15.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.397 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:15.397 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.487949+0000", 2024-04-26T03:39:15.397 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000112822 2024-04-26T03:39:15.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.397 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.397 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:15.397 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.488660+0000", 2024-04-26T03:39:15.397 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00071102899999999996 2024-04-26T03:39:15.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.397 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.397 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:15.397 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.490973+0000", 2024-04-26T03:39:15.397 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023132510000000001 2024-04-26T03:39:15.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.397 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.398 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:39:15.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.502166+0000", 2024-04-26T03:39:15.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011193639999999999 2024-04-26T03:39:15.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.398 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:15.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.502255+0000", 2024-04-26T03:39:15.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8395999999999999e-05 2024-04-26T03:39:15.398 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.398 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:15.398 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.399 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:15 3.0s0 3:0a8403ef:::smithi09235267-5:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:15.399 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.484947+0000", 2024-04-26T03:39:15.399 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.899808020000002, 2024-04-26T03:39:15.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024468274000000002, 2024-04-26T03:39:15.399 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:15.400 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:15.400 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-04-26T03:39:15.400 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:15.400 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:15.400 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:15.400 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15 2024-04-26T03:39:15.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.400 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:15.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:15.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484947+0000", 2024-04-26T03:39:15.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.401 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:15.401 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484947+0000", 2024-04-26T03:39:15.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.401 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.401 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:15.401 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484947+0000", 2024-04-26T03:39:15.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0999999999999997e-07 2024-04-26T03:39:15.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.401 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.401 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:15.401 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484952+0000", 2024-04-26T03:39:15.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.549e-06 2024-04-26T03:39:15.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.402 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:15.402 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484962+0000", 2024-04-26T03:39:15.402 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0064999999999999e-05 2024-04-26T03:39:15.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.402 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:15.402 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484972+0000", 2024-04-26T03:39:15.402 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7529999999999996e-06 2024-04-26T03:39:15.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.402 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:15.402 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.489714+0000", 2024-04-26T03:39:15.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0047418649999999996 2024-04-26T03:39:15.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:15.403 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.489836+0000", 2024-04-26T03:39:15.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012253399999999999 2024-04-26T03:39:15.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:15.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.502300+0000", 2024-04-26T03:39:15.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012464098 2024-04-26T03:39:15.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:15.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.502390+0000", 2024-04-26T03:39:15.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9954e-05 2024-04-26T03:39:15.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:15.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.509415+0000", 2024-04-26T03:39:15.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0070245459999999996 2024-04-26T03:39:15.405 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.405 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:15.405 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.405 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:40 3.0s0 3:0e9b8628:::smithi09235267-14:head [append 0~229376,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:15.405 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.495863+0000", 2024-04-26T03:39:15.405 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.888891526999998, 2024-04-26T03:39:15.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013720777999999999, 2024-04-26T03:39:15.405 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:15.405 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:15.405 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:15.406 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:15.406 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:15.406 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:15.406 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 40 2024-04-26T03:39:15.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.406 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:15.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.406 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:15.406 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.495863+0000", 2024-04-26T03:39:15.406 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.406 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:15.407 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.495863+0000", 2024-04-26T03:39:15.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.407 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:15.407 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.495865+0000", 2024-04-26T03:39:15.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4090000000000001e-06 2024-04-26T03:39:15.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.407 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:15.407 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496307+0000", 2024-04-26T03:39:15.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00044159800000000002 2024-04-26T03:39:15.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:15.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496322+0000", 2024-04-26T03:39:15.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4935e-05 2024-04-26T03:39:15.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:15.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496338+0000", 2024-04-26T03:39:15.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6155e-05 2024-04-26T03:39:15.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:15.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496408+0000", 2024-04-26T03:39:15.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0183000000000002e-05 2024-04-26T03:39:15.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:15.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496716+0000", 2024-04-26T03:39:15.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000308024 2024-04-26T03:39:15.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:15.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.497681+0000", 2024-04-26T03:39:15.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00096483300000000001 2024-04-26T03:39:15.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:15.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.505243+0000", 2024-04-26T03:39:15.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0075613329999999999 2024-04-26T03:39:15.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.410 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:39:15.410 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.509516+0000", 2024-04-26T03:39:15.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0042732480000000003 2024-04-26T03:39:15.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.410 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:15.410 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.509584+0000", 2024-04-26T03:39:15.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8059999999999996e-05 2024-04-26T03:39:15.410 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.410 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:15.410 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.411 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:41 3.0s0 3:0e9b8628:::smithi09235267-14:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:15.411 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.496478+0000", 2024-04-26T03:39:15.411 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.888276266999998, 2024-04-26T03:39:15.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018225094000000001, 2024-04-26T03:39:15.411 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:15.411 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:15.411 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:15.411 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:15.411 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:15.411 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:15.412 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 41 2024-04-26T03:39:15.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.412 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:15.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.412 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:15.412 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496478+0000", 2024-04-26T03:39:15.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.412 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:15.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496478+0000", 2024-04-26T03:39:15.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:15.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496479+0000", 2024-04-26T03:39:15.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7700000000000003e-07 2024-04-26T03:39:15.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:15.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496563+0000", 2024-04-26T03:39:15.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3778999999999999e-05 2024-04-26T03:39:15.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:15.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496574+0000", 2024-04-26T03:39:15.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1144999999999999e-05 2024-04-26T03:39:15.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:15.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496586+0000", 2024-04-26T03:39:15.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1457999999999999e-05 2024-04-26T03:39:15.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:15.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.503975+0000", 2024-04-26T03:39:15.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0073896480000000004 2024-04-26T03:39:15.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.415 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:15.415 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504089+0000", 2024-04-26T03:39:15.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011359299999999999 2024-04-26T03:39:15.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.415 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:15.415 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504968+0000", 2024-04-26T03:39:15.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00087954500000000002 2024-04-26T03:39:15.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.415 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:15.415 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.508613+0000", 2024-04-26T03:39:15.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0036446500000000001 2024-04-26T03:39:15.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.416 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:39:15.416 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.514613+0000", 2024-04-26T03:39:15.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0060000000000000001 2024-04-26T03:39:15.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.416 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:15.416 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.514703+0000", 2024-04-26T03:39:15.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0399e-05 2024-04-26T03:39:15.416 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.416 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:15.416 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.417 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:42 3.0s0 3:0e9b8628:::smithi09235267-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:15.417 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.496943+0000", 2024-04-26T03:39:15.417 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.887811491000001, 2024-04-26T03:39:15.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021178255999999999, 2024-04-26T03:39:15.417 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:15.417 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:15.417 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-04-26T03:39:15.417 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:15.417 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:15.418 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:15.418 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 42 2024-04-26T03:39:15.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.418 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:15.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:15.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496943+0000", 2024-04-26T03:39:15.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:15.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496943+0000", 2024-04-26T03:39:15.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.419 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:15.419 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496944+0000", 2024-04-26T03:39:15.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9000000000000005e-07 2024-04-26T03:39:15.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.419 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:15.419 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496951+0000", 2024-04-26T03:39:15.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6470000000000001e-06 2024-04-26T03:39:15.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.419 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:15.420 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496962+0000", 2024-04-26T03:39:15.420 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1618e-05 2024-04-26T03:39:15.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.420 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:15.420 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496971+0000", 2024-04-26T03:39:15.420 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6170000000000007e-06 2024-04-26T03:39:15.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.420 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:15.420 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.505538+0000", 2024-04-26T03:39:15.420 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0085666530000000005 2024-04-26T03:39:15.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.421 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:15.421 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.505620+0000", 2024-04-26T03:39:15.421 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2669000000000004e-05 2024-04-26T03:39:15.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.421 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:15.421 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.515261+0000", 2024-04-26T03:39:15.421 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0096405210000000009 2024-04-26T03:39:15.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.421 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:15.421 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.515342+0000", 2024-04-26T03:39:15.421 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1642000000000002e-05 2024-04-26T03:39:15.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.422 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:15.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.518121+0000", 2024-04-26T03:39:15.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002778899 2024-04-26T03:39:15.422 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.422 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:15.422 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.422 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds3 35/32 ECSubWrite(tid=1, reqid=client.4556.0:44, at_version=35'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:39:15.422 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.504056+0000", 2024-04-26T03:39:15.423 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.880698403, 2024-04-26T03:39:15.423 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0085269190000000009, 2024-04-26T03:39:15.423 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:15.423 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:15.423 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:15.423 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:15.423 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.423 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:15.423 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504056+0000", 2024-04-26T03:39:15.423 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.423 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.423 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:15.424 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504056+0000", 2024-04-26T03:39:15.424 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.424 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:15.424 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504057+0000", 2024-04-26T03:39:15.424 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2600000000000001e-07 2024-04-26T03:39:15.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.424 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:15.424 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504078+0000", 2024-04-26T03:39:15.424 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1250000000000002e-05 2024-04-26T03:39:15.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.425 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:15.425 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504092+0000", 2024-04-26T03:39:15.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3370000000000001e-05 2024-04-26T03:39:15.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.425 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:15.425 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504102+0000", 2024-04-26T03:39:15.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0047000000000001e-05 2024-04-26T03:39:15.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.425 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:15.425 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504408+0000", 2024-04-26T03:39:15.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030663799999999997 2024-04-26T03:39:15.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.426 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:15.426 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504417+0000", 2024-04-26T03:39:15.426 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8400000000000001e-06 2024-04-26T03:39:15.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.426 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:15.426 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.512551+0000", 2024-04-26T03:39:15.426 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0081339870000000005 2024-04-26T03:39:15.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.426 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:15.426 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.512583+0000", 2024-04-26T03:39:15.427 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1961000000000001e-05 2024-04-26T03:39:15.427 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.427 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:15.427 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.427 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds3 35/32 ECSubWrite(tid=2, reqid=client.4556.0:52, at_version=35'3, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:39:15.427 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.506681+0000", 2024-04-26T03:39:15.427 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.878073220000001, 2024-04-26T03:39:15.427 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0070048899999999997, 2024-04-26T03:39:15.427 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:15.427 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:15.428 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:15.428 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:15.428 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.428 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:15.428 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.506681+0000", 2024-04-26T03:39:15.428 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.428 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.428 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:15.428 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.506681+0000", 2024-04-26T03:39:15.428 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.428 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:15.429 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.506682+0000", 2024-04-26T03:39:15.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.04e-06 2024-04-26T03:39:15.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:15.429 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.506853+0000", 2024-04-26T03:39:15.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017012599999999999 2024-04-26T03:39:15.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:15.429 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.506866+0000", 2024-04-26T03:39:15.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3244999999999999e-05 2024-04-26T03:39:15.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.430 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:15.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.506874+0000", 2024-04-26T03:39:15.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0229999999999992e-06 2024-04-26T03:39:15.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.430 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:15.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.508635+0000", 2024-04-26T03:39:15.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0017609380000000001 2024-04-26T03:39:15.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:15.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.508644+0000", 2024-04-26T03:39:15.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0569999999999999e-06 2024-04-26T03:39:15.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:15.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.513648+0000", 2024-04-26T03:39:15.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0050038920000000002 2024-04-26T03:39:15.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:15.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.513686+0000", 2024-04-26T03:39:15.432 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8569000000000003e-05 2024-04-26T03:39:15.432 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.432 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:15.432 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.432 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.as5 35/32 ECSubWrite(tid=4, reqid=client.4556.0:73, at_version=35'3, trim_to=0'0, roll_forward_to=34'2))", 2024-04-26T03:39:15.432 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.525773+0000", 2024-04-26T03:39:15.432 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.858982021999999, 2024-04-26T03:39:15.432 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0073138420000000001, 2024-04-26T03:39:15.432 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:15.432 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:15.432 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:15.432 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:15.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:15.433 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.525773+0000", 2024-04-26T03:39:15.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:15.433 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.525773+0000", 2024-04-26T03:39:15.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:15.433 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.525773+0000", 2024-04-26T03:39:15.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0699999999999996e-07 2024-04-26T03:39:15.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.434 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:15.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.525972+0000", 2024-04-26T03:39:15.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019857100000000001 2024-04-26T03:39:15.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.434 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:15.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.525984+0000", 2024-04-26T03:39:15.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2099999999999999e-05 2024-04-26T03:39:15.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.434 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:15.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.525990+0000", 2024-04-26T03:39:15.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7320000000000003e-06 2024-04-26T03:39:15.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:15.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.526063+0000", 2024-04-26T03:39:15.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3499999999999998e-05 2024-04-26T03:39:15.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:15.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.526074+0000", 2024-04-26T03:39:15.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0899e-05 2024-04-26T03:39:15.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:15.436 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.533045+0000", 2024-04-26T03:39:15.436 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0069703400000000002 2024-04-26T03:39:15.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.436 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:15.436 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.533086+0000", 2024-04-26T03:39:15.436 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1893e-05 2024-04-26T03:39:15.436 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.436 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:15.436 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.436 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.8s2 40/32 ECSubWrite(tid=178, reqid=client.4556.0:266, at_version=40'8, trim_to=0'0, roll_forward_to=35'6))", 2024-04-26T03:39:15.436 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:56.534993+0000", 2024-04-26T03:39:15.437 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.849762069000001, 2024-04-26T03:39:15.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0072588619999999996, 2024-04-26T03:39:15.437 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:15.437 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:15.437 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:15.437 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:15.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.437 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:15.437 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.534993+0000", 2024-04-26T03:39:15.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.437 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:15.437 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.534993+0000", 2024-04-26T03:39:15.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.438 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:15.438 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.534993+0000", 2024-04-26T03:39:15.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4799999999999998e-07 2024-04-26T03:39:15.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.438 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:15.438 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.535281+0000", 2024-04-26T03:39:15.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00028805200000000001 2024-04-26T03:39:15.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.438 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:15.439 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.535292+0000", 2024-04-26T03:39:15.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.045e-05 2024-04-26T03:39:15.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.439 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:15.439 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.535298+0000", 2024-04-26T03:39:15.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0959999999999997e-06 2024-04-26T03:39:15.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.439 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:15.439 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.535358+0000", 2024-04-26T03:39:15.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0448000000000001e-05 2024-04-26T03:39:15.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.440 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:15.440 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.535366+0000", 2024-04-26T03:39:15.440 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9419999999999995e-06 2024-04-26T03:39:15.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.440 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:15.440 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.542217+0000", 2024-04-26T03:39:15.440 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0068503519999999997 2024-04-26T03:39:15.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.440 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:15.440 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.542251+0000", 2024-04-26T03:39:15.440 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4873999999999998e-05 2024-04-26T03:39:15.440 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.440 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:15.441 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.441 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.441 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.18s2 48/44 ECSubWrite(tid=199, reqid=osd.6.0:199, at_version=48'28, trim_to=0'0, roll_forward_to=47'26))", 2024-04-26T03:39:15.441 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:04.554232+0000", 2024-04-26T03:39:15.441 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.830522216, 2024-04-26T03:39:15.441 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0076038520000000004, 2024-04-26T03:39:15.441 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:15.441 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:15.441 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:15.441 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:15.441 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.441 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:15.441 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.554232+0000", 2024-04-26T03:39:15.442 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.442 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:15.442 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.554232+0000", 2024-04-26T03:39:15.442 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.442 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:15.442 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.554233+0000", 2024-04-26T03:39:15.442 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.37e-07 2024-04-26T03:39:15.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.442 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:15.443 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.554263+0000", 2024-04-26T03:39:15.443 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0463999999999998e-05 2024-04-26T03:39:15.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.443 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.443 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:15.443 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.554274+0000", 2024-04-26T03:39:15.443 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1e-05 2024-04-26T03:39:15.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.443 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.443 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:15.443 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.554499+0000", 2024-04-26T03:39:15.443 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000224596 2024-04-26T03:39:15.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.444 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:15.444 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.555156+0000", 2024-04-26T03:39:15.444 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00065746000000000001 2024-04-26T03:39:15.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.444 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:15.444 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.555168+0000", 2024-04-26T03:39:15.444 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1248e-05 2024-04-26T03:39:15.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.444 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:15.444 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.561800+0000", 2024-04-26T03:39:15.445 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0066318460000000003 2024-04-26T03:39:15.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.445 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.445 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:15.445 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.561836+0000", 2024-04-26T03:39:15.445 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6701e-05 2024-04-26T03:39:15.445 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.445 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:15.445 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.445 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.445 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.bs4 48/32 ECSubWrite(tid=78, reqid=osd.4.0:78, at_version=48'40, trim_to=0'0, roll_forward_to=47'36))", 2024-04-26T03:39:15.445 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:04.557110+0000", 2024-04-26T03:39:15.445 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.827644204, 2024-04-26T03:39:15.446 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0084124439999999998, 2024-04-26T03:39:15.446 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:15.446 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:15.446 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:15.446 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:15.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.446 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:15.446 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.557110+0000", 2024-04-26T03:39:15.446 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.446 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:15.447 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.557110+0000", 2024-04-26T03:39:15.447 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.447 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:15.447 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.557111+0000", 2024-04-26T03:39:15.447 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7199999999999997e-07 2024-04-26T03:39:15.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.447 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:15.447 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.557136+0000", 2024-04-26T03:39:15.447 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4270999999999999e-05 2024-04-26T03:39:15.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.448 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:15.448 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.557146+0000", 2024-04-26T03:39:15.448 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0832999999999999e-05 2024-04-26T03:39:15.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.448 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:15.448 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.557155+0000", 2024-04-26T03:39:15.448 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3960000000000006e-06 2024-04-26T03:39:15.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.448 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:15.448 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.557237+0000", 2024-04-26T03:39:15.448 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2167000000000001e-05 2024-04-26T03:39:15.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.449 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:15.449 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.557248+0000", 2024-04-26T03:39:15.449 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1283e-05 2024-04-26T03:39:15.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.449 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:15.449 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.565491+0000", 2024-04-26T03:39:15.449 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0082427439999999998 2024-04-26T03:39:15.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.449 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:15.449 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.565523+0000", 2024-04-26T03:39:15.449 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1778e-05 2024-04-26T03:39:15.450 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.450 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:15.450 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.450 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.450 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.3s4 48/44 ECSubWrite(tid=60, reqid=osd.10.0:60, at_version=48'28, trim_to=0'0, roll_forward_to=47'26))", 2024-04-26T03:39:15.450 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:04.559452+0000", 2024-04-26T03:39:15.450 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.825302851, 2024-04-26T03:39:15.450 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0069494960000000003, 2024-04-26T03:39:15.450 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:15.450 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:15.450 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:15.450 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:15.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.451 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:15.451 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.559452+0000", 2024-04-26T03:39:15.451 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.451 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:15.451 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.559452+0000", 2024-04-26T03:39:15.451 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.451 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:15.451 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.559452+0000", 2024-04-26T03:39:15.451 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-04-26T03:39:15.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.452 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.452 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:15.452 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.559480+0000", 2024-04-26T03:39:15.452 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7826000000000001e-05 2024-04-26T03:39:15.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.452 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.452 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:15.452 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.559489+0000", 2024-04-26T03:39:15.452 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8510000000000005e-06 2024-04-26T03:39:15.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.452 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.452 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:15.452 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.559498+0000", 2024-04-26T03:39:15.453 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7730000000000005e-06 2024-04-26T03:39:15.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.453 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:15.453 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.560563+0000", 2024-04-26T03:39:15.453 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001065046 2024-04-26T03:39:15.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.453 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:15.453 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.560573+0000", 2024-04-26T03:39:15.453 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0049e-05 2024-04-26T03:39:15.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.453 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:15.453 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.566367+0000", 2024-04-26T03:39:15.454 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0057945849999999997 2024-04-26T03:39:15.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.454 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.454 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:15.454 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.566401+0000", 2024-04-26T03:39:15.454 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3958999999999998e-05 2024-04-26T03:39:15.454 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.454 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:15.454 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.454 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.454 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.6s6 49/44 ECSubWrite(tid=7, reqid=client.4556.0:431, at_version=49'6, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:39:15.454 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:04.690491+0000", 2024-04-26T03:39:15.454 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.694263276999999, 2024-04-26T03:39:15.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0073474559999999996, 2024-04-26T03:39:15.455 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:15.455 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:15.455 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:15.455 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:15.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.455 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:15.455 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.690491+0000", 2024-04-26T03:39:15.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.455 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:15.455 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.690491+0000", 2024-04-26T03:39:15.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.456 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:15.456 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.690492+0000", 2024-04-26T03:39:15.456 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4399999999999999e-07 2024-04-26T03:39:15.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.456 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:15.456 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.690513+0000", 2024-04-26T03:39:15.456 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1177000000000001e-05 2024-04-26T03:39:15.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.456 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:15.456 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.690532+0000", 2024-04-26T03:39:15.457 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9162000000000001e-05 2024-04-26T03:39:15.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.457 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:15.457 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.690544+0000", 2024-04-26T03:39:15.457 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1487e-05 2024-04-26T03:39:15.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.457 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:15.457 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.693752+0000", 2024-04-26T03:39:15.457 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0032081119999999999 2024-04-26T03:39:15.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.457 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:15.458 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.693764+0000", 2024-04-26T03:39:15.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.222e-05 2024-04-26T03:39:15.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.458 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:15.458 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.697796+0000", 2024-04-26T03:39:15.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0040317039999999997 2024-04-26T03:39:15.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.458 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:15.458 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.697839+0000", 2024-04-26T03:39:15.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2750000000000002e-05 2024-04-26T03:39:15.458 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.459 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:15.459 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.459 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.13s4 50/44 ECSubWrite(tid=74, reqid=client.4556.0:446, at_version=50'37, trim_to=0'0, roll_forward_to=50'36))", 2024-04-26T03:39:15.459 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:05.702160+0000", 2024-04-26T03:39:15.459 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.6825947909999996, 2024-04-26T03:39:15.459 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.008210953, 2024-04-26T03:39:15.459 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:15.459 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:15.459 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:15.459 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:15.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.459 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:15.460 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.702160+0000", 2024-04-26T03:39:15.460 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.460 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.460 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:15.460 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.702160+0000", 2024-04-26T03:39:15.460 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.460 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.460 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:15.460 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.702161+0000", 2024-04-26T03:39:15.460 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3900000000000003e-07 2024-04-26T03:39:15.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.461 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:15.461 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.702542+0000", 2024-04-26T03:39:15.461 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038117199999999999 2024-04-26T03:39:15.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.461 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:15.461 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.702556+0000", 2024-04-26T03:39:15.461 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3720000000000001e-05 2024-04-26T03:39:15.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.461 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:15.461 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.702562+0000", 2024-04-26T03:39:15.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3899999999999998e-06 2024-04-26T03:39:15.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:15.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.702632+0000", 2024-04-26T03:39:15.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9956999999999994e-05 2024-04-26T03:39:15.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:15.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.702644+0000", 2024-04-26T03:39:15.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1627000000000001e-05 2024-04-26T03:39:15.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:15.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.710338+0000", 2024-04-26T03:39:15.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0076939809999999999 2024-04-26T03:39:15.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:15.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.710371+0000", 2024-04-26T03:39:15.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3167000000000003e-05 2024-04-26T03:39:15.463 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.463 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:15.463 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.463 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.18s2 51/44 ECSubWrite(tid=216, reqid=client.4556.0:485, at_version=51'46, trim_to=0'0, roll_forward_to=50'43))", 2024-04-26T03:39:15.464 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:06.709835+0000", 2024-04-26T03:39:15.464 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.6749199719999996, 2024-04-26T03:39:15.464 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0077202110000000003, 2024-04-26T03:39:15.464 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:15.464 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:15.464 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:15.464 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:15.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.464 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:15.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.709835+0000", 2024-04-26T03:39:15.464 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.465 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:15.465 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.709835+0000", 2024-04-26T03:39:15.465 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.465 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.465 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:15.465 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.709835+0000", 2024-04-26T03:39:15.465 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.23e-07 2024-04-26T03:39:15.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.465 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.465 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:15.465 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.710337+0000", 2024-04-26T03:39:15.465 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00050177999999999996 2024-04-26T03:39:15.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.466 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:15.466 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.710351+0000", 2024-04-26T03:39:15.466 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3573e-05 2024-04-26T03:39:15.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.466 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:15.466 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.710358+0000", 2024-04-26T03:39:15.466 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7959999999999998e-06 2024-04-26T03:39:15.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.466 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:15.467 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.710421+0000", 2024-04-26T03:39:15.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3349000000000006e-05 2024-04-26T03:39:15.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.467 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:15.467 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.710431+0000", 2024-04-26T03:39:15.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9210000000000004e-06 2024-04-26T03:39:15.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.467 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:15.467 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.717526+0000", 2024-04-26T03:39:15.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0070952710000000002 2024-04-26T03:39:15.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.468 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:15.468 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.717555+0000", 2024-04-26T03:39:15.468 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8697999999999999e-05 2024-04-26T03:39:15.468 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.468 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:15.468 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.468 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds3 56/32 ECSubWrite(tid=29, reqid=client.4556.0:608, at_version=56'26, trim_to=0'0, roll_forward_to=50'23))", 2024-04-26T03:39:15.468 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:11.731789+0000", 2024-04-26T03:39:15.468 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.6529651269999999, 2024-04-26T03:39:15.468 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012144034999999999, 2024-04-26T03:39:15.468 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:15.468 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:15.469 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:15.469 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:15.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.469 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:15.469 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.731789+0000", 2024-04-26T03:39:15.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.469 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:15.469 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.731789+0000", 2024-04-26T03:39:15.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.469 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:15.469 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.731790+0000", 2024-04-26T03:39:15.470 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4000000000000001e-07 2024-04-26T03:39:15.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.470 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:15.470 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.732064+0000", 2024-04-26T03:39:15.470 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00027422299999999998 2024-04-26T03:39:15.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.470 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:15.470 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.732077+0000", 2024-04-26T03:39:15.470 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3016999999999999e-05 2024-04-26T03:39:15.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.471 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:15.471 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.732084+0000", 2024-04-26T03:39:15.471 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5780000000000004e-06 2024-04-26T03:39:15.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.471 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:15.471 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.732145+0000", 2024-04-26T03:39:15.471 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1008000000000003e-05 2024-04-26T03:39:15.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.471 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:15.471 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.732157+0000", 2024-04-26T03:39:15.471 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1875000000000001e-05 2024-04-26T03:39:15.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.472 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:15.472 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.743838+0000", 2024-04-26T03:39:15.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011680885 2024-04-26T03:39:15.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.472 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:15.472 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.743933+0000", 2024-04-26T03:39:15.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5709000000000001e-05 2024-04-26T03:39:15.472 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.472 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:15.472 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.472 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds3 56/32 ECSubWrite(tid=30, reqid=client.4556.0:609, at_version=56'27, trim_to=0'0, roll_forward_to=50'23))", 2024-04-26T03:39:15.473 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:11.737358+0000", 2024-04-26T03:39:15.473 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.6473961350000002, 2024-04-26T03:39:15.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0071715549999999996, 2024-04-26T03:39:15.473 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:15.473 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:15.473 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:15.473 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:15.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:15.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.737358+0000", 2024-04-26T03:39:15.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:15.474 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.737358+0000", 2024-04-26T03:39:15.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.474 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:15.474 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.737359+0000", 2024-04-26T03:39:15.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4000000000000001e-07 2024-04-26T03:39:15.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.474 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:15.474 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.737610+0000", 2024-04-26T03:39:15.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025033899999999998 2024-04-26T03:39:15.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.475 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:15.475 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.737638+0000", 2024-04-26T03:39:15.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8929999999999999e-05 2024-04-26T03:39:15.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.475 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:15.475 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.737656+0000", 2024-04-26T03:39:15.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7612999999999999e-05 2024-04-26T03:39:15.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.475 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:15.475 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.737764+0000", 2024-04-26T03:39:15.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000107599 2024-04-26T03:39:15.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:15.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.737775+0000", 2024-04-26T03:39:15.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1226000000000001e-05 2024-04-26T03:39:15.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:15.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.744490+0000", 2024-04-26T03:39:15.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0067152100000000001 2024-04-26T03:39:15.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:15.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.744530+0000", 2024-04-26T03:39:15.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9898000000000003e-05 2024-04-26T03:39:15.477 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.477 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:15.477 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.477 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.13s6 56/55 ECSubWrite(tid=104, reqid=client.4556.0:604, at_version=56'47, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:39:15.477 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:11.774744+0000", 2024-04-26T03:39:15.477 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.6100107609999998, 2024-04-26T03:39:15.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.016770859999999999, 2024-04-26T03:39:15.477 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:15.477 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:15.477 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:15.477 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:15.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:15.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.774744+0000", 2024-04-26T03:39:15.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:15.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.774744+0000", 2024-04-26T03:39:15.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:15.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.774746+0000", 2024-04-26T03:39:15.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3829999999999999e-06 2024-04-26T03:39:15.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.479 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:15.479 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.776913+0000", 2024-04-26T03:39:15.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0021663659999999999 2024-04-26T03:39:15.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.479 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:15.479 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.776926+0000", 2024-04-26T03:39:15.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3059e-05 2024-04-26T03:39:15.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.479 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:15.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.776932+0000", 2024-04-26T03:39:15.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9699999999999996e-06 2024-04-26T03:39:15.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.480 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:15.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.776980+0000", 2024-04-26T03:39:15.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8177000000000003e-05 2024-04-26T03:39:15.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.480 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:15.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.776986+0000", 2024-04-26T03:39:15.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.122e-06 2024-04-26T03:39:15.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.481 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:15.481 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.791479+0000", 2024-04-26T03:39:15.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.014492777 2024-04-26T03:39:15.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.481 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:15.481 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.791515+0000", 2024-04-26T03:39:15.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6006000000000001e-05 2024-04-26T03:39:15.481 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.481 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:15.481 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.482 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDPGPush(3.13s6 56/55 [PushOp(3:c8a9e28b:::smithi09235267-30:head, version: 49'32, data_included: [0~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 4, recovery_info: ObjectRecoveryInfo(3:c8a9e28b:::smithi09235267-30:head@49'32, size: 2768896, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-04-26T03:39:15.482 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:11.811532+0000", 2024-04-26T03:39:15.482 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.5732224619999999, 2024-04-26T03:39:15.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010374899999999999, 2024-04-26T03:39:15.482 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:15.482 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:15.482 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:15.482 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:15.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.482 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:15.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.811532+0000", 2024-04-26T03:39:15.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.483 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:15.483 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.811532+0000", 2024-04-26T03:39:15.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.483 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:15.483 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.811533+0000", 2024-04-26T03:39:15.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0800000000000005e-07 2024-04-26T03:39:15.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.483 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:15.483 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.812185+0000", 2024-04-26T03:39:15.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00065258999999999998 2024-04-26T03:39:15.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.484 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:15.484 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.812203+0000", 2024-04-26T03:39:15.484 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7367000000000002e-05 2024-04-26T03:39:15.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.484 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:15.484 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.812213+0000", 2024-04-26T03:39:15.484 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0431999999999999e-05 2024-04-26T03:39:15.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.484 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:15.484 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.816475+0000", 2024-04-26T03:39:15.485 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0042618659999999996 2024-04-26T03:39:15.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.485 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.485 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:15.485 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.821907+0000", 2024-04-26T03:39:15.485 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0054321370000000001 2024-04-26T03:39:15.485 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.485 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:15.485 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.485 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.485 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:15.485 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-26T03:39:15.487 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-26T03:39:15.487 INFO:tasks.ceph.osd.4.smithi028.stderr:2024-04-26T03:39:15.486+0000 7fd9511b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31963) UID: 0 2024-04-26T03:39:15.588 INFO:tasks.ceph.osd.15.smithi092.stderr:2024-04-26T03:39:15.587+0000 7f6d12dec640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 31824) UID: 0 2024-04-26T03:39:15.636 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-26T03:39:15.636 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-04-26T03:39:15.636 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-04-26T03:39:15.636 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-26T03:39:15.646 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-26T03:39:15.689 INFO:tasks.rados.rados.0.smithi092.stdout:260: done (6 left) 2024-04-26T03:39:15.689 INFO:tasks.rados.rados.0.smithi092.stdout:261: delete oid 15 current snap is 14 2024-04-26T03:39:15.690 INFO:tasks.rados.rados.0.smithi092.stdout:252: finishing write tid 3 to smithi09235267-46 2024-04-26T03:39:15.690 INFO:tasks.rados.rados.0.smithi092.stdout:252: finishing write tid 4 to smithi09235267-46 2024-04-26T03:39:15.690 INFO:tasks.rados.rados.0.smithi092.stdout:252: finishing write tid 5 to smithi09235267-46 2024-04-26T03:39:15.690 INFO:tasks.rados.rados.0.smithi092.stdout:254: finishing copy_from to smithi09235267-22 2024-04-26T03:39:15.690 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 22 v 10 (ObjNum 44 snap 0 seq_num 44) dirty exists 2024-04-26T03:39:15.690 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 50 v 52 (ObjNum 89 snap 7 seq_num 89) dirty exists 2024-04-26T03:39:15.690 INFO:tasks.rados.rados.0.smithi092.stdout:252: finishing write tid 6 to smithi09235267-46 2024-04-26T03:39:15.690 INFO:tasks.rados.rados.0.smithi092.stdout:252: oid 46 updating version 0 to 42 2024-04-26T03:39:15.690 INFO:tasks.rados.rados.0.smithi092.stdout:252: oid 46 version 42 is already newer than 38 2024-04-26T03:39:15.690 INFO:tasks.rados.rados.0.smithi092.stdout:252: oid 46 version 42 is already newer than 40 2024-04-26T03:39:15.690 INFO:tasks.rados.rados.0.smithi092.stdout:252: oid 46 version 42 is already newer than 39 2024-04-26T03:39:15.691 INFO:tasks.rados.rados.0.smithi092.stdout:252: oid 46 version 42 is already newer than 41 2024-04-26T03:39:15.691 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 46 v 42 (ObjNum 120 snap 14 seq_num 120) dirty exists 2024-04-26T03:39:15.691 INFO:tasks.rados.rados.0.smithi092.stdout:252: left oid 46 (ObjNum 120 snap 14 seq_num 120) 2024-04-26T03:39:15.691 INFO:tasks.rados.rados.0.smithi092.stdout:256: finishing write tid 1 to smithi09235267-13 2024-04-26T03:39:15.691 INFO:tasks.rados.rados.0.smithi092.stdout:258: expect (ObjNum 28 snap 0 seq_num 28) 2024-04-26T03:39:15.691 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:39:15.688+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:39:15.709 INFO:tasks.rados.rados.0.smithi092.stdout:256: finishing write tid 2 to smithi09235267-13 2024-04-26T03:39:15.709 INFO:tasks.rados.rados.0.smithi092.stdout:253: expect (ObjNum 118 snap 13 seq_num 118) 2024-04-26T03:39:15.725 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-04-26T03:39:15.726 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-26T03:39:15.726 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-04-26T03:39:15.726 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-04-26T03:39:15.762 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-26T03:39:15.762 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-04-26T03:39:15.762 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-04-26T03:39:15.762 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-04-26T03:39:15.762 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-26T03:39:15.771 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-26T03:39:15.813 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:39:15.811+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:39:15.890 INFO:tasks.ceph.osd.9.smithi042.stderr:2024-04-26T03:39:15.889+0000 7fdac63b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 31960) UID: 0 2024-04-26T03:39:15.911 INFO:tasks.rados.rados.0.smithi092.stdout:256: finishing write tid 3 to smithi09235267-13 2024-04-26T03:39:15.911 INFO:tasks.rados.rados.0.smithi092.stdout:256: finishing write tid 4 to smithi09235267-13 2024-04-26T03:39:15.911 INFO:tasks.rados.rados.0.smithi092.stdout:256: finishing write tid 5 to smithi09235267-13 2024-04-26T03:39:15.911 INFO:tasks.rados.rados.0.smithi092.stdout:256: finishing write tid 6 to smithi09235267-13 2024-04-26T03:39:15.911 INFO:tasks.rados.rados.0.smithi092.stdout:256: oid 13 updating version 0 to 66 2024-04-26T03:39:15.911 INFO:tasks.rados.rados.0.smithi092.stdout:256: oid 13 updating version 66 to 68 2024-04-26T03:39:15.911 INFO:tasks.rados.rados.0.smithi092.stdout:256: oid 13 version 68 is already newer than 67 2024-04-26T03:39:15.911 INFO:tasks.rados.rados.0.smithi092.stdout:256: oid 13 version 68 is already newer than 64 2024-04-26T03:39:15.911 INFO:tasks.rados.rados.0.smithi092.stdout:256: oid 13 version 68 is already newer than 65 2024-04-26T03:39:15.911 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 13 v 68 (ObjNum 121 snap 14 seq_num 121) dirty exists 2024-04-26T03:39:15.911 INFO:tasks.rados.rados.0.smithi092.stdout:256: left oid 13 (ObjNum 121 snap 14 seq_num 121) 2024-04-26T03:39:15.911 INFO:tasks.rados.rados.0.smithi092.stdout:252: done (6 left) 2024-04-26T03:39:15.911 INFO:tasks.rados.rados.0.smithi092.stdout:253: done (5 left) 2024-04-26T03:39:15.912 INFO:tasks.rados.rados.0.smithi092.stdout:254: done (4 left) 2024-04-26T03:39:15.912 INFO:tasks.rados.rados.0.smithi092.stdout:256: done (3 left) 2024-04-26T03:39:15.912 INFO:tasks.rados.rados.0.smithi092.stdout:257: done (2 left) 2024-04-26T03:39:15.912 INFO:tasks.rados.rados.0.smithi092.stdout:258: done (1 left) 2024-04-26T03:39:15.912 INFO:tasks.rados.rados.0.smithi092.stdout:261: done (0 left) 2024-04-26T03:39:15.912 INFO:tasks.rados.rados.0.smithi092.stdout:262: rollback oid 24 current snap is 14 2024-04-26T03:39:15.912 INFO:tasks.rados.rados.0.smithi092.stdout:rollback oid 24 to 10 2024-04-26T03:39:15.912 INFO:tasks.rados.rados.0.smithi092.stdout:263: setattr oid 4 current snap is 14 2024-04-26T03:39:15.914 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 13 current snap is 14 2024-04-26T03:39:15.914 INFO:tasks.rados.rados.0.smithi092.stdout:264: seq_num 123 ranges {2048000=524288,2572288=573440,3145728=688128,3833856=294912} 2024-04-26T03:39:15.924 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-26T03:39:15.924 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-04-26T03:39:15.924 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-04-26T03:39:15.924 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-04-26T03:39:15.924 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.924 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds4 35/32 ECSubWrite(tid=7, reqid=client.4556.0:64, at_version=35'5, trim_to=0'0, roll_forward_to=35'4))", 2024-04-26T03:39:15.924 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.528665+0000", 2024-04-26T03:39:15.925 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 24.393398987000001, 2024-04-26T03:39:15.925 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0061053569999999996, 2024-04-26T03:39:15.925 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:15.925 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:15.925 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:15.925 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:15.925 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.925 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:15.925 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528665+0000", 2024-04-26T03:39:15.925 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.925 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.925 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:15.926 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528665+0000", 2024-04-26T03:39:15.926 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.926 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.926 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:15.926 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528666+0000", 2024-04-26T03:39:15.926 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.071e-06 2024-04-26T03:39:15.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.926 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.926 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:15.926 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528902+0000", 2024-04-26T03:39:15.926 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00023567299999999999 2024-04-26T03:39:15.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.926 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.927 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:15.927 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528920+0000", 2024-04-26T03:39:15.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8252000000000001e-05 2024-04-26T03:39:15.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.927 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.927 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:15.927 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528930+0000", 2024-04-26T03:39:15.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7329999999999998e-06 2024-04-26T03:39:15.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.927 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.927 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:15.927 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.529014+0000", 2024-04-26T03:39:15.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3945999999999999e-05 2024-04-26T03:39:15.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.928 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:15.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.529031+0000", 2024-04-26T03:39:15.928 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6960999999999999e-05 2024-04-26T03:39:15.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.928 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:15.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.534732+0000", 2024-04-26T03:39:15.928 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0057011630000000004 2024-04-26T03:39:15.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.928 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:15.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.534771+0000", 2024-04-26T03:39:15.929 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8557999999999999e-05 2024-04-26T03:39:15.929 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.929 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:15.929 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.929 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.929 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.4s6 35/32 ECSubWrite(tid=11, reqid=client.4556.0:100, at_version=35'7, trim_to=0'0, roll_forward_to=35'6))", 2024-04-26T03:39:15.929 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.560536+0000", 2024-04-26T03:39:15.929 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 24.361528471, 2024-04-26T03:39:15.929 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0052869859999999996, 2024-04-26T03:39:15.929 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:15.929 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:15.929 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:15.929 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:15.930 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.930 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:15.930 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.560536+0000", 2024-04-26T03:39:15.930 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.930 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.930 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:15.930 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.560536+0000", 2024-04-26T03:39:15.930 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.930 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.930 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:15.930 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.560536+0000", 2024-04-26T03:39:15.930 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.23e-07 2024-04-26T03:39:15.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.931 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.931 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:15.931 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.560987+0000", 2024-04-26T03:39:15.931 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00045054099999999999 2024-04-26T03:39:15.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.931 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.931 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:15.931 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.561004+0000", 2024-04-26T03:39:15.931 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6691999999999999e-05 2024-04-26T03:39:15.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.931 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.931 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:15.931 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.561012+0000", 2024-04-26T03:39:15.932 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.225e-06 2024-04-26T03:39:15.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.932 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:15.932 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.561073+0000", 2024-04-26T03:39:15.932 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1017000000000002e-05 2024-04-26T03:39:15.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.932 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:15.932 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.561082+0000", 2024-04-26T03:39:15.932 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2830000000000007e-06 2024-04-26T03:39:15.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.932 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:15.933 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.565784+0000", 2024-04-26T03:39:15.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0047014530000000004 2024-04-26T03:39:15.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.933 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.933 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:15.933 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.565823+0000", 2024-04-26T03:39:15.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8952000000000002e-05 2024-04-26T03:39:15.933 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.933 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:15.933 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.933 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.933 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.fs2 35/32 ECSubWrite(tid=15, reqid=client.4556.0:136, at_version=35'5, trim_to=0'0, roll_forward_to=35'4))", 2024-04-26T03:39:15.933 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.615680+0000", 2024-04-26T03:39:15.933 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 24.306383808, 2024-04-26T03:39:15.934 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0055997080000000001, 2024-04-26T03:39:15.934 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:15.934 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:15.934 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:15.934 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:15.934 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.934 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:15.934 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.615680+0000", 2024-04-26T03:39:15.934 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.934 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.934 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.934 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:15.934 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.615680+0000", 2024-04-26T03:39:15.935 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.935 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.935 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:15.935 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.615682+0000", 2024-04-26T03:39:15.935 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.221e-06 2024-04-26T03:39:15.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.935 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.935 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:15.935 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.616158+0000", 2024-04-26T03:39:15.935 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00047638800000000001 2024-04-26T03:39:15.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.935 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.935 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:15.935 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.616172+0000", 2024-04-26T03:39:15.936 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3984e-05 2024-04-26T03:39:15.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.936 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.936 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:15.936 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.616179+0000", 2024-04-26T03:39:15.936 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5490000000000001e-06 2024-04-26T03:39:15.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.936 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.936 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:15.936 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.616232+0000", 2024-04-26T03:39:15.936 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2528000000000003e-05 2024-04-26T03:39:15.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.936 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.936 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:15.937 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.616240+0000", 2024-04-26T03:39:15.937 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7889999999999995e-06 2024-04-26T03:39:15.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.937 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.937 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:15.937 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.621245+0000", 2024-04-26T03:39:15.937 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0050054909999999999 2024-04-26T03:39:15.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.937 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.937 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:15.937 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.621280+0000", 2024-04-26T03:39:15.937 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4758000000000002e-05 2024-04-26T03:39:15.937 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.937 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:15.938 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.938 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.938 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.2s1 35/32 ECSubWrite(tid=8, reqid=client.4556.0:139, at_version=35'5, trim_to=0'0, roll_forward_to=35'4))", 2024-04-26T03:39:15.938 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.620822+0000", 2024-04-26T03:39:15.938 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 24.30124202, 2024-04-26T03:39:15.938 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.005803173, 2024-04-26T03:39:15.938 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:15.938 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:15.938 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:15.938 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:15.938 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.939 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:15.939 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.620822+0000", 2024-04-26T03:39:15.939 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.939 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.939 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:15.939 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.620822+0000", 2024-04-26T03:39:15.939 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.939 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.939 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:15.939 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.620823+0000", 2024-04-26T03:39:15.939 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1399999999999996e-07 2024-04-26T03:39:15.940 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.940 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.940 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:15.940 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.621346+0000", 2024-04-26T03:39:15.940 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00052321400000000004 2024-04-26T03:39:15.940 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.940 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.940 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:15.940 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.621364+0000", 2024-04-26T03:39:15.940 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8360999999999999e-05 2024-04-26T03:39:15.940 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.940 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.940 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:15.940 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.621373+0000", 2024-04-26T03:39:15.941 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4649999999999994e-06 2024-04-26T03:39:15.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.941 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.941 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:15.941 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.621428+0000", 2024-04-26T03:39:15.941 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5062000000000002e-05 2024-04-26T03:39:15.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.941 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.941 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:15.941 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.621437+0000", 2024-04-26T03:39:15.941 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3600000000000002e-06 2024-04-26T03:39:15.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.941 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.942 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:15.942 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.626599+0000", 2024-04-26T03:39:15.942 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.005162191 2024-04-26T03:39:15.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.942 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.942 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:15.942 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.626625+0000", 2024-04-26T03:39:15.942 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5806e-05 2024-04-26T03:39:15.942 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.942 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:15.942 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.942 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.942 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.4s6 40/32 ECSubWrite(tid=22, reqid=client.4556.0:262, at_version=40'14, trim_to=0'0, roll_forward_to=35'12))", 2024-04-26T03:39:15.943 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:56.521608+0000", 2024-04-26T03:39:15.943 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.40045581, 2024-04-26T03:39:15.943 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.006347615, 2024-04-26T03:39:15.943 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:15.943 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:15.943 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:15.943 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:15.943 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.943 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:15.943 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.521608+0000", 2024-04-26T03:39:15.943 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.943 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.943 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.943 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:15.943 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.521608+0000", 2024-04-26T03:39:15.944 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.944 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.944 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:15.944 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.521609+0000", 2024-04-26T03:39:15.944 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9000000000000001e-07 2024-04-26T03:39:15.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.944 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.944 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:15.944 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.521956+0000", 2024-04-26T03:39:15.944 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00034728800000000001 2024-04-26T03:39:15.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.944 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.944 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:15.944 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.521972+0000", 2024-04-26T03:39:15.945 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6150999999999998e-05 2024-04-26T03:39:15.945 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.945 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.945 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:15.945 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.521980+0000", 2024-04-26T03:39:15.945 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1990000000000006e-06 2024-04-26T03:39:15.945 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.945 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.945 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:15.945 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.522048+0000", 2024-04-26T03:39:15.945 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8235e-05 2024-04-26T03:39:15.945 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.945 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.945 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:15.946 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.522055+0000", 2024-04-26T03:39:15.946 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0639999999999996e-06 2024-04-26T03:39:15.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.946 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.946 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:15.946 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.527920+0000", 2024-04-26T03:39:15.946 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0058647300000000003 2024-04-26T03:39:15.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.946 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.946 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:15.946 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.527956+0000", 2024-04-26T03:39:15.946 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5757999999999999e-05 2024-04-26T03:39:15.946 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.947 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:15.947 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.947 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.947 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.8s6 40/32 ECSubWrite(tid=178, reqid=client.4556.0:266, at_version=40'8, trim_to=0'0, roll_forward_to=35'6))", 2024-04-26T03:39:15.947 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:56.533409+0000", 2024-04-26T03:39:15.947 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.388655198999999, 2024-04-26T03:39:15.947 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0056447629999999997, 2024-04-26T03:39:15.947 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:15.947 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:15.947 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:15.947 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:15.947 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.947 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:15.947 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.533409+0000", 2024-04-26T03:39:15.948 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.948 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.948 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.948 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:15.948 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.533409+0000", 2024-04-26T03:39:15.948 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.948 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.948 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.948 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:15.948 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.533409+0000", 2024-04-26T03:39:15.948 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9000000000000002e-07 2024-04-26T03:39:15.948 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.948 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.948 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:15.949 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.533800+0000", 2024-04-26T03:39:15.949 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000391037 2024-04-26T03:39:15.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.949 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.949 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:15.949 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.533813+0000", 2024-04-26T03:39:15.949 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.255e-05 2024-04-26T03:39:15.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.949 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.949 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:15.949 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.533819+0000", 2024-04-26T03:39:15.949 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2190000000000001e-06 2024-04-26T03:39:15.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.950 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.950 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:15.950 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.533896+0000", 2024-04-26T03:39:15.950 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6477999999999995e-05 2024-04-26T03:39:15.950 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.950 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.950 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:15.950 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.533902+0000", 2024-04-26T03:39:15.950 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7009999999999999e-06 2024-04-26T03:39:15.950 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.950 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.950 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:15.950 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.539008+0000", 2024-04-26T03:39:15.951 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0051058409999999999 2024-04-26T03:39:15.951 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.951 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.951 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:15.951 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.539054+0000", 2024-04-26T03:39:15.951 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5547000000000003e-05 2024-04-26T03:39:15.951 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.951 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:15.951 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.951 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.951 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.951 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.fs2 45/32 ECSubWrite(tid=50, reqid=client.4556.0:326, at_version=45'18, trim_to=0'0, roll_forward_to=42'15))", 2024-04-26T03:39:15.951 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:01.561083+0000", 2024-04-26T03:39:15.951 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.36098146, 2024-04-26T03:39:15.952 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0051491890000000002, 2024-04-26T03:39:15.952 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:15.952 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:15.952 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:15.952 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:15.952 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.952 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:15.952 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.561083+0000", 2024-04-26T03:39:15.952 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.952 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.952 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.952 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:15.952 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.561083+0000", 2024-04-26T03:39:15.953 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.953 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.953 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:15.953 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.561083+0000", 2024-04-26T03:39:15.953 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0399999999999995e-07 2024-04-26T03:39:15.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.953 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.953 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:15.953 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.561462+0000", 2024-04-26T03:39:15.953 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000378828 2024-04-26T03:39:15.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.953 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.953 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:15.954 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.561475+0000", 2024-04-26T03:39:15.954 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2506e-05 2024-04-26T03:39:15.954 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.954 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.954 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:15.954 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.561483+0000", 2024-04-26T03:39:15.954 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2849999999999995e-06 2024-04-26T03:39:15.954 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.954 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.954 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:15.954 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.561543+0000", 2024-04-26T03:39:15.954 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0176000000000001e-05 2024-04-26T03:39:15.954 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.954 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.955 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:15.955 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.561553+0000", 2024-04-26T03:39:15.955 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9669999999999996e-06 2024-04-26T03:39:15.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.955 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.955 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:15.955 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.566196+0000", 2024-04-26T03:39:15.955 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0046429990000000001 2024-04-26T03:39:15.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.955 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.955 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:15.955 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.566232+0000", 2024-04-26T03:39:15.955 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5723999999999999e-05 2024-04-26T03:39:15.955 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.956 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:15.956 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.956 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.956 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.956 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.2s1 49/44 ECSubWrite(tid=32, reqid=client.4556.0:433, at_version=49'20, trim_to=0'0, roll_forward_to=48'19))", 2024-04-26T03:39:15.956 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:04.699918+0000", 2024-04-26T03:39:15.956 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 11.222146266999999, 2024-04-26T03:39:15.956 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0082878350000000003, 2024-04-26T03:39:15.956 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:15.956 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:15.956 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:15.956 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:15.956 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.956 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:15.957 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.699918+0000", 2024-04-26T03:39:15.957 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.957 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.957 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:15.957 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.699918+0000", 2024-04-26T03:39:15.957 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.957 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.957 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:15.957 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.699918+0000", 2024-04-26T03:39:15.957 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6300000000000005e-07 2024-04-26T03:39:15.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.957 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.958 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:15.958 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.700410+0000", 2024-04-26T03:39:15.958 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00049148099999999999 2024-04-26T03:39:15.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.958 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.958 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:15.958 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.700422+0000", 2024-04-26T03:39:15.958 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2466000000000001e-05 2024-04-26T03:39:15.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.958 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.958 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:15.958 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.700431+0000", 2024-04-26T03:39:15.958 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2479999999999996e-06 2024-04-26T03:39:15.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.959 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.959 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:15.959 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.700559+0000", 2024-04-26T03:39:15.959 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012855599999999999 2024-04-26T03:39:15.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.959 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.959 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:15.959 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.700570+0000", 2024-04-26T03:39:15.959 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0652000000000001e-05 2024-04-26T03:39:15.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.959 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.959 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:15.960 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.708166+0000", 2024-04-26T03:39:15.960 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0075965770000000002 2024-04-26T03:39:15.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.960 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:15.960 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.708206+0000", 2024-04-26T03:39:15.960 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9292000000000003e-05 2024-04-26T03:39:15.960 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.960 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:15.960 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.960 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.2s1 49/44 ECSubWrite(tid=34, reqid=client.4556.0:434, at_version=49'23, trim_to=0'0, roll_forward_to=48'19))", 2024-04-26T03:39:15.960 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:04.712506+0000", 2024-04-26T03:39:15.961 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 11.209558431, 2024-04-26T03:39:15.961 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0055605059999999998, 2024-04-26T03:39:15.961 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:15.961 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:15.961 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:15.961 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:15.961 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.961 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:15.961 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.712506+0000", 2024-04-26T03:39:15.961 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.961 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.961 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:15.961 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.712506+0000", 2024-04-26T03:39:15.961 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.962 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.962 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:15.962 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.712507+0000", 2024-04-26T03:39:15.962 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9500000000000001e-07 2024-04-26T03:39:15.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.962 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.962 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:15.962 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.712718+0000", 2024-04-26T03:39:15.962 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000211728 2024-04-26T03:39:15.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:15.963 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.712731+0000", 2024-04-26T03:39:15.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2986000000000001e-05 2024-04-26T03:39:15.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:15.964 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.712740+0000", 2024-04-26T03:39:15.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8310000000000007e-06 2024-04-26T03:39:15.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.964 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.964 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:15.964 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.712810+0000", 2024-04-26T03:39:15.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9560000000000005e-05 2024-04-26T03:39:15.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.964 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.964 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:15.964 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.712820+0000", 2024-04-26T03:39:15.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0710000000000001e-05 2024-04-26T03:39:15.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.964 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.965 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:15.965 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.718031+0000", 2024-04-26T03:39:15.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0052108229999999998 2024-04-26T03:39:15.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.965 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:15.965 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.718066+0000", 2024-04-26T03:39:15.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5073000000000001e-05 2024-04-26T03:39:15.965 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.965 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:15.965 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.966 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.14s6 49/44 ECSubWrite(tid=35, reqid=client.4556.0:438, at_version=49'24, trim_to=0'0, roll_forward_to=46'22))", 2024-04-26T03:39:15.966 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:04.726292+0000", 2024-04-26T03:39:15.966 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 11.195772627, 2024-04-26T03:39:15.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0054626309999999999, 2024-04-26T03:39:15.966 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:15.966 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:15.966 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:15.966 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:15.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:15.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.726292+0000", 2024-04-26T03:39:15.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:15.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.726292+0000", 2024-04-26T03:39:15.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:15.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.726292+0000", 2024-04-26T03:39:15.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1999999999999999e-07 2024-04-26T03:39:15.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:15.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.726713+0000", 2024-04-26T03:39:15.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00042059200000000001 2024-04-26T03:39:15.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.968 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:15.968 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.726726+0000", 2024-04-26T03:39:15.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.362e-05 2024-04-26T03:39:15.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.968 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:15.968 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.726735+0000", 2024-04-26T03:39:15.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7700000000000007e-06 2024-04-26T03:39:15.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.968 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:15.968 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.726811+0000", 2024-04-26T03:39:15.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5953999999999998e-05 2024-04-26T03:39:15.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.969 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:15.969 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.726818+0000", 2024-04-26T03:39:15.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.17e-06 2024-04-26T03:39:15.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.969 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:15.969 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.731718+0000", 2024-04-26T03:39:15.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.004899965 2024-04-26T03:39:15.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.969 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:15.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.731754+0000", 2024-04-26T03:39:15.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5939999999999998e-05 2024-04-26T03:39:15.970 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.970 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:15.970 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.970 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.18s6 49/44 ECSubWrite(tid=206, reqid=client.4556.0:443, at_version=49'36, trim_to=0'0, roll_forward_to=49'34))", 2024-04-26T03:39:15.970 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:04.734641+0000", 2024-04-26T03:39:15.970 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 11.187423365000001, 2024-04-26T03:39:15.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0066362670000000004, 2024-04-26T03:39:15.970 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:15.970 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:15.971 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:15.971 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:15.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:15.971 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.734641+0000", 2024-04-26T03:39:15.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:15.971 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.734641+0000", 2024-04-26T03:39:15.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:15.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.734642+0000", 2024-04-26T03:39:15.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.516e-06 2024-04-26T03:39:15.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:15.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.735354+0000", 2024-04-26T03:39:15.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00071165699999999998 2024-04-26T03:39:15.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:15.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.735370+0000", 2024-04-26T03:39:15.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6175e-05 2024-04-26T03:39:15.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.973 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:15.973 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.735377+0000", 2024-04-26T03:39:15.973 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1559999999999998e-06 2024-04-26T03:39:15.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.973 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.973 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:15.973 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.735445+0000", 2024-04-26T03:39:15.973 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7557000000000004e-05 2024-04-26T03:39:15.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.973 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.973 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:15.973 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.735455+0000", 2024-04-26T03:39:15.973 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0165e-05 2024-04-26T03:39:15.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.974 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:15.974 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.741242+0000", 2024-04-26T03:39:15.974 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.005787309 2024-04-26T03:39:15.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.974 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:15.974 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.741277+0000", 2024-04-26T03:39:15.974 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4731999999999999e-05 2024-04-26T03:39:15.974 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.974 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:15.974 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.975 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.14s6 49/44 ECSubWrite(tid=36, reqid=client.4556.0:439, at_version=49'25, trim_to=0'0, roll_forward_to=46'22))", 2024-04-26T03:39:15.975 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:04.734819+0000", 2024-04-26T03:39:15.975 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 11.18724559, 2024-04-26T03:39:15.975 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0079486820000000003, 2024-04-26T03:39:15.975 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:15.975 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:15.975 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:15.975 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:15.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.975 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:15.975 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.734819+0000", 2024-04-26T03:39:15.975 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.976 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:15.976 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.734819+0000", 2024-04-26T03:39:15.976 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.976 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:15.976 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.734819+0000", 2024-04-26T03:39:15.976 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9999999999999997e-07 2024-04-26T03:39:15.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.976 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:15.976 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.735094+0000", 2024-04-26T03:39:15.976 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00027495999999999998 2024-04-26T03:39:15.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.977 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:15.977 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.735111+0000", 2024-04-26T03:39:15.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6872999999999999e-05 2024-04-26T03:39:15.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.977 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:15.977 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.735122+0000", 2024-04-26T03:39:15.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1055e-05 2024-04-26T03:39:15.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.977 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:15.977 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.735189+0000", 2024-04-26T03:39:15.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7075000000000004e-05 2024-04-26T03:39:15.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.978 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.978 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:15.978 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.735199+0000", 2024-04-26T03:39:15.978 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8479999999999996e-06 2024-04-26T03:39:15.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.978 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.978 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:15.978 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.742740+0000", 2024-04-26T03:39:15.978 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0075408699999999999 2024-04-26T03:39:15.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.978 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.978 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:15.978 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.742767+0000", 2024-04-26T03:39:15.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7300999999999998e-05 2024-04-26T03:39:15.979 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.979 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:15.979 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.979 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.5s6 50/44 ECSubWrite(tid=51, reqid=client.4556.0:462, at_version=50'47, trim_to=0'0, roll_forward_to=50'46))", 2024-04-26T03:39:15.979 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:05.751045+0000", 2024-04-26T03:39:15.979 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.171019318000001, 2024-04-26T03:39:15.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0052423519999999996, 2024-04-26T03:39:15.979 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:15.979 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:15.979 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:15.979 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:15.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.980 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:15.980 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.751045+0000", 2024-04-26T03:39:15.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.980 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:15.980 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.751045+0000", 2024-04-26T03:39:15.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.980 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:15.980 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.751047+0000", 2024-04-26T03:39:15.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3269999999999999e-06 2024-04-26T03:39:15.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.981 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:15.981 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.751699+0000", 2024-04-26T03:39:15.981 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00065218399999999999 2024-04-26T03:39:15.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.981 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:15.981 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.751716+0000", 2024-04-26T03:39:15.981 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6293000000000001e-05 2024-04-26T03:39:15.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.981 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:15.981 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.751727+0000", 2024-04-26T03:39:15.982 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.148e-05 2024-04-26T03:39:15.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.982 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.982 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:15.982 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.751794+0000", 2024-04-26T03:39:15.982 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7130000000000003e-05 2024-04-26T03:39:15.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.982 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.982 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:15.982 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.751803+0000", 2024-04-26T03:39:15.982 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5080000000000005e-06 2024-04-26T03:39:15.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.982 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.983 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:15.983 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.756254+0000", 2024-04-26T03:39:15.983 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.004451048 2024-04-26T03:39:15.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.983 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.983 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:15.983 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.756287+0000", 2024-04-26T03:39:15.983 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3382e-05 2024-04-26T03:39:15.983 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.983 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:15.983 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.983 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.983 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.2s1 50/44 ECSubWrite(tid=42, reqid=client.4556.0:478, at_version=50'28, trim_to=0'0, roll_forward_to=50'27))", 2024-04-26T03:39:15.984 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:05.912081+0000", 2024-04-26T03:39:15.984 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.009982802, 2024-04-26T03:39:15.984 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0054845720000000001, 2024-04-26T03:39:15.984 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:15.984 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:15.984 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:15.984 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:15.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.984 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:15.984 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.912081+0000", 2024-04-26T03:39:15.984 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.984 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:15.985 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.912081+0000", 2024-04-26T03:39:15.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.985 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:15.985 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.912082+0000", 2024-04-26T03:39:15.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.065e-06 2024-04-26T03:39:15.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.985 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:15.985 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.912484+0000", 2024-04-26T03:39:15.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00040203599999999998 2024-04-26T03:39:15.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.986 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:15.986 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.912500+0000", 2024-04-26T03:39:15.986 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5909e-05 2024-04-26T03:39:15.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.986 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:15.986 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.912527+0000", 2024-04-26T03:39:15.986 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6265999999999999e-05 2024-04-26T03:39:15.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.986 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:15.986 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.912620+0000", 2024-04-26T03:39:15.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3826999999999995e-05 2024-04-26T03:39:15.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.987 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:15.987 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.912636+0000", 2024-04-26T03:39:15.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5159e-05 2024-04-26T03:39:15.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.987 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:15.987 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.917539+0000", 2024-04-26T03:39:15.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0049031509999999997 2024-04-26T03:39:15.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.987 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:15.988 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.917566+0000", 2024-04-26T03:39:15.988 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7158999999999999e-05 2024-04-26T03:39:15.988 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.988 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:15.988 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.988 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.988 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.18s6 51/44 ECSubWrite(tid=215, reqid=client.4556.0:484, at_version=51'45, trim_to=0'0, roll_forward_to=50'43))", 2024-04-26T03:39:15.988 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:06.705341+0000", 2024-04-26T03:39:15.988 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.2167235739999995, 2024-04-26T03:39:15.988 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0089840560000000007, 2024-04-26T03:39:15.988 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:15.988 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:15.988 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:15.989 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:15.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.989 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:15.989 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.705341+0000", 2024-04-26T03:39:15.989 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.989 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:15.989 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.705341+0000", 2024-04-26T03:39:15.989 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.989 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:15.990 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.705341+0000", 2024-04-26T03:39:15.990 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7700000000000001e-07 2024-04-26T03:39:15.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.990 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.990 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:15.990 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.705617+0000", 2024-04-26T03:39:15.990 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000276655 2024-04-26T03:39:15.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.990 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.990 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:15.990 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.705633+0000", 2024-04-26T03:39:15.990 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.537e-05 2024-04-26T03:39:15.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.991 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.991 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:15.991 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.705643+0000", 2024-04-26T03:39:15.991 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0516000000000001e-05 2024-04-26T03:39:15.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.991 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.991 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:15.991 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.705722+0000", 2024-04-26T03:39:15.991 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8232999999999994e-05 2024-04-26T03:39:15.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.991 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.991 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:15.991 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.705735+0000", 2024-04-26T03:39:15.991 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3645000000000001e-05 2024-04-26T03:39:15.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.992 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:15.992 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.714285+0000", 2024-04-26T03:39:15.992 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0085501989999999996 2024-04-26T03:39:15.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.992 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:15.992 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.714325+0000", 2024-04-26T03:39:15.992 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9261000000000002e-05 2024-04-26T03:39:15.992 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.992 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:15.992 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.993 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.993 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.18s6 51/44 ECSubWrite(tid=216, reqid=client.4556.0:485, at_version=51'46, trim_to=0'0, roll_forward_to=50'43))", 2024-04-26T03:39:15.993 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:06.709609+0000", 2024-04-26T03:39:15.993 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.2124555689999994, 2024-04-26T03:39:15.993 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0051560080000000001, 2024-04-26T03:39:15.993 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:15.993 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:15.993 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:15.993 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:15.993 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.993 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:15.993 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.709609+0000", 2024-04-26T03:39:15.993 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.994 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.994 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:15.994 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.709609+0000", 2024-04-26T03:39:15.994 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.994 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.994 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:15.994 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.709609+0000", 2024-04-26T03:39:15.994 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5600000000000005e-07 2024-04-26T03:39:15.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.994 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.994 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:15.994 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.709896+0000", 2024-04-26T03:39:15.995 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000286946 2024-04-26T03:39:15.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.995 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:15.995 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.709925+0000", 2024-04-26T03:39:15.995 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8751e-05 2024-04-26T03:39:15.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.995 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:15.995 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.709933+0000", 2024-04-26T03:39:15.995 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9850000000000006e-06 2024-04-26T03:39:15.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.995 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:15.996 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.709988+0000", 2024-04-26T03:39:15.996 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.499e-05 2024-04-26T03:39:15.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.996 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:15.996 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.709995+0000", 2024-04-26T03:39:15.996 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6830000000000002e-06 2024-04-26T03:39:15.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.996 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:15.996 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.714723+0000", 2024-04-26T03:39:15.996 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0047280200000000003 2024-04-26T03:39:15.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.997 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.997 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:15.997 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.714765+0000", 2024-04-26T03:39:15.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1876999999999999e-05 2024-04-26T03:39:15.997 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.997 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:15.997 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:15.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.997 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.997 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds4 56/32 ECSubWrite(tid=28, reqid=client.4556.0:607, at_version=56'25, trim_to=0'0, roll_forward_to=50'23))", 2024-04-26T03:39:15.997 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:11.724542+0000", 2024-04-26T03:39:15.997 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.197522073, 2024-04-26T03:39:15.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.014413491, 2024-04-26T03:39:15.998 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:15.998 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:15.998 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:15.998 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:15.998 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.998 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:15.998 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.724542+0000", 2024-04-26T03:39:15.998 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.998 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.998 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:15.998 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.724542+0000", 2024-04-26T03:39:15.999 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:15.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.999 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:15.999 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.724542+0000", 2024-04-26T03:39:15.999 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8700000000000001e-07 2024-04-26T03:39:15.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.999 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:15.999 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.725098+0000", 2024-04-26T03:39:15.999 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000555465 2024-04-26T03:39:15.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:15.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:15.999 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:16.000 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.725111+0000", 2024-04-26T03:39:16.000 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3043e-05 2024-04-26T03:39:16.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.000 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.000 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:16.000 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.725120+0000", 2024-04-26T03:39:16.000 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0710000000000001e-06 2024-04-26T03:39:16.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.000 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.000 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:16.000 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.725201+0000", 2024-04-26T03:39:16.000 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0914000000000005e-05 2024-04-26T03:39:16.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.001 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:16.001 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.725211+0000", 2024-04-26T03:39:16.001 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7440000000000002e-06 2024-04-26T03:39:16.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.001 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:16.001 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.738695+0000", 2024-04-26T03:39:16.001 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013484489000000001 2024-04-26T03:39:16.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.001 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:16.001 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.738956+0000", 2024-04-26T03:39:16.002 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026037800000000002 2024-04-26T03:39:16.002 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.002 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:16.002 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.002 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.002 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds4 56/32 ECSubWrite(tid=29, reqid=client.4556.0:608, at_version=56'26, trim_to=0'0, roll_forward_to=50'23))", 2024-04-26T03:39:16.002 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:11.731977+0000", 2024-04-26T03:39:16.002 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.190086923, 2024-04-26T03:39:16.002 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012989461000000001, 2024-04-26T03:39:16.002 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:16.002 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:16.003 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:16.003 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:16.003 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.003 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:16.003 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.731977+0000", 2024-04-26T03:39:16.003 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.003 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.003 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:16.003 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.731977+0000", 2024-04-26T03:39:16.003 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.003 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.004 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:16.004 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.731978+0000", 2024-04-26T03:39:16.004 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8100000000000001e-07 2024-04-26T03:39:16.004 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.004 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.004 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:16.004 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.732398+0000", 2024-04-26T03:39:16.004 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00041989499999999999 2024-04-26T03:39:16.004 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.004 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.004 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:16.004 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.732413+0000", 2024-04-26T03:39:16.004 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4861999999999999e-05 2024-04-26T03:39:16.004 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.005 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.005 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:16.005 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.732423+0000", 2024-04-26T03:39:16.005 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.026e-05 2024-04-26T03:39:16.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.005 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.005 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:16.005 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.732866+0000", 2024-04-26T03:39:16.005 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00044276299999999999 2024-04-26T03:39:16.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.005 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.005 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:16.005 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.732879+0000", 2024-04-26T03:39:16.006 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2632999999999999e-05 2024-04-26T03:39:16.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.006 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.006 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:16.006 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.744874+0000", 2024-04-26T03:39:16.006 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011995530000000001 2024-04-26T03:39:16.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.006 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.006 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:16.006 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.744967+0000", 2024-04-26T03:39:16.006 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2637000000000001e-05 2024-04-26T03:39:16.006 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.006 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:16.006 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.007 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.007 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.5s6 56/44 ECSubWrite(tid=70, reqid=client.4556.0:615, at_version=56'57, trim_to=0'0, roll_forward_to=52'55))", 2024-04-26T03:39:16.007 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:11.733862+0000", 2024-04-26T03:39:16.007 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.1882017359999999, 2024-04-26T03:39:16.007 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011141698, 2024-04-26T03:39:16.007 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:16.007 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:16.007 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:16.007 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:16.007 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.007 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:16.007 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.733862+0000", 2024-04-26T03:39:16.007 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.008 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.008 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:16.008 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.733862+0000", 2024-04-26T03:39:16.008 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.008 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.008 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:16.008 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.733863+0000", 2024-04-26T03:39:16.008 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7100000000000001e-07 2024-04-26T03:39:16.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.008 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.008 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:16.008 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.733978+0000", 2024-04-26T03:39:16.008 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000115505 2024-04-26T03:39:16.009 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.009 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.009 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:16.009 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.733991+0000", 2024-04-26T03:39:16.009 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3251999999999999e-05 2024-04-26T03:39:16.009 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.009 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.009 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:16.009 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.733998+0000", 2024-04-26T03:39:16.009 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.392e-06 2024-04-26T03:39:16.009 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.009 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.009 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:16.009 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.738752+0000", 2024-04-26T03:39:16.009 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0047539269999999998 2024-04-26T03:39:16.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.010 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.010 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:16.010 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.738762+0000", 2024-04-26T03:39:16.010 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0429e-05 2024-04-26T03:39:16.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.010 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.010 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:16.010 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.744977+0000", 2024-04-26T03:39:16.010 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0062144380000000001 2024-04-26T03:39:16.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.010 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.010 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:16.011 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.745004+0000", 2024-04-26T03:39:16.011 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7583999999999999e-05 2024-04-26T03:39:16.011 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.011 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:16.011 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.011 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds4 56/32 ECSubWrite(tid=30, reqid=client.4556.0:609, at_version=56'27, trim_to=0'0, roll_forward_to=50'23))", 2024-04-26T03:39:16.011 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:11.737431+0000", 2024-04-26T03:39:16.011 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.1846329109999996, 2024-04-26T03:39:16.011 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0080512009999999991, 2024-04-26T03:39:16.011 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:16.011 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:16.011 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:16.012 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:16.012 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.012 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:16.012 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.737431+0000", 2024-04-26T03:39:16.012 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.012 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.012 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:16.012 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.737431+0000", 2024-04-26T03:39:16.012 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.012 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.012 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:16.012 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.737433+0000", 2024-04-26T03:39:16.013 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.717e-06 2024-04-26T03:39:16.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.013 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.013 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:16.013 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.737960+0000", 2024-04-26T03:39:16.013 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00052740199999999995 2024-04-26T03:39:16.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.013 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.013 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:16.013 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.738089+0000", 2024-04-26T03:39:16.013 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012901699999999999 2024-04-26T03:39:16.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.013 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.013 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:16.013 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.738102+0000", 2024-04-26T03:39:16.014 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2456999999999999e-05 2024-04-26T03:39:16.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.014 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.014 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:16.014 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.739001+0000", 2024-04-26T03:39:16.014 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00089964699999999995 2024-04-26T03:39:16.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.014 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.014 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:16.014 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.739011+0000", 2024-04-26T03:39:16.014 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6800000000000005e-06 2024-04-26T03:39:16.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.014 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.014 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:16.015 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.745459+0000", 2024-04-26T03:39:16.015 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0064480750000000002 2024-04-26T03:39:16.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.015 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:16.015 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.745482+0000", 2024-04-26T03:39:16.015 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3206000000000001e-05 2024-04-26T03:39:16.015 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.015 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:16.015 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.015 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.015 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:16.015 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-26T03:39:16.017 INFO:tasks.ceph.osd.4.smithi028.stderr:2024-04-26T03:39:16.016+0000 7fd9511b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31963) UID: 0 2024-04-26T03:39:16.018 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-04-26T03:39:16.021 INFO:tasks.rados.rados.0.smithi092.stdout:264: writing smithi09235267-13 from 2048000 to 2572288 tid 1 2024-04-26T03:39:16.021 INFO:tasks.rados.rados.0.smithi092.stdout:264: writing smithi09235267-13 from 2572288 to 3145728 tid 2 2024-04-26T03:39:16.021 INFO:tasks.rados.rados.0.smithi092.stdout:264: writing smithi09235267-13 from 3145728 to 3833856 tid 3 2024-04-26T03:39:16.021 INFO:tasks.rados.rados.0.smithi092.stdout:264: writing smithi09235267-13 from 3833856 to 4128768 tid 4 2024-04-26T03:39:16.021 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 18 current snap is 14 2024-04-26T03:39:16.021 INFO:tasks.rados.rados.0.smithi092.stdout:262: finishing rollback tid 0 to smithi09235267-24 2024-04-26T03:39:16.022 INFO:tasks.rados.rados.0.smithi092.stdout:262: finishing rollback tid 1 to smithi09235267-24 2024-04-26T03:39:16.022 INFO:tasks.rados.rados.0.smithi092.stdout:262: finishing rollback tid 2 to smithi09235267-24 2024-04-26T03:39:16.022 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 24 v 46 (ObjNum 78 snap 6 seq_num 78) dirty exists 2024-04-26T03:39:16.022 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 4 v 42 (ObjNum 56 snap 0 seq_num 56) dirty exists 2024-04-26T03:39:16.022 INFO:tasks.rados.rados.0.smithi092.stdout:265: seq_num 124 ranges {0=540672,540672=720896,1261568=786432,2048000=49152} 2024-04-26T03:39:16.022 INFO:tasks.rados.rados.0.smithi092.stdout:265: writing smithi09235267-18 from 0 to 540672 tid 1 2024-04-26T03:39:16.022 INFO:tasks.rados.rados.0.smithi092.stdout:265: writing smithi09235267-18 from 540672 to 1261568 tid 2 2024-04-26T03:39:16.022 INFO:tasks.rados.rados.0.smithi092.stdout:265: writing smithi09235267-18 from 1261568 to 2048000 tid 3 2024-04-26T03:39:16.022 INFO:tasks.rados.rados.0.smithi092.stdout:265: writing smithi09235267-18 from 2048000 to 2097152 tid 4 2024-04-26T03:39:16.022 INFO:tasks.rados.rados.0.smithi092.stdout:262: done (3 left) 2024-04-26T03:39:16.022 INFO:tasks.rados.rados.0.smithi092.stdout:263: done (2 left) 2024-04-26T03:39:16.022 INFO:tasks.rados.rados.0.smithi092.stdout:266: read oid 41 snap 13 2024-04-26T03:39:16.022 INFO:tasks.rados.rados.0.smithi092.stdout:266: expect (ObjNum 40 snap 0 seq_num 40) 2024-04-26T03:39:16.023 INFO:tasks.rados.rados.0.smithi092.stdout:267: read oid 46 snap -1 2024-04-26T03:39:16.023 INFO:tasks.rados.rados.0.smithi092.stdout:267: expect (ObjNum 120 snap 14 seq_num 120) 2024-04-26T03:39:16.023 INFO:tasks.rados.rados.0.smithi092.stdout:268: copy_from oid 22 from oid 35 current snap is 14 2024-04-26T03:39:16.023 INFO:tasks.rados.rados.0.smithi092.stdout:264: finishing write tid 1 to smithi09235267-13 2024-04-26T03:39:16.023 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 5 current snap is 14 2024-04-26T03:39:16.023 INFO:tasks.rados.rados.0.smithi092.stdout:269: seq_num 125 ranges {0=557056,557056=475136,1032192=507904,1540096=475136,2015232=98304} 2024-04-26T03:39:16.023 INFO:tasks.rados.rados.0.smithi092.stdout:269: writing smithi09235267-5 from 0 to 557056 tid 1 2024-04-26T03:39:16.023 INFO:tasks.rados.rados.0.smithi092.stdout:269: writing smithi09235267-5 from 557056 to 1032192 tid 2 2024-04-26T03:39:16.023 INFO:tasks.rados.rados.0.smithi092.stdout:269: writing smithi09235267-5 from 1032192 to 1540096 tid 3 2024-04-26T03:39:16.023 INFO:tasks.rados.rados.0.smithi092.stdout:269: writing smithi09235267-5 from 1540096 to 2015232 tid 4 2024-04-26T03:39:16.023 INFO:tasks.rados.rados.0.smithi092.stdout:269: writing smithi09235267-5 from 2015232 to 2113536 tid 5 2024-04-26T03:39:16.023 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 32 current snap is 14 2024-04-26T03:39:16.023 INFO:tasks.rados.rados.0.smithi092.stdout:270: seq_num 126 ranges {0=573440,573440=622592,1196032=622592,1818624=311296} 2024-04-26T03:39:16.023 INFO:tasks.rados.rados.0.smithi092.stdout:270: writing smithi09235267-32 from 0 to 573440 tid 1 2024-04-26T03:39:16.024 INFO:tasks.rados.rados.0.smithi092.stdout:270: writing smithi09235267-32 from 573440 to 1196032 tid 2 2024-04-26T03:39:16.024 INFO:tasks.rados.rados.0.smithi092.stdout:270: writing smithi09235267-32 from 1196032 to 1818624 tid 3 2024-04-26T03:39:16.024 INFO:tasks.rados.rados.0.smithi092.stdout:270: writing smithi09235267-32 from 1818624 to 2129920 tid 4 2024-04-26T03:39:16.024 INFO:tasks.rados.rados.0.smithi092.stdout:271: snap_create 2024-04-26T03:39:16.024 INFO:tasks.rados.rados.0.smithi092.stdout:264: finishing write tid 2 to smithi09235267-13 2024-04-26T03:39:16.024 INFO:tasks.rados.rados.0.smithi092.stdout:264: finishing write tid 3 to smithi09235267-13 2024-04-26T03:39:16.024 INFO:tasks.rados.rados.0.smithi092.stdout:265: finishing write tid 1 to smithi09235267-18 2024-04-26T03:39:16.024 INFO:tasks.rados.rados.0.smithi092.stdout:264: finishing write tid 4 to smithi09235267-13 2024-04-26T03:39:16.024 INFO:tasks.rados.rados.0.smithi092.stdout:264: finishing write tid 5 to smithi09235267-13 2024-04-26T03:39:16.024 INFO:tasks.rados.rados.0.smithi092.stdout:266: expect (ObjNum 40 snap 0 seq_num 40) 2024-04-26T03:39:16.024 INFO:tasks.rados.rados.0.smithi092.stdout:264: finishing write tid 6 to smithi09235267-13 2024-04-26T03:39:16.024 INFO:tasks.rados.rados.0.smithi092.stdout:264: oid 13 updating version 0 to 74 2024-04-26T03:39:16.024 INFO:tasks.rados.rados.0.smithi092.stdout:264: oid 13 updating version 74 to 76 2024-04-26T03:39:16.024 INFO:tasks.rados.rados.0.smithi092.stdout:264: oid 13 version 76 is already newer than 75 2024-04-26T03:39:16.025 INFO:tasks.rados.rados.0.smithi092.stdout:264: oid 13 version 76 is already newer than 73 2024-04-26T03:39:16.025 INFO:tasks.rados.rados.0.smithi092.stdout:264: oid 13 updating version 76 to 77 2024-04-26T03:39:16.025 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 13 v 77 (ObjNum 123 snap 14 seq_num 123) dirty exists 2024-04-26T03:39:16.025 INFO:tasks.rados.rados.0.smithi092.stdout:264: left oid 13 (ObjNum 123 snap 14 seq_num 123) 2024-04-26T03:39:16.025 INFO:tasks.rados.rados.0.smithi092.stdout:265: finishing write tid 2 to smithi09235267-18 2024-04-26T03:39:16.025 INFO:tasks.rados.rados.0.smithi092.stdout:265: finishing write tid 3 to smithi09235267-18 2024-04-26T03:39:16.025 INFO:tasks.rados.rados.0.smithi092.stdout:265: finishing write tid 4 to smithi09235267-18 2024-04-26T03:39:16.025 INFO:tasks.rados.rados.0.smithi092.stdout:268: finishing copy_from to smithi09235267-22 2024-04-26T03:39:16.025 INFO:tasks.rados.rados.0.smithi092.stdout:268: got expected ENOENT (src dne) 2024-04-26T03:39:16.025 INFO:tasks.rados.rados.0.smithi092.stdout:268: finishing copy_from racing read to smithi09235267-22 2024-04-26T03:39:16.025 INFO:tasks.rados.rados.0.smithi092.stdout:265: finishing write tid 5 to smithi09235267-18 2024-04-26T03:39:16.025 INFO:tasks.rados.rados.0.smithi092.stdout:267: expect (ObjNum 120 snap 14 seq_num 120) 2024-04-26T03:39:16.061 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:39:16.061 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:39:16.061 INFO:teuthology.orchestra.run.smithi028.stdout: "epoch": 60, 2024-04-26T03:39:16.061 INFO:teuthology.orchestra.run.smithi028.stdout: "fsid": "c5a13c99-5c9a-4b8b-a893-4f6f4211677a", 2024-04-26T03:39:16.061 INFO:teuthology.orchestra.run.smithi028.stdout: "created": "2024-04-26T03:38:19.896648+0000", 2024-04-26T03:39:16.062 INFO:teuthology.orchestra.run.smithi028.stdout: "modified": "2024-04-26T03:39:15.682373+0000", 2024-04-26T03:39:16.062 INFO:teuthology.orchestra.run.smithi028.stdout: "last_up_change": "2024-04-26T03:38:33.308561+0000", 2024-04-26T03:39:16.062 INFO:teuthology.orchestra.run.smithi028.stdout: "last_in_change": "2024-04-26T03:38:24.150336+0000", 2024-04-26T03:39:16.062 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-26T03:39:16.062 INFO:teuthology.orchestra.run.smithi028.stdout: "flags_num": 5806080, 2024-04-26T03:39:16.062 INFO:teuthology.orchestra.run.smithi028.stdout: "flags_set": [ 2024-04-26T03:39:16.062 INFO:teuthology.orchestra.run.smithi028.stdout: "nodeep-scrub", 2024-04-26T03:39:16.062 INFO:teuthology.orchestra.run.smithi028.stdout: "noscrub", 2024-04-26T03:39:16.062 INFO:teuthology.orchestra.run.smithi028.stdout: "pglog_hardlimit", 2024-04-26T03:39:16.062 INFO:teuthology.orchestra.run.smithi028.stdout: "purged_snapdirs", 2024-04-26T03:39:16.062 INFO:teuthology.orchestra.run.smithi028.stdout: "recovery_deletes", 2024-04-26T03:39:16.062 INFO:teuthology.orchestra.run.smithi028.stdout: "sortbitwise" 2024-04-26T03:39:16.062 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-04-26T03:39:16.063 INFO:teuthology.orchestra.run.smithi028.stdout: "crush_version": 6, 2024-04-26T03:39:16.063 INFO:teuthology.orchestra.run.smithi028.stdout: "full_ratio": 0.94999998807907104, 2024-04-26T03:39:16.063 INFO:teuthology.orchestra.run.smithi028.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-04-26T03:39:16.063 INFO:teuthology.orchestra.run.smithi028.stdout: "nearfull_ratio": 0.85000002384185791, 2024-04-26T03:39:16.063 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_snapshot": "", 2024-04-26T03:39:16.063 INFO:teuthology.orchestra.run.smithi028.stdout: "pool_max": 3, 2024-04-26T03:39:16.063 INFO:teuthology.orchestra.run.smithi028.stdout: "max_osd": 16, 2024-04-26T03:39:16.063 INFO:teuthology.orchestra.run.smithi028.stdout: "require_min_compat_client": "luminous", 2024-04-26T03:39:16.063 INFO:teuthology.orchestra.run.smithi028.stdout: "min_compat_client": "luminous", 2024-04-26T03:39:16.063 INFO:teuthology.orchestra.run.smithi028.stdout: "require_osd_release": "squid", 2024-04-26T03:39:16.063 INFO:teuthology.orchestra.run.smithi028.stdout: "allow_crimson": false, 2024-04-26T03:39:16.063 INFO:teuthology.orchestra.run.smithi028.stdout: "pools": [ 2024-04-26T03:39:16.063 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.064 INFO:teuthology.orchestra.run.smithi028.stdout: "pool": 1, 2024-04-26T03:39:16.064 INFO:teuthology.orchestra.run.smithi028.stdout: "pool_name": ".mgr", 2024-04-26T03:39:16.064 INFO:teuthology.orchestra.run.smithi028.stdout: "create_time": "2024-04-26T03:38:32.306696+0000", 2024-04-26T03:39:16.064 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-04-26T03:39:16.064 INFO:teuthology.orchestra.run.smithi028.stdout: "flags_names": "hashpspool", 2024-04-26T03:39:16.064 INFO:teuthology.orchestra.run.smithi028.stdout: "type": 1, 2024-04-26T03:39:16.064 INFO:teuthology.orchestra.run.smithi028.stdout: "size": 2, 2024-04-26T03:39:16.064 INFO:teuthology.orchestra.run.smithi028.stdout: "min_size": 1, 2024-04-26T03:39:16.064 INFO:teuthology.orchestra.run.smithi028.stdout: "crush_rule": 0, 2024-04-26T03:39:16.064 INFO:teuthology.orchestra.run.smithi028.stdout: "peering_crush_bucket_count": 0, 2024-04-26T03:39:16.064 INFO:teuthology.orchestra.run.smithi028.stdout: "peering_crush_bucket_target": 0, 2024-04-26T03:39:16.064 INFO:teuthology.orchestra.run.smithi028.stdout: "peering_crush_bucket_barrier": 0, 2024-04-26T03:39:16.064 INFO:teuthology.orchestra.run.smithi028.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-26T03:39:16.065 INFO:teuthology.orchestra.run.smithi028.stdout: "object_hash": 2, 2024-04-26T03:39:16.065 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_autoscale_mode": "off", 2024-04-26T03:39:16.065 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_num": 1, 2024-04-26T03:39:16.065 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_placement_num": 1, 2024-04-26T03:39:16.065 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_placement_num_target": 1, 2024-04-26T03:39:16.065 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_num_target": 1, 2024-04-26T03:39:16.065 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_num_pending": 1, 2024-04-26T03:39:16.065 INFO:teuthology.orchestra.run.smithi028.stdout: "last_pg_merge_meta": { 2024-04-26T03:39:16.065 INFO:teuthology.orchestra.run.smithi028.stdout: "source_pgid": "0.0", 2024-04-26T03:39:16.065 INFO:teuthology.orchestra.run.smithi028.stdout: "ready_epoch": 0, 2024-04-26T03:39:16.065 INFO:teuthology.orchestra.run.smithi028.stdout: "last_epoch_started": 0, 2024-04-26T03:39:16.065 INFO:teuthology.orchestra.run.smithi028.stdout: "last_epoch_clean": 0, 2024-04-26T03:39:16.065 INFO:teuthology.orchestra.run.smithi028.stdout: "source_version": "0'0", 2024-04-26T03:39:16.066 INFO:teuthology.orchestra.run.smithi028.stdout: "target_version": "0'0" 2024-04-26T03:39:16.066 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.066 INFO:teuthology.orchestra.run.smithi028.stdout: "last_change": "26", 2024-04-26T03:39:16.066 INFO:teuthology.orchestra.run.smithi028.stdout: "last_force_op_resend": "0", 2024-04-26T03:39:16.066 INFO:teuthology.orchestra.run.smithi028.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-26T03:39:16.066 INFO:teuthology.orchestra.run.smithi028.stdout: "last_force_op_resend_preluminous": "0", 2024-04-26T03:39:16.066 INFO:teuthology.orchestra.run.smithi028.stdout: "auid": 0, 2024-04-26T03:39:16.066 INFO:teuthology.orchestra.run.smithi028.stdout: "snap_mode": "selfmanaged", 2024-04-26T03:39:16.066 INFO:teuthology.orchestra.run.smithi028.stdout: "snap_seq": 0, 2024-04-26T03:39:16.066 INFO:teuthology.orchestra.run.smithi028.stdout: "snap_epoch": 0, 2024-04-26T03:39:16.066 INFO:teuthology.orchestra.run.smithi028.stdout: "pool_snaps": [], 2024-04-26T03:39:16.066 INFO:teuthology.orchestra.run.smithi028.stdout: "removed_snaps": "[]", 2024-04-26T03:39:16.066 INFO:teuthology.orchestra.run.smithi028.stdout: "quota_max_bytes": 0, 2024-04-26T03:39:16.067 INFO:teuthology.orchestra.run.smithi028.stdout: "quota_max_objects": 0, 2024-04-26T03:39:16.067 INFO:teuthology.orchestra.run.smithi028.stdout: "tiers": [], 2024-04-26T03:39:16.067 INFO:teuthology.orchestra.run.smithi028.stdout: "tier_of": -1, 2024-04-26T03:39:16.067 INFO:teuthology.orchestra.run.smithi028.stdout: "read_tier": -1, 2024-04-26T03:39:16.067 INFO:teuthology.orchestra.run.smithi028.stdout: "write_tier": -1, 2024-04-26T03:39:16.067 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_mode": "none", 2024-04-26T03:39:16.067 INFO:teuthology.orchestra.run.smithi028.stdout: "target_max_bytes": 0, 2024-04-26T03:39:16.067 INFO:teuthology.orchestra.run.smithi028.stdout: "target_max_objects": 0, 2024-04-26T03:39:16.067 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-26T03:39:16.067 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-26T03:39:16.067 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-26T03:39:16.067 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_min_flush_age": 0, 2024-04-26T03:39:16.068 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_min_evict_age": 0, 2024-04-26T03:39:16.068 INFO:teuthology.orchestra.run.smithi028.stdout: "erasure_code_profile": "", 2024-04-26T03:39:16.068 INFO:teuthology.orchestra.run.smithi028.stdout: "hit_set_params": { 2024-04-26T03:39:16.068 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "none" 2024-04-26T03:39:16.068 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.068 INFO:teuthology.orchestra.run.smithi028.stdout: "hit_set_period": 0, 2024-04-26T03:39:16.068 INFO:teuthology.orchestra.run.smithi028.stdout: "hit_set_count": 0, 2024-04-26T03:39:16.068 INFO:teuthology.orchestra.run.smithi028.stdout: "use_gmt_hitset": true, 2024-04-26T03:39:16.068 INFO:teuthology.orchestra.run.smithi028.stdout: "min_read_recency_for_promote": 0, 2024-04-26T03:39:16.068 INFO:teuthology.orchestra.run.smithi028.stdout: "min_write_recency_for_promote": 0, 2024-04-26T03:39:16.068 INFO:teuthology.orchestra.run.smithi028.stdout: "hit_set_grade_decay_rate": 0, 2024-04-26T03:39:16.068 INFO:teuthology.orchestra.run.smithi028.stdout: "hit_set_search_last_n": 0, 2024-04-26T03:39:16.068 INFO:teuthology.orchestra.run.smithi028.stdout: "grade_table": [], 2024-04-26T03:39:16.068 INFO:teuthology.orchestra.run.smithi028.stdout: "stripe_width": 0, 2024-04-26T03:39:16.069 INFO:teuthology.orchestra.run.smithi028.stdout: "expected_num_objects": 0, 2024-04-26T03:39:16.069 INFO:teuthology.orchestra.run.smithi028.stdout: "fast_read": false, 2024-04-26T03:39:16.069 INFO:teuthology.orchestra.run.smithi028.stdout: "options": { 2024-04-26T03:39:16.069 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_num_max": 32, 2024-04-26T03:39:16.069 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_num_min": 1 2024-04-26T03:39:16.069 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.069 INFO:teuthology.orchestra.run.smithi028.stdout: "application_metadata": { 2024-04-26T03:39:16.069 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": {} 2024-04-26T03:39:16.069 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.069 INFO:teuthology.orchestra.run.smithi028.stdout: "read_balance": { 2024-04-26T03:39:16.069 INFO:teuthology.orchestra.run.smithi028.stdout: "score_type": "Fair distribution", 2024-04-26T03:39:16.069 INFO:teuthology.orchestra.run.smithi028.stdout: "score_acting": 15.380000114440918, 2024-04-26T03:39:16.069 INFO:teuthology.orchestra.run.smithi028.stdout: "score_stable": 15.380000114440918, 2024-04-26T03:39:16.070 INFO:teuthology.orchestra.run.smithi028.stdout: "optimal_score": 0.12999999523162842, 2024-04-26T03:39:16.070 INFO:teuthology.orchestra.run.smithi028.stdout: "raw_score_acting": 2, 2024-04-26T03:39:16.070 INFO:teuthology.orchestra.run.smithi028.stdout: "raw_score_stable": 2, 2024-04-26T03:39:16.070 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity_weighted": 1, 2024-04-26T03:39:16.070 INFO:teuthology.orchestra.run.smithi028.stdout: "average_primary_affinity": 1, 2024-04-26T03:39:16.070 INFO:teuthology.orchestra.run.smithi028.stdout: "average_primary_affinity_weighted": 1 2024-04-26T03:39:16.070 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.070 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.070 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.070 INFO:teuthology.orchestra.run.smithi028.stdout: "pool": 2, 2024-04-26T03:39:16.070 INFO:teuthology.orchestra.run.smithi028.stdout: "pool_name": "rbd", 2024-04-26T03:39:16.070 INFO:teuthology.orchestra.run.smithi028.stdout: "create_time": "2024-04-26T03:38:34.592633+0000", 2024-04-26T03:39:16.070 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-04-26T03:39:16.071 INFO:teuthology.orchestra.run.smithi028.stdout: "flags_names": "hashpspool", 2024-04-26T03:39:16.071 INFO:teuthology.orchestra.run.smithi028.stdout: "type": 1, 2024-04-26T03:39:16.071 INFO:teuthology.orchestra.run.smithi028.stdout: "size": 2, 2024-04-26T03:39:16.071 INFO:teuthology.orchestra.run.smithi028.stdout: "min_size": 1, 2024-04-26T03:39:16.071 INFO:teuthology.orchestra.run.smithi028.stdout: "crush_rule": 0, 2024-04-26T03:39:16.071 INFO:teuthology.orchestra.run.smithi028.stdout: "peering_crush_bucket_count": 0, 2024-04-26T03:39:16.071 INFO:teuthology.orchestra.run.smithi028.stdout: "peering_crush_bucket_target": 0, 2024-04-26T03:39:16.071 INFO:teuthology.orchestra.run.smithi028.stdout: "peering_crush_bucket_barrier": 0, 2024-04-26T03:39:16.071 INFO:teuthology.orchestra.run.smithi028.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-26T03:39:16.071 INFO:teuthology.orchestra.run.smithi028.stdout: "object_hash": 2, 2024-04-26T03:39:16.071 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_autoscale_mode": "off", 2024-04-26T03:39:16.072 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_num": 8, 2024-04-26T03:39:16.072 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_placement_num": 8, 2024-04-26T03:39:16.072 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_placement_num_target": 8, 2024-04-26T03:39:16.072 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_num_target": 8, 2024-04-26T03:39:16.072 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_num_pending": 8, 2024-04-26T03:39:16.072 INFO:teuthology.orchestra.run.smithi028.stdout: "last_pg_merge_meta": { 2024-04-26T03:39:16.072 INFO:teuthology.orchestra.run.smithi028.stdout: "source_pgid": "0.0", 2024-04-26T03:39:16.072 INFO:teuthology.orchestra.run.smithi028.stdout: "ready_epoch": 0, 2024-04-26T03:39:16.072 INFO:teuthology.orchestra.run.smithi028.stdout: "last_epoch_started": 0, 2024-04-26T03:39:16.072 INFO:teuthology.orchestra.run.smithi028.stdout: "last_epoch_clean": 0, 2024-04-26T03:39:16.072 INFO:teuthology.orchestra.run.smithi028.stdout: "source_version": "0'0", 2024-04-26T03:39:16.073 INFO:teuthology.orchestra.run.smithi028.stdout: "target_version": "0'0" 2024-04-26T03:39:16.073 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.073 INFO:teuthology.orchestra.run.smithi028.stdout: "last_change": "29", 2024-04-26T03:39:16.073 INFO:teuthology.orchestra.run.smithi028.stdout: "last_force_op_resend": "0", 2024-04-26T03:39:16.073 INFO:teuthology.orchestra.run.smithi028.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-26T03:39:16.073 INFO:teuthology.orchestra.run.smithi028.stdout: "last_force_op_resend_preluminous": "0", 2024-04-26T03:39:16.073 INFO:teuthology.orchestra.run.smithi028.stdout: "auid": 0, 2024-04-26T03:39:16.073 INFO:teuthology.orchestra.run.smithi028.stdout: "snap_mode": "selfmanaged", 2024-04-26T03:39:16.073 INFO:teuthology.orchestra.run.smithi028.stdout: "snap_seq": 0, 2024-04-26T03:39:16.073 INFO:teuthology.orchestra.run.smithi028.stdout: "snap_epoch": 0, 2024-04-26T03:39:16.073 INFO:teuthology.orchestra.run.smithi028.stdout: "pool_snaps": [], 2024-04-26T03:39:16.074 INFO:teuthology.orchestra.run.smithi028.stdout: "removed_snaps": "[]", 2024-04-26T03:39:16.074 INFO:teuthology.orchestra.run.smithi028.stdout: "quota_max_bytes": 0, 2024-04-26T03:39:16.074 INFO:teuthology.orchestra.run.smithi028.stdout: "quota_max_objects": 0, 2024-04-26T03:39:16.074 INFO:teuthology.orchestra.run.smithi028.stdout: "tiers": [], 2024-04-26T03:39:16.074 INFO:teuthology.orchestra.run.smithi028.stdout: "tier_of": -1, 2024-04-26T03:39:16.074 INFO:teuthology.orchestra.run.smithi028.stdout: "read_tier": -1, 2024-04-26T03:39:16.074 INFO:teuthology.orchestra.run.smithi028.stdout: "write_tier": -1, 2024-04-26T03:39:16.074 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_mode": "none", 2024-04-26T03:39:16.074 INFO:teuthology.orchestra.run.smithi028.stdout: "target_max_bytes": 0, 2024-04-26T03:39:16.074 INFO:teuthology.orchestra.run.smithi028.stdout: "target_max_objects": 0, 2024-04-26T03:39:16.074 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-26T03:39:16.075 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-26T03:39:16.075 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-26T03:39:16.075 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_min_flush_age": 0, 2024-04-26T03:39:16.075 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_min_evict_age": 0, 2024-04-26T03:39:16.075 INFO:teuthology.orchestra.run.smithi028.stdout: "erasure_code_profile": "", 2024-04-26T03:39:16.075 INFO:teuthology.orchestra.run.smithi028.stdout: "hit_set_params": { 2024-04-26T03:39:16.075 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "none" 2024-04-26T03:39:16.075 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.075 INFO:teuthology.orchestra.run.smithi028.stdout: "hit_set_period": 0, 2024-04-26T03:39:16.075 INFO:teuthology.orchestra.run.smithi028.stdout: "hit_set_count": 0, 2024-04-26T03:39:16.075 INFO:teuthology.orchestra.run.smithi028.stdout: "use_gmt_hitset": true, 2024-04-26T03:39:16.075 INFO:teuthology.orchestra.run.smithi028.stdout: "min_read_recency_for_promote": 0, 2024-04-26T03:39:16.075 INFO:teuthology.orchestra.run.smithi028.stdout: "min_write_recency_for_promote": 0, 2024-04-26T03:39:16.076 INFO:teuthology.orchestra.run.smithi028.stdout: "hit_set_grade_decay_rate": 0, 2024-04-26T03:39:16.076 INFO:teuthology.orchestra.run.smithi028.stdout: "hit_set_search_last_n": 0, 2024-04-26T03:39:16.076 INFO:teuthology.orchestra.run.smithi028.stdout: "grade_table": [], 2024-04-26T03:39:16.076 INFO:teuthology.orchestra.run.smithi028.stdout: "stripe_width": 0, 2024-04-26T03:39:16.076 INFO:teuthology.orchestra.run.smithi028.stdout: "expected_num_objects": 0, 2024-04-26T03:39:16.076 INFO:teuthology.orchestra.run.smithi028.stdout: "fast_read": false, 2024-04-26T03:39:16.076 INFO:teuthology.orchestra.run.smithi028.stdout: "options": {}, 2024-04-26T03:39:16.076 INFO:teuthology.orchestra.run.smithi028.stdout: "application_metadata": { 2024-04-26T03:39:16.076 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd": {} 2024-04-26T03:39:16.076 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.076 INFO:teuthology.orchestra.run.smithi028.stdout: "read_balance": { 2024-04-26T03:39:16.077 INFO:teuthology.orchestra.run.smithi028.stdout: "score_type": "Fair distribution", 2024-04-26T03:39:16.077 INFO:teuthology.orchestra.run.smithi028.stdout: "score_acting": 4, 2024-04-26T03:39:16.077 INFO:teuthology.orchestra.run.smithi028.stdout: "score_stable": 4, 2024-04-26T03:39:16.077 INFO:teuthology.orchestra.run.smithi028.stdout: "optimal_score": 0.75, 2024-04-26T03:39:16.077 INFO:teuthology.orchestra.run.smithi028.stdout: "raw_score_acting": 3, 2024-04-26T03:39:16.077 INFO:teuthology.orchestra.run.smithi028.stdout: "raw_score_stable": 3, 2024-04-26T03:39:16.077 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity_weighted": 1, 2024-04-26T03:39:16.077 INFO:teuthology.orchestra.run.smithi028.stdout: "average_primary_affinity": 1, 2024-04-26T03:39:16.077 INFO:teuthology.orchestra.run.smithi028.stdout: "average_primary_affinity_weighted": 1 2024-04-26T03:39:16.077 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.077 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.077 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.078 INFO:teuthology.orchestra.run.smithi028.stdout: "pool": 3, 2024-04-26T03:39:16.078 INFO:teuthology.orchestra.run.smithi028.stdout: "pool_name": "unique_pool_0", 2024-04-26T03:39:16.078 INFO:teuthology.orchestra.run.smithi028.stdout: "create_time": "2024-04-26T03:38:47.497176+0000", 2024-04-26T03:39:16.078 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 8193, 2024-04-26T03:39:16.078 INFO:teuthology.orchestra.run.smithi028.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-04-26T03:39:16.078 INFO:teuthology.orchestra.run.smithi028.stdout: "type": 3, 2024-04-26T03:39:16.078 INFO:teuthology.orchestra.run.smithi028.stdout: "size": 7, 2024-04-26T03:39:16.078 INFO:teuthology.orchestra.run.smithi028.stdout: "min_size": 5, 2024-04-26T03:39:16.078 INFO:teuthology.orchestra.run.smithi028.stdout: "crush_rule": 1, 2024-04-26T03:39:16.078 INFO:teuthology.orchestra.run.smithi028.stdout: "peering_crush_bucket_count": 0, 2024-04-26T03:39:16.078 INFO:teuthology.orchestra.run.smithi028.stdout: "peering_crush_bucket_target": 0, 2024-04-26T03:39:16.078 INFO:teuthology.orchestra.run.smithi028.stdout: "peering_crush_bucket_barrier": 0, 2024-04-26T03:39:16.079 INFO:teuthology.orchestra.run.smithi028.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-26T03:39:16.079 INFO:teuthology.orchestra.run.smithi028.stdout: "object_hash": 2, 2024-04-26T03:39:16.079 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_autoscale_mode": "off", 2024-04-26T03:39:16.079 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_num": 25, 2024-04-26T03:39:16.079 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_placement_num": 20, 2024-04-26T03:39:16.079 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_placement_num_target": 16, 2024-04-26T03:39:16.079 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_num_target": 16, 2024-04-26T03:39:16.079 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_num_pending": 25, 2024-04-26T03:39:16.079 INFO:teuthology.orchestra.run.smithi028.stdout: "last_pg_merge_meta": { 2024-04-26T03:39:16.079 INFO:teuthology.orchestra.run.smithi028.stdout: "source_pgid": "3.19", 2024-04-26T03:39:16.079 INFO:teuthology.orchestra.run.smithi028.stdout: "ready_epoch": 59, 2024-04-26T03:39:16.079 INFO:teuthology.orchestra.run.smithi028.stdout: "last_epoch_started": 58, 2024-04-26T03:39:16.079 INFO:teuthology.orchestra.run.smithi028.stdout: "last_epoch_clean": 58, 2024-04-26T03:39:16.080 INFO:teuthology.orchestra.run.smithi028.stdout: "source_version": "52'24", 2024-04-26T03:39:16.080 INFO:teuthology.orchestra.run.smithi028.stdout: "target_version": "52'11" 2024-04-26T03:39:16.080 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.080 INFO:teuthology.orchestra.run.smithi028.stdout: "last_change": "60", 2024-04-26T03:39:16.080 INFO:teuthology.orchestra.run.smithi028.stdout: "last_force_op_resend": "0", 2024-04-26T03:39:16.080 INFO:teuthology.orchestra.run.smithi028.stdout: "last_force_op_resend_prenautilus": "59", 2024-04-26T03:39:16.080 INFO:teuthology.orchestra.run.smithi028.stdout: "last_force_op_resend_preluminous": "57", 2024-04-26T03:39:16.080 INFO:teuthology.orchestra.run.smithi028.stdout: "auid": 0, 2024-04-26T03:39:16.080 INFO:teuthology.orchestra.run.smithi028.stdout: "snap_mode": "selfmanaged", 2024-04-26T03:39:16.080 INFO:teuthology.orchestra.run.smithi028.stdout: "snap_seq": 15, 2024-04-26T03:39:16.080 INFO:teuthology.orchestra.run.smithi028.stdout: "snap_epoch": 60, 2024-04-26T03:39:16.081 INFO:teuthology.orchestra.run.smithi028.stdout: "pool_snaps": [], 2024-04-26T03:39:16.081 INFO:teuthology.orchestra.run.smithi028.stdout: "removed_snaps": "[]", 2024-04-26T03:39:16.081 INFO:teuthology.orchestra.run.smithi028.stdout: "quota_max_bytes": 0, 2024-04-26T03:39:16.081 INFO:teuthology.orchestra.run.smithi028.stdout: "quota_max_objects": 0, 2024-04-26T03:39:16.081 INFO:teuthology.orchestra.run.smithi028.stdout: "tiers": [], 2024-04-26T03:39:16.081 INFO:teuthology.orchestra.run.smithi028.stdout: "tier_of": -1, 2024-04-26T03:39:16.081 INFO:teuthology.orchestra.run.smithi028.stdout: "read_tier": -1, 2024-04-26T03:39:16.081 INFO:teuthology.orchestra.run.smithi028.stdout: "write_tier": -1, 2024-04-26T03:39:16.081 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_mode": "none", 2024-04-26T03:39:16.081 INFO:teuthology.orchestra.run.smithi028.stdout: "target_max_bytes": 0, 2024-04-26T03:39:16.081 INFO:teuthology.orchestra.run.smithi028.stdout: "target_max_objects": 0, 2024-04-26T03:39:16.081 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-26T03:39:16.082 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-26T03:39:16.082 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-26T03:39:16.082 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_min_flush_age": 0, 2024-04-26T03:39:16.082 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_min_evict_age": 0, 2024-04-26T03:39:16.082 INFO:teuthology.orchestra.run.smithi028.stdout: "erasure_code_profile": "shecprofile", 2024-04-26T03:39:16.082 INFO:teuthology.orchestra.run.smithi028.stdout: "hit_set_params": { 2024-04-26T03:39:16.082 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "none" 2024-04-26T03:39:16.082 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.082 INFO:teuthology.orchestra.run.smithi028.stdout: "hit_set_period": 0, 2024-04-26T03:39:16.082 INFO:teuthology.orchestra.run.smithi028.stdout: "hit_set_count": 0, 2024-04-26T03:39:16.082 INFO:teuthology.orchestra.run.smithi028.stdout: "use_gmt_hitset": true, 2024-04-26T03:39:16.082 INFO:teuthology.orchestra.run.smithi028.stdout: "min_read_recency_for_promote": 0, 2024-04-26T03:39:16.082 INFO:teuthology.orchestra.run.smithi028.stdout: "min_write_recency_for_promote": 0, 2024-04-26T03:39:16.083 INFO:teuthology.orchestra.run.smithi028.stdout: "hit_set_grade_decay_rate": 0, 2024-04-26T03:39:16.083 INFO:teuthology.orchestra.run.smithi028.stdout: "hit_set_search_last_n": 0, 2024-04-26T03:39:16.083 INFO:teuthology.orchestra.run.smithi028.stdout: "grade_table": [], 2024-04-26T03:39:16.083 INFO:teuthology.orchestra.run.smithi028.stdout: "stripe_width": 16384, 2024-04-26T03:39:16.083 INFO:teuthology.orchestra.run.smithi028.stdout: "expected_num_objects": 0, 2024-04-26T03:39:16.083 INFO:teuthology.orchestra.run.smithi028.stdout: "fast_read": false, 2024-04-26T03:39:16.083 INFO:teuthology.orchestra.run.smithi028.stdout: "options": {}, 2024-04-26T03:39:16.083 INFO:teuthology.orchestra.run.smithi028.stdout: "application_metadata": { 2024-04-26T03:39:16.083 INFO:teuthology.orchestra.run.smithi028.stdout: "rados": {} 2024-04-26T03:39:16.083 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.083 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.083 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-04-26T03:39:16.084 INFO:teuthology.orchestra.run.smithi028.stdout: "osds": [ 2024-04-26T03:39:16.084 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.084 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 0, 2024-04-26T03:39:16.084 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "31b0e7a9-455d-4874-9c60-26efac8d0b1e", 2024-04-26T03:39:16.084 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:39:16.084 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:39:16.084 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:39:16.084 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:39:16.084 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:39:16.084 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:39:16.084 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 23, 2024-04-26T03:39:16.084 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 44, 2024-04-26T03:39:16.085 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:39:16.085 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:39:16.085 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:39:16.085 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.085 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.085 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.085 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6800", 2024-04-26T03:39:16.085 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377656090 2024-04-26T03:39:16.085 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.085 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.085 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.085 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6801", 2024-04-26T03:39:16.085 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377656090 2024-04-26T03:39:16.085 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.086 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.086 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.086 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:39:16.086 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.086 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.086 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.086 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6802", 2024-04-26T03:39:16.086 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377656090 2024-04-26T03:39:16.086 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.087 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.087 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.087 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6803", 2024-04-26T03:39:16.087 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377656090 2024-04-26T03:39:16.087 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.087 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.087 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.087 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:39:16.087 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.087 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.087 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.087 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6806", 2024-04-26T03:39:16.087 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377656090 2024-04-26T03:39:16.087 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.087 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.088 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.088 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6807", 2024-04-26T03:39:16.088 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377656090 2024-04-26T03:39:16.088 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.088 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.088 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.088 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:39:16.088 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.088 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.088 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.088 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6804", 2024-04-26T03:39:16.088 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377656090 2024-04-26T03:39:16.088 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.089 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.089 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.089 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6805", 2024-04-26T03:39:16.089 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377656090 2024-04-26T03:39:16.089 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.089 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.089 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.089 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.28:6801/2377656090", 2024-04-26T03:39:16.089 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.28:6803/2377656090", 2024-04-26T03:39:16.089 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.28:6807/2377656090", 2024-04-26T03:39:16.089 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.28:6805/2377656090", 2024-04-26T03:39:16.089 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:39:16.089 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:39:16.089 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:39:16.090 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.090 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.090 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.090 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 1, 2024-04-26T03:39:16.090 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "52819ace-a070-4822-b33e-3c54466e61ff", 2024-04-26T03:39:16.090 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:39:16.090 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:39:16.090 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:39:16.090 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:39:16.090 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:39:16.090 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:39:16.090 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 23, 2024-04-26T03:39:16.090 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 44, 2024-04-26T03:39:16.091 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:39:16.091 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:39:16.091 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:39:16.091 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.091 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.091 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.091 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6800", 2024-04-26T03:39:16.091 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1678150206 2024-04-26T03:39:16.091 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.091 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.091 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.091 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6801", 2024-04-26T03:39:16.091 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1678150206 2024-04-26T03:39:16.091 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.092 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.092 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.092 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:39:16.092 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.092 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.092 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.092 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6802", 2024-04-26T03:39:16.092 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1678150206 2024-04-26T03:39:16.092 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.092 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.092 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.092 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6803", 2024-04-26T03:39:16.092 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1678150206 2024-04-26T03:39:16.093 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.093 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.093 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.093 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:39:16.093 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.093 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.093 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.093 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6806", 2024-04-26T03:39:16.093 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1678150206 2024-04-26T03:39:16.093 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.093 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.093 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.093 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6807", 2024-04-26T03:39:16.093 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1678150206 2024-04-26T03:39:16.094 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.094 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.094 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.094 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:39:16.094 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.094 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.094 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.094 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6804", 2024-04-26T03:39:16.094 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1678150206 2024-04-26T03:39:16.094 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.094 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.094 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.094 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6805", 2024-04-26T03:39:16.095 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1678150206 2024-04-26T03:39:16.095 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.095 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.095 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.095 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.42:6801/1678150206", 2024-04-26T03:39:16.095 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.42:6803/1678150206", 2024-04-26T03:39:16.095 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.42:6807/1678150206", 2024-04-26T03:39:16.095 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.42:6805/1678150206", 2024-04-26T03:39:16.095 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:39:16.095 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:39:16.095 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:39:16.095 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.095 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.096 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.096 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 2, 2024-04-26T03:39:16.096 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "858f8ac1-2edc-43d5-914c-c1914bdc0c9a", 2024-04-26T03:39:16.096 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:39:16.096 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:39:16.096 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:39:16.096 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:39:16.096 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:39:16.096 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:39:16.096 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 23, 2024-04-26T03:39:16.096 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 0, 2024-04-26T03:39:16.096 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:39:16.097 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:39:16.097 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:39:16.097 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.097 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.097 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.097 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6800", 2024-04-26T03:39:16.097 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 714535415 2024-04-26T03:39:16.097 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.097 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.097 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.097 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6801", 2024-04-26T03:39:16.097 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 714535415 2024-04-26T03:39:16.097 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.098 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.098 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.098 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:39:16.098 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.098 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.098 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.098 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6802", 2024-04-26T03:39:16.098 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 714535415 2024-04-26T03:39:16.098 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.098 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.098 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.098 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6803", 2024-04-26T03:39:16.098 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 714535415 2024-04-26T03:39:16.098 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.099 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.099 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.099 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:39:16.099 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.099 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.099 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.099 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6806", 2024-04-26T03:39:16.099 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 714535415 2024-04-26T03:39:16.099 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.099 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.099 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.099 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6807", 2024-04-26T03:39:16.099 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 714535415 2024-04-26T03:39:16.100 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.100 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.100 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.100 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:39:16.100 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.100 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.100 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.100 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6804", 2024-04-26T03:39:16.100 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 714535415 2024-04-26T03:39:16.100 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.100 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.100 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.100 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6805", 2024-04-26T03:39:16.101 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 714535415 2024-04-26T03:39:16.101 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.101 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.101 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.101 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.66:6801/714535415", 2024-04-26T03:39:16.101 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.66:6803/714535415", 2024-04-26T03:39:16.101 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.66:6807/714535415", 2024-04-26T03:39:16.101 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.66:6805/714535415", 2024-04-26T03:39:16.101 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:39:16.101 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:39:16.101 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:39:16.101 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.101 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.101 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.102 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 3, 2024-04-26T03:39:16.102 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "531a2e30-3351-4f84-804b-5fb5050b84ed", 2024-04-26T03:39:16.102 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:39:16.102 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:39:16.102 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:39:16.102 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:39:16.102 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:39:16.102 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:39:16.102 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 24, 2024-04-26T03:39:16.102 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 32, 2024-04-26T03:39:16.102 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:39:16.102 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:39:16.102 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:39:16.103 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.103 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.103 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.103 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6810", 2024-04-26T03:39:16.103 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377983625 2024-04-26T03:39:16.103 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.103 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.103 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.103 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6811", 2024-04-26T03:39:16.103 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377983625 2024-04-26T03:39:16.103 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.103 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.103 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.103 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:39:16.104 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.104 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.104 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.104 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6812", 2024-04-26T03:39:16.104 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377983625 2024-04-26T03:39:16.104 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.104 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.104 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.104 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6813", 2024-04-26T03:39:16.104 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377983625 2024-04-26T03:39:16.104 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.104 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.104 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.105 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:39:16.105 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.105 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.105 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.105 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6816", 2024-04-26T03:39:16.105 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377983625 2024-04-26T03:39:16.105 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.105 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.105 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.105 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6817", 2024-04-26T03:39:16.105 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377983625 2024-04-26T03:39:16.105 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.105 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.105 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.106 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:39:16.106 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.106 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.106 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.106 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6814", 2024-04-26T03:39:16.106 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377983625 2024-04-26T03:39:16.106 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.106 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.106 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.106 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6815", 2024-04-26T03:39:16.106 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377983625 2024-04-26T03:39:16.106 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.106 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.106 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.107 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.92:6811/2377983625", 2024-04-26T03:39:16.107 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.92:6813/2377983625", 2024-04-26T03:39:16.107 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.92:6817/2377983625", 2024-04-26T03:39:16.107 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.92:6815/2377983625", 2024-04-26T03:39:16.107 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:39:16.107 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:39:16.107 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:39:16.107 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.107 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.107 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.107 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 4, 2024-04-26T03:39:16.107 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "6d38ffda-40eb-4fd7-9300-2a09f2673e66", 2024-04-26T03:39:16.107 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:39:16.108 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:39:16.108 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:39:16.108 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:39:16.108 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:39:16.108 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:39:16.108 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 23, 2024-04-26T03:39:16.108 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 44, 2024-04-26T03:39:16.108 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:39:16.108 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:39:16.108 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:39:16.108 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.108 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.108 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.108 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6808", 2024-04-26T03:39:16.108 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 183752455 2024-04-26T03:39:16.109 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.109 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.109 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.109 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6809", 2024-04-26T03:39:16.109 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 183752455 2024-04-26T03:39:16.109 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.109 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.109 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.109 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:39:16.109 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.109 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.109 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.109 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6810", 2024-04-26T03:39:16.110 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 183752455 2024-04-26T03:39:16.110 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.110 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.110 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.110 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6811", 2024-04-26T03:39:16.110 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 183752455 2024-04-26T03:39:16.110 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.110 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.110 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.110 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:39:16.111 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.111 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.111 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.111 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6814", 2024-04-26T03:39:16.111 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 183752455 2024-04-26T03:39:16.111 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.111 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.111 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.111 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6815", 2024-04-26T03:39:16.111 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 183752455 2024-04-26T03:39:16.111 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.111 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.111 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.111 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:39:16.112 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.112 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.112 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.112 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6812", 2024-04-26T03:39:16.112 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 183752455 2024-04-26T03:39:16.112 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.112 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.112 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.112 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6813", 2024-04-26T03:39:16.112 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 183752455 2024-04-26T03:39:16.112 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.112 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.112 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.112 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.28:6809/183752455", 2024-04-26T03:39:16.113 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.28:6811/183752455", 2024-04-26T03:39:16.113 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.28:6815/183752455", 2024-04-26T03:39:16.113 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.28:6813/183752455", 2024-04-26T03:39:16.113 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:39:16.113 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:39:16.113 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:39:16.113 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.113 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.113 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.113 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 5, 2024-04-26T03:39:16.113 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "361aa40d-e53b-42af-afd4-81fbcac28725", 2024-04-26T03:39:16.113 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:39:16.114 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:39:16.114 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:39:16.114 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:39:16.114 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:39:16.114 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:39:16.114 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 24, 2024-04-26T03:39:16.114 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 51, 2024-04-26T03:39:16.114 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:39:16.114 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:39:16.114 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:39:16.114 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.114 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.114 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.114 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6808", 2024-04-26T03:39:16.115 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2012261939 2024-04-26T03:39:16.115 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.115 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.115 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.115 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6809", 2024-04-26T03:39:16.115 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2012261939 2024-04-26T03:39:16.115 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.115 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.115 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.115 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:39:16.115 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.115 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.115 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.115 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6810", 2024-04-26T03:39:16.115 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2012261939 2024-04-26T03:39:16.116 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.116 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.116 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.116 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6811", 2024-04-26T03:39:16.116 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2012261939 2024-04-26T03:39:16.116 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.116 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.116 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.116 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:39:16.116 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.116 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.116 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.116 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6814", 2024-04-26T03:39:16.116 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2012261939 2024-04-26T03:39:16.117 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.117 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.117 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.117 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6815", 2024-04-26T03:39:16.117 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2012261939 2024-04-26T03:39:16.117 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.117 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.117 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.117 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:39:16.117 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.117 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.117 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.117 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6812", 2024-04-26T03:39:16.117 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2012261939 2024-04-26T03:39:16.118 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.118 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.118 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.118 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6813", 2024-04-26T03:39:16.118 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2012261939 2024-04-26T03:39:16.118 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.118 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.118 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.118 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.42:6809/2012261939", 2024-04-26T03:39:16.118 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.42:6811/2012261939", 2024-04-26T03:39:16.118 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.42:6815/2012261939", 2024-04-26T03:39:16.119 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.42:6813/2012261939", 2024-04-26T03:39:16.119 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:39:16.119 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:39:16.119 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:39:16.119 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.119 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.119 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.119 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 6, 2024-04-26T03:39:16.119 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "f610ad3e-1fd8-42bd-af4b-0d79c3bb9cf8", 2024-04-26T03:39:16.119 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:39:16.119 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:39:16.119 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:39:16.120 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:39:16.120 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:39:16.120 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:39:16.120 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 22, 2024-04-26T03:39:16.120 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 48, 2024-04-26T03:39:16.120 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:39:16.120 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:39:16.120 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:39:16.120 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.120 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.120 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.120 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6808", 2024-04-26T03:39:16.120 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3533460882 2024-04-26T03:39:16.121 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.121 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.121 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.121 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6809", 2024-04-26T03:39:16.121 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3533460882 2024-04-26T03:39:16.121 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.121 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.121 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.121 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:39:16.121 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.121 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.121 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.122 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6810", 2024-04-26T03:39:16.122 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3533460882 2024-04-26T03:39:16.122 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.122 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.122 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.122 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6811", 2024-04-26T03:39:16.122 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3533460882 2024-04-26T03:39:16.122 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.122 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.122 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.122 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:39:16.123 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.123 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.123 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.123 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6814", 2024-04-26T03:39:16.123 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3533460882 2024-04-26T03:39:16.123 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.123 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.123 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.123 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6815", 2024-04-26T03:39:16.123 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3533460882 2024-04-26T03:39:16.123 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.123 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.124 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.124 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:39:16.124 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.124 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.124 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.124 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6812", 2024-04-26T03:39:16.124 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3533460882 2024-04-26T03:39:16.124 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.124 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.124 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.124 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6813", 2024-04-26T03:39:16.124 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3533460882 2024-04-26T03:39:16.125 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.125 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.125 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.125 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.66:6809/3533460882", 2024-04-26T03:39:16.125 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.66:6811/3533460882", 2024-04-26T03:39:16.125 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.66:6815/3533460882", 2024-04-26T03:39:16.125 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.66:6813/3533460882", 2024-04-26T03:39:16.125 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:39:16.125 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:39:16.125 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:39:16.125 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.125 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.125 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.125 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 7, 2024-04-26T03:39:16.126 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "a4bae6fb-e087-4563-a129-6938e18eed8c", 2024-04-26T03:39:16.126 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:39:16.126 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:39:16.126 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:39:16.126 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:39:16.126 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:39:16.126 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:39:16.126 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 22, 2024-04-26T03:39:16.126 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 32, 2024-04-26T03:39:16.126 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:39:16.126 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:39:16.126 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:39:16.126 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.126 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.127 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.127 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6802", 2024-04-26T03:39:16.127 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1760163933 2024-04-26T03:39:16.127 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.127 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.127 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.127 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6803", 2024-04-26T03:39:16.127 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1760163933 2024-04-26T03:39:16.127 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.127 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.127 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.127 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:39:16.127 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.128 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.128 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.128 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6804", 2024-04-26T03:39:16.128 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1760163933 2024-04-26T03:39:16.128 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.128 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.128 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.128 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6805", 2024-04-26T03:39:16.128 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1760163933 2024-04-26T03:39:16.128 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.128 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.128 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.128 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:39:16.129 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.129 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.129 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.129 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6808", 2024-04-26T03:39:16.129 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1760163933 2024-04-26T03:39:16.129 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.129 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.129 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.129 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6809", 2024-04-26T03:39:16.129 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1760163933 2024-04-26T03:39:16.129 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.129 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.129 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.129 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:39:16.129 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.130 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.130 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.130 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6806", 2024-04-26T03:39:16.130 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1760163933 2024-04-26T03:39:16.130 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.130 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.130 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.130 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6807", 2024-04-26T03:39:16.130 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1760163933 2024-04-26T03:39:16.130 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.130 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.130 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.131 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.92:6803/1760163933", 2024-04-26T03:39:16.131 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.92:6805/1760163933", 2024-04-26T03:39:16.131 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.92:6809/1760163933", 2024-04-26T03:39:16.131 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.92:6807/1760163933", 2024-04-26T03:39:16.131 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:39:16.131 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:39:16.131 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:39:16.131 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.131 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.131 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.131 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 8, 2024-04-26T03:39:16.131 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "479f3885-452e-4231-bdc2-b79668384ba6", 2024-04-26T03:39:16.132 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:39:16.132 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:39:16.132 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:39:16.132 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:39:16.132 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:39:16.132 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:39:16.132 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 22, 2024-04-26T03:39:16.132 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 44, 2024-04-26T03:39:16.132 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:39:16.132 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:39:16.132 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:39:16.132 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.132 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.133 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.133 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6816", 2024-04-26T03:39:16.133 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4127602407 2024-04-26T03:39:16.133 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.133 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.133 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.133 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6817", 2024-04-26T03:39:16.133 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4127602407 2024-04-26T03:39:16.133 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.133 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.133 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.133 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:39:16.133 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.134 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.134 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.134 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6818", 2024-04-26T03:39:16.134 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4127602407 2024-04-26T03:39:16.134 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.134 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.134 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.134 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6819", 2024-04-26T03:39:16.134 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4127602407 2024-04-26T03:39:16.134 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.134 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.135 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.135 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:39:16.135 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.135 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.135 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.135 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6822", 2024-04-26T03:39:16.135 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4127602407 2024-04-26T03:39:16.135 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.135 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.135 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.135 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6823", 2024-04-26T03:39:16.136 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4127602407 2024-04-26T03:39:16.136 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.136 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.136 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.136 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:39:16.136 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.136 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.136 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.136 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6820", 2024-04-26T03:39:16.136 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4127602407 2024-04-26T03:39:16.136 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.136 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.136 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.136 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6821", 2024-04-26T03:39:16.137 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4127602407 2024-04-26T03:39:16.137 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.137 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.137 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.137 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.28:6817/4127602407", 2024-04-26T03:39:16.137 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.28:6819/4127602407", 2024-04-26T03:39:16.137 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.28:6823/4127602407", 2024-04-26T03:39:16.137 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.28:6821/4127602407", 2024-04-26T03:39:16.137 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:39:16.137 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:39:16.137 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:39:16.137 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.138 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.138 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.138 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 9, 2024-04-26T03:39:16.138 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "9a603a47-7c5a-47d4-a298-56b3ac991892", 2024-04-26T03:39:16.138 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:39:16.138 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:39:16.138 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:39:16.138 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:39:16.138 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:39:16.138 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:39:16.139 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 22, 2024-04-26T03:39:16.139 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 26, 2024-04-26T03:39:16.139 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:39:16.139 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:39:16.139 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:39:16.139 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.139 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.139 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.139 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6816", 2024-04-26T03:39:16.139 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4168729845 2024-04-26T03:39:16.139 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.140 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.140 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.140 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6818", 2024-04-26T03:39:16.140 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4168729845 2024-04-26T03:39:16.140 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.140 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.140 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.140 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:39:16.140 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.140 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.140 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.140 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6820", 2024-04-26T03:39:16.140 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4168729845 2024-04-26T03:39:16.141 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.141 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.141 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.141 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6822", 2024-04-26T03:39:16.141 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4168729845 2024-04-26T03:39:16.141 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.141 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.141 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.141 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:39:16.141 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.141 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.141 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.141 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6828", 2024-04-26T03:39:16.141 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4168729845 2024-04-26T03:39:16.142 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.142 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.142 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.142 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6830", 2024-04-26T03:39:16.142 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4168729845 2024-04-26T03:39:16.142 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.142 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.142 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.142 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:39:16.142 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.142 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.142 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.142 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6824", 2024-04-26T03:39:16.143 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4168729845 2024-04-26T03:39:16.143 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.143 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.143 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.143 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6826", 2024-04-26T03:39:16.143 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4168729845 2024-04-26T03:39:16.143 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.143 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.143 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.143 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.42:6818/4168729845", 2024-04-26T03:39:16.143 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.42:6822/4168729845", 2024-04-26T03:39:16.143 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.42:6830/4168729845", 2024-04-26T03:39:16.144 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.42:6826/4168729845", 2024-04-26T03:39:16.144 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:39:16.144 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:39:16.144 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:39:16.144 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.144 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.144 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.144 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 10, 2024-04-26T03:39:16.144 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "1cd42945-6ee1-49f0-b17b-4093dd36b4ec", 2024-04-26T03:39:16.144 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:39:16.144 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:39:16.145 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:39:16.145 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:39:16.145 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:39:16.145 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:39:16.145 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 22, 2024-04-26T03:39:16.145 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 44, 2024-04-26T03:39:16.145 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:39:16.145 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:39:16.145 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:39:16.145 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.145 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.145 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.145 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6816", 2024-04-26T03:39:16.145 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4180572718 2024-04-26T03:39:16.146 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.146 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.146 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.146 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6817", 2024-04-26T03:39:16.146 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4180572718 2024-04-26T03:39:16.146 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.146 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.146 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.146 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:39:16.146 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.146 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.146 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.146 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6818", 2024-04-26T03:39:16.146 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4180572718 2024-04-26T03:39:16.147 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.147 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.147 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.147 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6819", 2024-04-26T03:39:16.147 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4180572718 2024-04-26T03:39:16.147 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.147 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.147 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.147 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:39:16.147 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.147 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.147 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.148 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6822", 2024-04-26T03:39:16.148 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4180572718 2024-04-26T03:39:16.148 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.148 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.148 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.148 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6823", 2024-04-26T03:39:16.148 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4180572718 2024-04-26T03:39:16.148 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.148 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.148 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.148 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:39:16.148 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.148 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.148 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.149 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6820", 2024-04-26T03:39:16.149 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4180572718 2024-04-26T03:39:16.149 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.149 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.149 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.149 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6821", 2024-04-26T03:39:16.149 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4180572718 2024-04-26T03:39:16.149 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.149 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.149 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.149 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.66:6817/4180572718", 2024-04-26T03:39:16.149 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.66:6819/4180572718", 2024-04-26T03:39:16.149 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.66:6823/4180572718", 2024-04-26T03:39:16.149 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.66:6821/4180572718", 2024-04-26T03:39:16.149 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:39:16.150 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:39:16.150 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:39:16.150 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.150 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.150 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.150 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 11, 2024-04-26T03:39:16.150 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "b41e8100-1d4f-4b7a-991a-7a27bc29feec", 2024-04-26T03:39:16.150 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:39:16.150 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:39:16.150 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:39:16.150 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:39:16.150 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:39:16.150 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:39:16.151 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 21, 2024-04-26T03:39:16.151 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 55, 2024-04-26T03:39:16.151 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:39:16.151 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:39:16.151 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:39:16.151 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.151 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.151 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.151 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6818", 2024-04-26T03:39:16.151 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 295713815 2024-04-26T03:39:16.151 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.151 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.151 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.152 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6819", 2024-04-26T03:39:16.152 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 295713815 2024-04-26T03:39:16.152 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.152 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.152 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.152 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:39:16.152 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.152 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.152 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.152 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6820", 2024-04-26T03:39:16.152 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 295713815 2024-04-26T03:39:16.152 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.152 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.152 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.152 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6821", 2024-04-26T03:39:16.153 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 295713815 2024-04-26T03:39:16.153 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.153 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.153 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.153 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:39:16.153 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.153 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.153 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.153 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6824", 2024-04-26T03:39:16.153 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 295713815 2024-04-26T03:39:16.153 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.153 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.153 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.154 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6825", 2024-04-26T03:39:16.154 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 295713815 2024-04-26T03:39:16.154 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.154 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.154 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.154 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:39:16.154 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.154 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.154 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.154 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6822", 2024-04-26T03:39:16.154 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 295713815 2024-04-26T03:39:16.154 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.154 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.155 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.155 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6823", 2024-04-26T03:39:16.155 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 295713815 2024-04-26T03:39:16.155 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.155 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.155 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.155 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.92:6819/295713815", 2024-04-26T03:39:16.155 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.92:6821/295713815", 2024-04-26T03:39:16.155 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.92:6825/295713815", 2024-04-26T03:39:16.155 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.92:6823/295713815", 2024-04-26T03:39:16.155 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:39:16.155 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:39:16.155 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:39:16.156 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.156 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.156 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.156 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 12, 2024-04-26T03:39:16.156 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "458cf150-32e1-449e-b533-ac6ab5226f85", 2024-04-26T03:39:16.156 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:39:16.156 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:39:16.156 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:39:16.156 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:39:16.156 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:39:16.156 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:39:16.156 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 22, 2024-04-26T03:39:16.156 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 0, 2024-04-26T03:39:16.156 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:39:16.157 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:39:16.157 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:39:16.157 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.157 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.157 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.157 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6824", 2024-04-26T03:39:16.157 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2081141953 2024-04-26T03:39:16.157 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.157 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.157 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.157 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6825", 2024-04-26T03:39:16.157 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2081141953 2024-04-26T03:39:16.157 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.157 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.158 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.158 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:39:16.158 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.158 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.158 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.158 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6826", 2024-04-26T03:39:16.158 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2081141953 2024-04-26T03:39:16.158 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.158 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.158 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.158 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6827", 2024-04-26T03:39:16.159 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2081141953 2024-04-26T03:39:16.159 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.159 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.159 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.159 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:39:16.159 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.159 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.159 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.159 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6830", 2024-04-26T03:39:16.159 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2081141953 2024-04-26T03:39:16.159 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.159 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.159 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.159 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6831", 2024-04-26T03:39:16.160 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2081141953 2024-04-26T03:39:16.160 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.160 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.160 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.160 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:39:16.160 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.160 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.160 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.160 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6828", 2024-04-26T03:39:16.160 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2081141953 2024-04-26T03:39:16.160 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.160 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.160 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.160 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6829", 2024-04-26T03:39:16.160 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2081141953 2024-04-26T03:39:16.161 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.161 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.161 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.161 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.28:6825/2081141953", 2024-04-26T03:39:16.161 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.28:6827/2081141953", 2024-04-26T03:39:16.161 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.28:6831/2081141953", 2024-04-26T03:39:16.161 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.28:6829/2081141953", 2024-04-26T03:39:16.161 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:39:16.161 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:39:16.161 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:39:16.161 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.161 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.161 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.161 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 13, 2024-04-26T03:39:16.161 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "f008462e-a735-4845-9ecf-dd7d4a6941af", 2024-04-26T03:39:16.162 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:39:16.162 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:39:16.162 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:39:16.162 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:39:16.162 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:39:16.162 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:39:16.162 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 22, 2024-04-26T03:39:16.162 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 59, 2024-04-26T03:39:16.162 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:39:16.162 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:39:16.162 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:39:16.162 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.162 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.163 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.163 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6817", 2024-04-26T03:39:16.163 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 301360893 2024-04-26T03:39:16.163 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.163 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.163 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.163 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6819", 2024-04-26T03:39:16.163 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 301360893 2024-04-26T03:39:16.163 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.163 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.163 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.163 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:39:16.163 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.163 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.163 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.164 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6821", 2024-04-26T03:39:16.164 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 301360893 2024-04-26T03:39:16.164 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.164 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.164 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.164 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6823", 2024-04-26T03:39:16.164 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 301360893 2024-04-26T03:39:16.164 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.164 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.164 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.164 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:39:16.164 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.164 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.164 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.165 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6829", 2024-04-26T03:39:16.165 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 301360893 2024-04-26T03:39:16.165 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.165 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.165 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.165 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6831", 2024-04-26T03:39:16.165 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 301360893 2024-04-26T03:39:16.165 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.165 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.165 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.165 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:39:16.165 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.165 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.166 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.166 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6825", 2024-04-26T03:39:16.166 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 301360893 2024-04-26T03:39:16.166 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.166 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.166 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.166 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6827", 2024-04-26T03:39:16.166 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 301360893 2024-04-26T03:39:16.166 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.166 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.166 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.167 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.42:6819/301360893", 2024-04-26T03:39:16.167 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.42:6823/301360893", 2024-04-26T03:39:16.167 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.42:6831/301360893", 2024-04-26T03:39:16.167 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.42:6827/301360893", 2024-04-26T03:39:16.167 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:39:16.167 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:39:16.167 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:39:16.167 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.167 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.167 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.167 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 14, 2024-04-26T03:39:16.167 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "6f66ae97-eb8a-48eb-9bcc-20f65fbf690b", 2024-04-26T03:39:16.167 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:39:16.168 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:39:16.168 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:39:16.168 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:39:16.168 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:39:16.168 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:39:16.168 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 22, 2024-04-26T03:39:16.168 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 53, 2024-04-26T03:39:16.168 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:39:16.168 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:39:16.168 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:39:16.168 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.168 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.169 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.169 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6824", 2024-04-26T03:39:16.169 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1534905602 2024-04-26T03:39:16.169 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.169 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.169 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.169 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6825", 2024-04-26T03:39:16.169 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1534905602 2024-04-26T03:39:16.169 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.169 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.169 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.169 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:39:16.169 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.169 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.170 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.170 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6826", 2024-04-26T03:39:16.170 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1534905602 2024-04-26T03:39:16.170 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.170 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.170 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.170 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6827", 2024-04-26T03:39:16.170 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1534905602 2024-04-26T03:39:16.170 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.170 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.170 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.171 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:39:16.171 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.171 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.171 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.171 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6830", 2024-04-26T03:39:16.171 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1534905602 2024-04-26T03:39:16.171 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.171 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.171 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.171 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6831", 2024-04-26T03:39:16.171 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1534905602 2024-04-26T03:39:16.171 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.171 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.172 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.172 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:39:16.172 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.172 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.172 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.172 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6828", 2024-04-26T03:39:16.172 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1534905602 2024-04-26T03:39:16.172 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.172 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.172 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.172 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6829", 2024-04-26T03:39:16.172 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1534905602 2024-04-26T03:39:16.172 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.172 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.173 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.173 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.66:6825/1534905602", 2024-04-26T03:39:16.173 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.66:6827/1534905602", 2024-04-26T03:39:16.173 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.66:6831/1534905602", 2024-04-26T03:39:16.173 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.66:6829/1534905602", 2024-04-26T03:39:16.173 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:39:16.173 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:39:16.173 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:39:16.173 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.173 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.173 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.173 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 15, 2024-04-26T03:39:16.173 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "b856bfc8-253b-4112-be5c-f67d61d01768", 2024-04-26T03:39:16.173 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:39:16.174 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:39:16.174 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:39:16.174 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:39:16.174 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:39:16.174 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:39:16.174 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 22, 2024-04-26T03:39:16.174 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 44, 2024-04-26T03:39:16.174 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:39:16.174 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:39:16.174 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:39:16.174 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.174 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.174 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.174 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6826", 2024-04-26T03:39:16.175 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3982634323 2024-04-26T03:39:16.175 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.175 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.175 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.175 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6827", 2024-04-26T03:39:16.175 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3982634323 2024-04-26T03:39:16.175 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.175 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.175 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.175 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:39:16.175 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.175 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.175 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.175 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6828", 2024-04-26T03:39:16.176 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3982634323 2024-04-26T03:39:16.176 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.176 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.176 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.176 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6829", 2024-04-26T03:39:16.176 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3982634323 2024-04-26T03:39:16.176 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.176 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.176 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.176 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:39:16.176 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.176 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.176 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.176 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6832", 2024-04-26T03:39:16.177 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3982634323 2024-04-26T03:39:16.177 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.177 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.177 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.177 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6833", 2024-04-26T03:39:16.177 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3982634323 2024-04-26T03:39:16.177 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.177 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.177 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.177 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:39:16.177 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:16.177 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.177 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:16.177 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6830", 2024-04-26T03:39:16.178 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3982634323 2024-04-26T03:39:16.178 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.178 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.178 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:16.178 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6831", 2024-04-26T03:39:16.178 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3982634323 2024-04-26T03:39:16.178 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.178 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.178 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.178 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.92:6827/3982634323", 2024-04-26T03:39:16.178 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.92:6829/3982634323", 2024-04-26T03:39:16.178 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.92:6833/3982634323", 2024-04-26T03:39:16.179 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.92:6831/3982634323", 2024-04-26T03:39:16.179 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:39:16.179 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:39:16.179 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:39:16.179 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.179 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.179 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-04-26T03:39:16.179 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_xinfo": [ 2024-04-26T03:39:16.179 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.179 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 0, 2024-04-26T03:39:16.179 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:39:16.179 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:39:16.179 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:39:16.179 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:39:16.180 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:39:16.180 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.265053+0000", 2024-04-26T03:39:16.180 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:39:16.180 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.180 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.180 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 1, 2024-04-26T03:39:16.180 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:39:16.180 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:39:16.180 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:39:16.180 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:39:16.180 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:39:16.180 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.243502+0000", 2024-04-26T03:39:16.180 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:39:16.180 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.181 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.181 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 2, 2024-04-26T03:39:16.181 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:39:16.181 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:39:16.181 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:39:16.181 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:39:16.181 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:39:16.181 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.252440+0000", 2024-04-26T03:39:16.181 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:39:16.181 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.181 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.181 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 3, 2024-04-26T03:39:16.181 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:39:16.182 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:39:16.182 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:39:16.182 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:39:16.182 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:39:16.182 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.295331+0000", 2024-04-26T03:39:16.182 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:39:16.182 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.182 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.182 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 4, 2024-04-26T03:39:16.182 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:39:16.182 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:39:16.182 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:39:16.183 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:39:16.183 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:39:16.183 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.213076+0000", 2024-04-26T03:39:16.183 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:39:16.183 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.183 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.183 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 5, 2024-04-26T03:39:16.183 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:39:16.183 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:39:16.183 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:39:16.183 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:39:16.183 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:39:16.183 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.281193+0000", 2024-04-26T03:39:16.183 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:39:16.184 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.184 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.184 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 6, 2024-04-26T03:39:16.184 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:39:16.184 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:39:16.184 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:39:16.184 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:39:16.184 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:39:16.184 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.315801+0000", 2024-04-26T03:39:16.184 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:39:16.184 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.184 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.184 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 7, 2024-04-26T03:39:16.185 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:39:16.185 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:39:16.185 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:39:16.185 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:39:16.185 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:39:16.185 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.329538+0000", 2024-04-26T03:39:16.185 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:39:16.185 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.185 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.185 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 8, 2024-04-26T03:39:16.185 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:39:16.185 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:39:16.185 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:39:16.186 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:39:16.186 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:39:16.186 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.293303+0000", 2024-04-26T03:39:16.186 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:39:16.186 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.186 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.186 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 9, 2024-04-26T03:39:16.186 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:39:16.186 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:39:16.187 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:39:16.187 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:39:16.187 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:39:16.187 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.334962+0000", 2024-04-26T03:39:16.187 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:39:16.187 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.187 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.187 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 10, 2024-04-26T03:39:16.187 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:39:16.187 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:39:16.187 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:39:16.187 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:39:16.188 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:39:16.188 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.233032+0000", 2024-04-26T03:39:16.188 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:39:16.188 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.188 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.188 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 11, 2024-04-26T03:39:16.188 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:39:16.188 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:39:16.188 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:39:16.188 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:39:16.188 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:39:16.188 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.305138+0000", 2024-04-26T03:39:16.188 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:39:16.188 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.189 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.189 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 12, 2024-04-26T03:39:16.189 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:39:16.189 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:39:16.189 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:39:16.189 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:39:16.189 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:39:16.189 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.351964+0000", 2024-04-26T03:39:16.189 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:39:16.189 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.189 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.189 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 13, 2024-04-26T03:39:16.189 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:39:16.190 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:39:16.190 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:39:16.190 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:39:16.190 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:39:16.190 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.241086+0000", 2024-04-26T03:39:16.190 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:39:16.190 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.190 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.190 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 14, 2024-04-26T03:39:16.190 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:39:16.190 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:39:16.190 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:39:16.190 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:39:16.191 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:39:16.191 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.352515+0000", 2024-04-26T03:39:16.191 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:39:16.191 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.191 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.191 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 15, 2024-04-26T03:39:16.191 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:39:16.191 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:39:16.191 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:39:16.191 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:39:16.191 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:39:16.191 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.357462+0000", 2024-04-26T03:39:16.191 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:39:16.191 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.192 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-04-26T03:39:16.192 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_upmap": [], 2024-04-26T03:39:16.192 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_upmap_items": [ 2024-04-26T03:39:16.192 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.192 INFO:teuthology.orchestra.run.smithi028.stdout: "pgid": "2.2", 2024-04-26T03:39:16.192 INFO:teuthology.orchestra.run.smithi028.stdout: "mappings": [ 2024-04-26T03:39:16.192 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.192 INFO:teuthology.orchestra.run.smithi028.stdout: "from": 3, 2024-04-26T03:39:16.192 INFO:teuthology.orchestra.run.smithi028.stdout: "to": 11 2024-04-26T03:39:16.192 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.192 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.192 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.192 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-04-26T03:39:16.192 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_upmap_primaries": [], 2024-04-26T03:39:16.193 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_temp": [], 2024-04-26T03:39:16.193 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_temp": [], 2024-04-26T03:39:16.193 INFO:teuthology.orchestra.run.smithi028.stdout: "blocklist": {}, 2024-04-26T03:39:16.193 INFO:teuthology.orchestra.run.smithi028.stdout: "range_blocklist": {}, 2024-04-26T03:39:16.193 INFO:teuthology.orchestra.run.smithi028.stdout: "erasure_code_profiles": { 2024-04-26T03:39:16.193 INFO:teuthology.orchestra.run.smithi028.stdout: "default": { 2024-04-26T03:39:16.193 INFO:teuthology.orchestra.run.smithi028.stdout: "crush-failure-domain": "osd", 2024-04-26T03:39:16.193 INFO:teuthology.orchestra.run.smithi028.stdout: "k": "2", 2024-04-26T03:39:16.193 INFO:teuthology.orchestra.run.smithi028.stdout: "m": "1", 2024-04-26T03:39:16.193 INFO:teuthology.orchestra.run.smithi028.stdout: "plugin": "jerasure", 2024-04-26T03:39:16.193 INFO:teuthology.orchestra.run.smithi028.stdout: "technique": "reed_sol_van" 2024-04-26T03:39:16.194 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.194 INFO:teuthology.orchestra.run.smithi028.stdout: "shecprofile": { 2024-04-26T03:39:16.194 INFO:teuthology.orchestra.run.smithi028.stdout: "c": "2", 2024-04-26T03:39:16.194 INFO:teuthology.orchestra.run.smithi028.stdout: "crush-device-class": "", 2024-04-26T03:39:16.194 INFO:teuthology.orchestra.run.smithi028.stdout: "crush-failure-domain": "osd", 2024-04-26T03:39:16.194 INFO:teuthology.orchestra.run.smithi028.stdout: "crush-num-failure-domains": "0", 2024-04-26T03:39:16.194 INFO:teuthology.orchestra.run.smithi028.stdout: "crush-osds-per-failure-domain": "0", 2024-04-26T03:39:16.194 INFO:teuthology.orchestra.run.smithi028.stdout: "crush-root": "default", 2024-04-26T03:39:16.194 INFO:teuthology.orchestra.run.smithi028.stdout: "k": "4", 2024-04-26T03:39:16.194 INFO:teuthology.orchestra.run.smithi028.stdout: "m": "3", 2024-04-26T03:39:16.194 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "shecprofile", 2024-04-26T03:39:16.194 INFO:teuthology.orchestra.run.smithi028.stdout: "plugin": "shec", 2024-04-26T03:39:16.195 INFO:teuthology.orchestra.run.smithi028.stdout: "technique": "multiple" 2024-04-26T03:39:16.195 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.195 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:16.195 INFO:teuthology.orchestra.run.smithi028.stdout: "removed_snaps_queue": [ 2024-04-26T03:39:16.195 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.195 INFO:teuthology.orchestra.run.smithi028.stdout: "pool": 3, 2024-04-26T03:39:16.195 INFO:teuthology.orchestra.run.smithi028.stdout: "snaps": [ 2024-04-26T03:39:16.195 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.195 INFO:teuthology.orchestra.run.smithi028.stdout: "begin": 9, 2024-04-26T03:39:16.195 INFO:teuthology.orchestra.run.smithi028.stdout: "length": 3 2024-04-26T03:39:16.195 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.195 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.196 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.196 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-04-26T03:39:16.196 INFO:teuthology.orchestra.run.smithi028.stdout: "new_removed_snaps": [ 2024-04-26T03:39:16.196 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.196 INFO:teuthology.orchestra.run.smithi028.stdout: "pool": 3, 2024-04-26T03:39:16.196 INFO:teuthology.orchestra.run.smithi028.stdout: "snaps": [ 2024-04-26T03:39:16.196 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:16.196 INFO:teuthology.orchestra.run.smithi028.stdout: "begin": 11, 2024-04-26T03:39:16.196 INFO:teuthology.orchestra.run.smithi028.stdout: "length": 1 2024-04-26T03:39:16.196 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.196 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:16.196 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.196 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-04-26T03:39:16.197 INFO:teuthology.orchestra.run.smithi028.stdout: "new_purged_snaps": [], 2024-04-26T03:39:16.197 INFO:teuthology.orchestra.run.smithi028.stdout: "crush_node_flags": {}, 2024-04-26T03:39:16.197 INFO:teuthology.orchestra.run.smithi028.stdout: "device_class_flags": {}, 2024-04-26T03:39:16.197 INFO:teuthology.orchestra.run.smithi028.stdout: "stretch_mode": { 2024-04-26T03:39:16.197 INFO:teuthology.orchestra.run.smithi028.stdout: "stretch_mode_enabled": false, 2024-04-26T03:39:16.197 INFO:teuthology.orchestra.run.smithi028.stdout: "stretch_bucket_count": 0, 2024-04-26T03:39:16.197 INFO:teuthology.orchestra.run.smithi028.stdout: "degraded_stretch_mode": 0, 2024-04-26T03:39:16.197 INFO:teuthology.orchestra.run.smithi028.stdout: "recovering_stretch_mode": 0, 2024-04-26T03:39:16.197 INFO:teuthology.orchestra.run.smithi028.stdout: "stretch_mode_bucket": 0 2024-04-26T03:39:16.197 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:16.197 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:39:16.200 INFO:tasks.rados.rados.0.smithi092.stdout:265: finishing write tid 6 to smithi09235267-18 2024-04-26T03:39:16.200 INFO:tasks.rados.rados.0.smithi092.stdout:265: oid 18 updating version 0 to 43 2024-04-26T03:39:16.200 INFO:tasks.rados.rados.0.smithi092.stdout:265: oid 18 updating version 43 to 45 2024-04-26T03:39:16.200 INFO:tasks.rados.rados.0.smithi092.stdout:265: oid 18 version 45 is already newer than 44 2024-04-26T03:39:16.200 INFO:tasks.rados.rados.0.smithi092.stdout:265: oid 18 version 45 is already newer than 41 2024-04-26T03:39:16.200 INFO:tasks.rados.rados.0.smithi092.stdout:265: oid 18 version 45 is already newer than 42 2024-04-26T03:39:16.200 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 18 v 45 (ObjNum 124 snap 14 seq_num 124) dirty exists 2024-04-26T03:39:16.200 INFO:tasks.rados.rados.0.smithi092.stdout:265: left oid 18 (ObjNum 124 snap 14 seq_num 124) 2024-04-26T03:39:16.200 INFO:tasks.rados.rados.0.smithi092.stdout:269: finishing write tid 1 to smithi09235267-5 2024-04-26T03:39:16.200 INFO:tasks.rados.rados.0.smithi092.stdout:269: finishing write tid 2 to smithi09235267-5 2024-04-26T03:39:16.201 INFO:tasks.rados.rados.0.smithi092.stdout:269: finishing write tid 3 to smithi09235267-5 2024-04-26T03:39:16.201 INFO:tasks.rados.rados.0.smithi092.stdout:269: finishing write tid 4 to smithi09235267-5 2024-04-26T03:39:16.201 INFO:tasks.rados.rados.0.smithi092.stdout:269: finishing write tid 5 to smithi09235267-5 2024-04-26T03:39:16.201 INFO:tasks.rados.rados.0.smithi092.stdout:269: finishing write tid 6 to smithi09235267-5 2024-04-26T03:39:16.201 INFO:tasks.rados.rados.0.smithi092.stdout:269: finishing write tid 7 to smithi09235267-5 2024-04-26T03:39:16.201 INFO:tasks.rados.rados.0.smithi092.stdout:269: oid 5 updating version 0 to 22 2024-04-26T03:39:16.201 INFO:tasks.rados.rados.0.smithi092.stdout:269: oid 5 updating version 22 to 24 2024-04-26T03:39:16.201 INFO:tasks.rados.rados.0.smithi092.stdout:269: oid 5 version 24 is already newer than 20 2024-04-26T03:39:16.201 INFO:tasks.rados.rados.0.smithi092.stdout:269: oid 5 updating version 24 to 25 2024-04-26T03:39:16.201 INFO:tasks.rados.rados.0.smithi092.stdout:269: oid 5 version 25 is already newer than 21 2024-04-26T03:39:16.201 INFO:tasks.rados.rados.0.smithi092.stdout:269: oid 5 version 25 is already newer than 23 2024-04-26T03:39:16.201 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 5 v 25 (ObjNum 125 snap 14 seq_num 125) dirty exists 2024-04-26T03:39:16.202 INFO:tasks.rados.rados.0.smithi092.stdout:269: left oid 5 (ObjNum 125 snap 14 seq_num 125) 2024-04-26T03:39:16.202 INFO:tasks.rados.rados.0.smithi092.stdout:270: finishing write tid 1 to smithi09235267-32 2024-04-26T03:39:16.202 INFO:tasks.rados.rados.0.smithi092.stdout:270: finishing write tid 2 to smithi09235267-32 2024-04-26T03:39:16.202 INFO:tasks.rados.rados.0.smithi092.stdout:270: finishing write tid 3 to smithi09235267-32 2024-04-26T03:39:16.202 INFO:tasks.rados.rados.0.smithi092.stdout:270: finishing write tid 4 to smithi09235267-32 2024-04-26T03:39:16.202 INFO:tasks.rados.rados.0.smithi092.stdout:270: finishing write tid 5 to smithi09235267-32 2024-04-26T03:39:16.202 INFO:tasks.rados.rados.0.smithi092.stdout:270: finishing write tid 6 to smithi09235267-32 2024-04-26T03:39:16.202 INFO:tasks.rados.rados.0.smithi092.stdout:270: oid 32 updating version 0 to 52 2024-04-26T03:39:16.202 INFO:tasks.rados.rados.0.smithi092.stdout:270: oid 32 updating version 52 to 54 2024-04-26T03:39:16.202 INFO:tasks.rados.rados.0.smithi092.stdout:270: oid 32 updating version 54 to 55 2024-04-26T03:39:16.202 INFO:tasks.rados.rados.0.smithi092.stdout:270: oid 32 version 55 is already newer than 53 2024-04-26T03:39:16.202 INFO:tasks.rados.rados.0.smithi092.stdout:270: oid 32 updating version 55 to 56 2024-04-26T03:39:16.202 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 32 v 56 (ObjNum 126 snap 14 seq_num 126) dirty exists 2024-04-26T03:39:16.203 INFO:tasks.rados.rados.0.smithi092.stdout:270: left oid 32 (ObjNum 126 snap 14 seq_num 126) 2024-04-26T03:39:16.204 INFO:tasks.thrashosds.thrasher:j is {'epoch': 60, 'fsid': 'c5a13c99-5c9a-4b8b-a893-4f6f4211677a', 'created': '2024-04-26T03:38:19.896648+0000', 'modified': '2024-04-26T03:39:15.682373+0000', 'last_up_change': '2024-04-26T03:38:33.308561+0000', 'last_in_change': '2024-04-26T03:38:24.150336+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 16, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T03:38:32.306696+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 15.380000114440918, 'score_stable': 15.380000114440918, 'optimal_score': 0.12999999523162842, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-04-26T03:38:34.592633+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '29', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 4, 'score_stable': 4, 'optimal_score': 0.75, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'unique_pool_0', 'create_time': '2024-04-26T03:38:47.497176+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 3, 'size': 7, 'min_size': 5, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 25, 'pg_placement_num': 20, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 25, 'last_pg_merge_meta': {'source_pgid': '3.19', 'ready_epoch': 59, 'last_epoch_started': 58, 'last_epoch_clean': 58, 'source_version': "52'24", 'target_version': "52'11"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '59', 'last_force_op_resend_preluminous': '57', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 60, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'shecprofile', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 16384, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': '31b0e7a9-455d-4874-9c60-26efac8d0b1e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6800', 'nonce': 2377656090}, {'type': 'v1', 'addr': '172.21.15.28:6801', 'nonce': 2377656090}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6802', 'nonce': 2377656090}, {'type': 'v1', 'addr': '172.21.15.28:6803', 'nonce': 2377656090}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6806', 'nonce': 2377656090}, {'type': 'v1', 'addr': '172.21.15.28:6807', 'nonce': 2377656090}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6804', 'nonce': 2377656090}, {'type': 'v1', 'addr': '172.21.15.28:6805', 'nonce': 2377656090}]}, 'public_addr': '172.21.15.28:6801/2377656090', 'cluster_addr': '172.21.15.28:6803/2377656090', 'heartbeat_back_addr': '172.21.15.28:6807/2377656090', 'heartbeat_front_addr': '172.21.15.28:6805/2377656090', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '52819ace-a070-4822-b33e-3c54466e61ff', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6800', 'nonce': 1678150206}, {'type': 'v1', 'addr': '172.21.15.42:6801', 'nonce': 1678150206}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6802', 'nonce': 1678150206}, {'type': 'v1', 'addr': '172.21.15.42:6803', 'nonce': 1678150206}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6806', 'nonce': 1678150206}, {'type': 'v1', 'addr': '172.21.15.42:6807', 'nonce': 1678150206}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6804', 'nonce': 1678150206}, {'type': 'v1', 'addr': '172.21.15.42:6805', 'nonce': 1678150206}]}, 'public_addr': '172.21.15.42:6801/1678150206', 'cluster_addr': '172.21.15.42:6803/1678150206', 'heartbeat_back_addr': '172.21.15.42:6807/1678150206', 'heartbeat_front_addr': '172.21.15.42:6805/1678150206', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '858f8ac1-2edc-43d5-914c-c1914bdc0c9a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6800', 'nonce': 714535415}, {'type': 'v1', 'addr': '172.21.15.66:6801', 'nonce': 714535415}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6802', 'nonce': 714535415}, {'type': 'v1', 'addr': '172.21.15.66:6803', 'nonce': 714535415}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6806', 'nonce': 714535415}, {'type': 'v1', 'addr': '172.21.15.66:6807', 'nonce': 714535415}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6804', 'nonce': 714535415}, {'type': 'v1', 'addr': '172.21.15.66:6805', 'nonce': 714535415}]}, 'public_addr': '172.21.15.66:6801/714535415', 'cluster_addr': '172.21.15.66:6803/714535415', 'heartbeat_back_addr': '172.21.15.66:6807/714535415', 'heartbeat_front_addr': '172.21.15.66:6805/714535415', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '531a2e30-3351-4f84-804b-5fb5050b84ed', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 24, 'up_thru': 32, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6810', 'nonce': 2377983625}, {'type': 'v1', 'addr': '172.21.15.92:6811', 'nonce': 2377983625}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6812', 'nonce': 2377983625}, {'type': 'v1', 'addr': '172.21.15.92:6813', 'nonce': 2377983625}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6816', 'nonce': 2377983625}, {'type': 'v1', 'addr': '172.21.15.92:6817', 'nonce': 2377983625}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6814', 'nonce': 2377983625}, {'type': 'v1', 'addr': '172.21.15.92:6815', 'nonce': 2377983625}]}, 'public_addr': '172.21.15.92:6811/2377983625', 'cluster_addr': '172.21.15.92:6813/2377983625', 'heartbeat_back_addr': '172.21.15.92:6817/2377983625', 'heartbeat_front_addr': '172.21.15.92:6815/2377983625', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '6d38ffda-40eb-4fd7-9300-2a09f2673e66', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6808', 'nonce': 183752455}, {'type': 'v1', 'addr': '172.21.15.28:6809', 'nonce': 183752455}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6810', 'nonce': 183752455}, {'type': 'v1', 'addr': '172.21.15.28:6811', 'nonce': 183752455}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6814', 'nonce': 183752455}, {'type': 'v1', 'addr': '172.21.15.28:6815', 'nonce': 183752455}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6812', 'nonce': 183752455}, {'type': 'v1', 'addr': '172.21.15.28:6813', 'nonce': 183752455}]}, 'public_addr': '172.21.15.28:6809/183752455', 'cluster_addr': '172.21.15.28:6811/183752455', 'heartbeat_back_addr': '172.21.15.28:6815/183752455', 'heartbeat_front_addr': '172.21.15.28:6813/183752455', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '361aa40d-e53b-42af-afd4-81fbcac28725', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 24, 'up_thru': 51, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6808', 'nonce': 2012261939}, {'type': 'v1', 'addr': '172.21.15.42:6809', 'nonce': 2012261939}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6810', 'nonce': 2012261939}, {'type': 'v1', 'addr': '172.21.15.42:6811', 'nonce': 2012261939}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6814', 'nonce': 2012261939}, {'type': 'v1', 'addr': '172.21.15.42:6815', 'nonce': 2012261939}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6812', 'nonce': 2012261939}, {'type': 'v1', 'addr': '172.21.15.42:6813', 'nonce': 2012261939}]}, 'public_addr': '172.21.15.42:6809/2012261939', 'cluster_addr': '172.21.15.42:6811/2012261939', 'heartbeat_back_addr': '172.21.15.42:6815/2012261939', 'heartbeat_front_addr': '172.21.15.42:6813/2012261939', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'f610ad3e-1fd8-42bd-af4b-0d79c3bb9cf8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 48, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6808', 'nonce': 3533460882}, {'type': 'v1', 'addr': '172.21.15.66:6809', 'nonce': 3533460882}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6810', 'nonce': 3533460882}, {'type': 'v1', 'addr': '172.21.15.66:6811', 'nonce': 3533460882}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6814', 'nonce': 3533460882}, {'type': 'v1', 'addr': '172.21.15.66:6815', 'nonce': 3533460882}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6812', 'nonce': 3533460882}, {'type': 'v1', 'addr': '172.21.15.66:6813', 'nonce': 3533460882}]}, 'public_addr': '172.21.15.66:6809/3533460882', 'cluster_addr': '172.21.15.66:6811/3533460882', 'heartbeat_back_addr': '172.21.15.66:6815/3533460882', 'heartbeat_front_addr': '172.21.15.66:6813/3533460882', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'a4bae6fb-e087-4563-a129-6938e18eed8c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 32, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6802', 'nonce': 1760163933}, {'type': 'v1', 'addr': '172.21.15.92:6803', 'nonce': 1760163933}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6804', 'nonce': 1760163933}, {'type': 'v1', 'addr': '172.21.15.92:6805', 'nonce': 1760163933}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6808', 'nonce': 1760163933}, {'type': 'v1', 'addr': '172.21.15.92:6809', 'nonce': 1760163933}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6806', 'nonce': 1760163933}, {'type': 'v1', 'addr': '172.21.15.92:6807', 'nonce': 1760163933}]}, 'public_addr': '172.21.15.92:6803/1760163933', 'cluster_addr': '172.21.15.92:6805/1760163933', 'heartbeat_back_addr': '172.21.15.92:6809/1760163933', 'heartbeat_front_addr': '172.21.15.92:6807/1760163933', 'state': ['exists', 'up']}, {'osd': 8, 'uuid': '479f3885-452e-4231-bdc2-b79668384ba6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6816', 'nonce': 4127602407}, {'type': 'v1', 'addr': '172.21.15.28:6817', 'nonce': 4127602407}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6818', 'nonce': 4127602407}, {'type': 'v1', 'addr': '172.21.15.28:6819', 'nonce': 4127602407}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6822', 'nonce': 4127602407}, {'type': 'v1', 'addr': '172.21.15.28:6823', 'nonce': 4127602407}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6820', 'nonce': 4127602407}, {'type': 'v1', 'addr': '172.21.15.28:6821', 'nonce': 4127602407}]}, 'public_addr': '172.21.15.28:6817/4127602407', 'cluster_addr': '172.21.15.28:6819/4127602407', 'heartbeat_back_addr': '172.21.15.28:6823/4127602407', 'heartbeat_front_addr': '172.21.15.28:6821/4127602407', 'state': ['exists', 'up']}, {'osd': 9, 'uuid': '9a603a47-7c5a-47d4-a298-56b3ac991892', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 26, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6816', 'nonce': 4168729845}, {'type': 'v1', 'addr': '172.21.15.42:6818', 'nonce': 4168729845}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6820', 'nonce': 4168729845}, {'type': 'v1', 'addr': '172.21.15.42:6822', 'nonce': 4168729845}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6828', 'nonce': 4168729845}, {'type': 'v1', 'addr': '172.21.15.42:6830', 'nonce': 4168729845}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6824', 'nonce': 4168729845}, {'type': 'v1', 'addr': '172.21.15.42:6826', 'nonce': 4168729845}]}, 'public_addr': '172.21.15.42:6818/4168729845', 'cluster_addr': '172.21.15.42:6822/4168729845', 'heartbeat_back_addr': '172.21.15.42:6830/4168729845', 'heartbeat_front_addr': '172.21.15.42:6826/4168729845', 'state': ['exists', 'up']}, {'osd': 10, 'uuid': '1cd42945-6ee1-49f0-b17b-4093dd36b4ec', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6816', 'nonce': 4180572718}, {'type': 'v1', 'addr': '172.21.15.66:6817', 'nonce': 4180572718}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6818', 'nonce': 4180572718}, {'type': 'v1', 'addr': '172.21.15.66:6819', 'nonce': 4180572718}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6822', 'nonce': 4180572718}, {'type': 'v1', 'addr': '172.21.15.66:6823', 'nonce': 4180572718}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6820', 'nonce': 4180572718}, {'type': 'v1', 'addr': '172.21.15.66:6821', 'nonce': 4180572718}]}, 'public_addr': '172.21.15.66:6817/4180572718', 'cluster_addr': '172.21.15.66:6819/4180572718', 'heartbeat_back_addr': '172.21.15.66:6823/4180572718', 'heartbeat_front_addr': '172.21.15.66:6821/4180572718', 'state': ['exists', 'up']}, {'osd': 11, 'uuid': 'b41e8100-1d4f-4b7a-991a-7a27bc29feec', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 55, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6818', 'nonce': 295713815}, {'type': 'v1', 'addr': '172.21.15.92:6819', 'nonce': 295713815}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6820', 'nonce': 295713815}, {'type': 'v1', 'addr': '172.21.15.92:6821', 'nonce': 295713815}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6824', 'nonce': 295713815}, {'type': 'v1', 'addr': '172.21.15.92:6825', 'nonce': 295713815}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6822', 'nonce': 295713815}, {'type': 'v1', 'addr': '172.21.15.92:6823', 'nonce': 295713815}]}, 'public_addr': '172.21.15.92:6819/295713815', 'cluster_addr': '172.21.15.92:6821/295713815', 'heartbeat_back_addr': '172.21.15.92:6825/295713815', 'heartbeat_front_addr': '172.21.15.92:6823/295713815', 'state': ['exists', 'up']}, {'osd': 12, 'uuid': '458cf150-32e1-449e-b533-ac6ab5226f85', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6824', 'nonce': 2081141953}, {'type': 'v1', 'addr': '172.21.15.28:6825', 'nonce': 2081141953}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6826', 'nonce': 2081141953}, {'type': 'v1', 'addr': '172.21.15.28:6827', 'nonce': 2081141953}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6830', 'nonce': 2081141953}, {'type': 'v1', 'addr': '172.21.15.28:6831', 'nonce': 2081141953}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6828', 'nonce': 2081141953}, {'type': 'v1', 'addr': '172.21.15.28:6829', 'nonce': 2081141953}]}, 'public_addr': '172.21.15.28:6825/2081141953', 'cluster_addr': '172.21.15.28:6827/2081141953', 'heartbeat_back_addr': '172.21.15.28:6831/2081141953', 'heartbeat_front_addr': '172.21.15.28:6829/2081141953', 'state': ['exists', 'up']}, {'osd': 13, 'uuid': 'f008462e-a735-4845-9ecf-dd7d4a6941af', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 59, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6817', 'nonce': 301360893}, {'type': 'v1', 'addr': '172.21.15.42:6819', 'nonce': 301360893}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6821', 'nonce': 301360893}, {'type': 'v1', 'addr': '172.21.15.42:6823', 'nonce': 301360893}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6829', 'nonce': 301360893}, {'type': 'v1', 'addr': '172.21.15.42:6831', 'nonce': 301360893}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6825', 'nonce': 301360893}, {'type': 'v1', 'addr': '172.21.15.42:6827', 'nonce': 301360893}]}, 'public_addr': '172.21.15.42:6819/301360893', 'cluster_addr': '172.21.15.42:6823/301360893', 'heartbeat_back_addr': '172.21.15.42:6831/301360893', 'heartbeat_front_addr': '172.21.15.42:6827/301360893', 'state': ['exists', 'up']}, {'osd': 14, 'uuid': '6f66ae97-eb8a-48eb-9bcc-20f65fbf690b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 53, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6824', 'nonce': 1534905602}, {'type': 'v1', 'addr': '172.21.15.66:6825', 'nonce': 1534905602}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6826', 'nonce': 1534905602}, {'type': 'v1', 'addr': '172.21.15.66:6827', 'nonce': 1534905602}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6830', 'nonce': 1534905602}, {'type': 'v1', 'addr': '172.21.15.66:6831', 'nonce': 1534905602}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6828', 'nonce': 1534905602}, {'type': 'v1', 'addr': '172.21.15.66:6829', 'nonce': 1534905602}]}, 'public_addr': '172.21.15.66:6825/1534905602', 'cluster_addr': '172.21.15.66:6827/1534905602', 'heartbeat_back_addr': '172.21.15.66:6831/1534905602', 'heartbeat_front_addr': '172.21.15.66:6829/1534905602', 'state': ['exists', 'up']}, {'osd': 15, 'uuid': 'b856bfc8-253b-4112-be5c-f67d61d01768', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6826', 'nonce': 3982634323}, {'type': 'v1', 'addr': '172.21.15.92:6827', 'nonce': 3982634323}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6828', 'nonce': 3982634323}, {'type': 'v1', 'addr': '172.21.15.92:6829', 'nonce': 3982634323}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6832', 'nonce': 3982634323}, {'type': 'v1', 'addr': '172.21.15.92:6833', 'nonce': 3982634323}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6830', 'nonce': 3982634323}, {'type': 'v1', 'addr': '172.21.15.92:6831', 'nonce': 3982634323}]}, 'public_addr': '172.21.15.92:6827/3982634323', 'cluster_addr': '172.21.15.92:6829/3982634323', 'heartbeat_back_addr': '172.21.15.92:6833/3982634323', 'heartbeat_front_addr': '172.21.15.92:6831/3982634323', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.265053+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.243502+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.252440+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.295331+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.213076+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.281193+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.315801+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.329538+0000', 'dead_epoch': 0}, {'osd': 8, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.293303+0000', 'dead_epoch': 0}, {'osd': 9, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.334962+0000', 'dead_epoch': 0}, {'osd': 10, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.233032+0000', 'dead_epoch': 0}, {'osd': 11, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.305138+0000', 'dead_epoch': 0}, {'osd': 12, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.351964+0000', 'dead_epoch': 0}, {'osd': 13, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.241086+0000', 'dead_epoch': 0}, {'osd': 14, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.352515+0000', 'dead_epoch': 0}, {'osd': 15, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.357462+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '2.2', 'mappings': [{'from': 3, 'to': 11}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}, 'shecprofile': {'c': '2', 'crush-device-class': '', 'crush-failure-domain': 'osd', 'crush-num-failure-domains': '0', 'crush-osds-per-failure-domain': '0', 'crush-root': 'default', 'k': '4', 'm': '3', 'name': 'shecprofile', 'plugin': 'shec', 'technique': 'multiple'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 9, 'length': 3}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 11, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-04-26T03:39:16.205 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T03:39:16.205 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-26T03:39:16.205 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-04-26T03:39:16.206 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-04-26T03:39:16.206 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-26T03:39:16.206 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-04-26T03:39:16.206 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:39:16.197+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:39:16.299 INFO:tasks.ceph.osd.9.smithi042.stderr:2024-04-26T03:39:16.298+0000 7fdac63b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 31960) UID: 0 2024-04-26T03:39:16.353 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-26T03:39:16.353 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-04-26T03:39:16.353 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-04-26T03:39:16.353 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-04-26T03:39:16.353 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-26T03:39:16.361 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-04-26T03:39:16.401 INFO:tasks.ceph.osd.14.smithi066.stderr:2024-04-26T03:39:16.400+0000 7f5395f08640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 31765) UID: 0 2024-04-26T03:39:16.502 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:39:16.501+0000 7f60d3c4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31757) UID: 0 2024-04-26T03:39:16.510 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-26T03:39:16.510 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-04-26T03:39:16.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-04-26T03:39:16.511 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-04-26T03:39:16.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.511 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds2 35/32 ECSubWrite(tid=0, reqid=client.4556.0:43, at_version=35'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:39:16.511 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.501816+0000", 2024-04-26T03:39:16.511 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.006191737999998, 2024-04-26T03:39:16.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0091953699999999996, 2024-04-26T03:39:16.511 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:16.511 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:16.511 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:16.511 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:16.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:16.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.501816+0000", 2024-04-26T03:39:16.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.512 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:16.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.501816+0000", 2024-04-26T03:39:16.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.512 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:16.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.501816+0000", 2024-04-26T03:39:16.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8099999999999998e-07 2024-04-26T03:39:16.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.513 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:16.513 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.502179+0000", 2024-04-26T03:39:16.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036287299999999999 2024-04-26T03:39:16.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.513 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:16.513 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.502197+0000", 2024-04-26T03:39:16.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7765e-05 2024-04-26T03:39:16.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.513 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:16.513 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.502207+0000", 2024-04-26T03:39:16.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6190000000000006e-06 2024-04-26T03:39:16.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:16.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.502306+0000", 2024-04-26T03:39:16.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9655999999999996e-05 2024-04-26T03:39:16.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:16.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.502315+0000", 2024-04-26T03:39:16.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5629999999999992e-06 2024-04-26T03:39:16.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:16.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.510976+0000", 2024-04-26T03:39:16.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0086612379999999999 2024-04-26T03:39:16.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:16.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.511011+0000", 2024-04-26T03:39:16.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5274999999999997e-05 2024-04-26T03:39:16.515 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.515 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:16.515 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.515 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds2 35/32 ECSubWrite(tid=1, reqid=client.4556.0:44, at_version=35'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:39:16.515 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.504658+0000", 2024-04-26T03:39:16.515 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.003349421999999, 2024-04-26T03:39:16.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0063889319999999999, 2024-04-26T03:39:16.516 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:16.516 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:16.516 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:16.516 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:16.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:16.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504658+0000", 2024-04-26T03:39:16.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:16.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504658+0000", 2024-04-26T03:39:16.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:16.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504659+0000", 2024-04-26T03:39:16.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3399999999999997e-07 2024-04-26T03:39:16.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:16.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504681+0000", 2024-04-26T03:39:16.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1452999999999999e-05 2024-04-26T03:39:16.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:16.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504700+0000", 2024-04-26T03:39:16.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9772999999999999e-05 2024-04-26T03:39:16.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:16.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504710+0000", 2024-04-26T03:39:16.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2660000000000007e-06 2024-04-26T03:39:16.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:16.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.505327+0000", 2024-04-26T03:39:16.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00061779000000000001 2024-04-26T03:39:16.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:16.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.505339+0000", 2024-04-26T03:39:16.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.147e-05 2024-04-26T03:39:16.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:16.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.511016+0000", 2024-04-26T03:39:16.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0056775439999999996 2024-04-26T03:39:16.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:16.520 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.511047+0000", 2024-04-26T03:39:16.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0701999999999998e-05 2024-04-26T03:39:16.520 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.520 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:16.520 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.520 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.es5 35/32 ECSubWrite(tid=2, reqid=client.4556.0:58, at_version=35'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:39:16.520 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.513076+0000", 2024-04-26T03:39:16.520 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 24.994931924999999, 2024-04-26T03:39:16.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0049094890000000004, 2024-04-26T03:39:16.520 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:16.520 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:16.520 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:16.521 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:16.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:16.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.513076+0000", 2024-04-26T03:39:16.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:16.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.513076+0000", 2024-04-26T03:39:16.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:16.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.513077+0000", 2024-04-26T03:39:16.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5000000000000001e-07 2024-04-26T03:39:16.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:16.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.513176+0000", 2024-04-26T03:39:16.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9588999999999998e-05 2024-04-26T03:39:16.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:16.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.513210+0000", 2024-04-26T03:39:16.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3800000000000002e-05 2024-04-26T03:39:16.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:16.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.513219+0000", 2024-04-26T03:39:16.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4050000000000006e-06 2024-04-26T03:39:16.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:16.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.513378+0000", 2024-04-26T03:39:16.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015877600000000001 2024-04-26T03:39:16.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:16.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.513388+0000", 2024-04-26T03:39:16.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4919999999999999e-06 2024-04-26T03:39:16.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:16.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.517940+0000", 2024-04-26T03:39:16.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0045520250000000003 2024-04-26T03:39:16.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:16.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.517985+0000", 2024-04-26T03:39:16.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5552e-05 2024-04-26T03:39:16.524 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.524 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:16.524 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.525 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.9s3 35/32 ECSubWrite(tid=10, reqid=client.4556.0:76, at_version=35'3, trim_to=0'0, roll_forward_to=35'2))", 2024-04-26T03:39:16.525 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.530631+0000", 2024-04-26T03:39:16.525 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 24.977376599999999, 2024-04-26T03:39:16.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0058993190000000001, 2024-04-26T03:39:16.525 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:16.525 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:16.525 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:16.525 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:16.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:16.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.530631+0000", 2024-04-26T03:39:16.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:16.526 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.530631+0000", 2024-04-26T03:39:16.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:16.526 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.530632+0000", 2024-04-26T03:39:16.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3900000000000004e-07 2024-04-26T03:39:16.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:16.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.531212+0000", 2024-04-26T03:39:16.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000579979 2024-04-26T03:39:16.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:16.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.531238+0000", 2024-04-26T03:39:16.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5927000000000001e-05 2024-04-26T03:39:16.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:16.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.531247+0000", 2024-04-26T03:39:16.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8519999999999993e-06 2024-04-26T03:39:16.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:16.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.531314+0000", 2024-04-26T03:39:16.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7070000000000007e-05 2024-04-26T03:39:16.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:16.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.531322+0000", 2024-04-26T03:39:16.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6859999999999995e-06 2024-04-26T03:39:16.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:16.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.536494+0000", 2024-04-26T03:39:16.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0051716319999999998 2024-04-26T03:39:16.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:16.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.536530+0000", 2024-04-26T03:39:16.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6334000000000001e-05 2024-04-26T03:39:16.529 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.529 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:16.529 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.529 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.2s3 35/32 ECSubWrite(tid=4, reqid=client.4556.0:79, at_version=35'3, trim_to=0'0, roll_forward_to=34'2))", 2024-04-26T03:39:16.529 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.533108+0000", 2024-04-26T03:39:16.529 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 24.974899488999998, 2024-04-26T03:39:16.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0051759689999999999, 2024-04-26T03:39:16.530 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:16.530 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:16.530 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:16.530 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:16.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:16.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.533108+0000", 2024-04-26T03:39:16.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:16.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.533108+0000", 2024-04-26T03:39:16.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:16.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.533109+0000", 2024-04-26T03:39:16.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6200000000000006e-07 2024-04-26T03:39:16.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:16.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.533421+0000", 2024-04-26T03:39:16.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00031138699999999999 2024-04-26T03:39:16.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:16.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.533433+0000", 2024-04-26T03:39:16.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2619000000000001e-05 2024-04-26T03:39:16.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:16.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.533441+0000", 2024-04-26T03:39:16.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2349999999999999e-06 2024-04-26T03:39:16.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:16.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.533511+0000", 2024-04-26T03:39:16.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9374000000000004e-05 2024-04-26T03:39:16.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:16.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.533520+0000", 2024-04-26T03:39:16.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3249999999999997e-06 2024-04-26T03:39:16.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:16.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.538235+0000", 2024-04-26T03:39:16.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0047147480000000004 2024-04-26T03:39:16.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:16.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.538284+0000", 2024-04-26T03:39:16.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9319e-05 2024-04-26T03:39:16.533 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.533 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:16.533 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.534 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.1s4 35/32 ECSubWrite(tid=16, reqid=client.4556.0:118, at_version=35'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:39:16.534 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.585779+0000", 2024-04-26T03:39:16.534 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 24.922229123000001, 2024-04-26T03:39:16.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0052872639999999999, 2024-04-26T03:39:16.534 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:16.534 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:16.534 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:16.534 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:16.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:16.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.585779+0000", 2024-04-26T03:39:16.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:16.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.585779+0000", 2024-04-26T03:39:16.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:16.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.585779+0000", 2024-04-26T03:39:16.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2100000000000001e-07 2024-04-26T03:39:16.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:16.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.586398+0000", 2024-04-26T03:39:16.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00061936399999999996 2024-04-26T03:39:16.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:16.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.586412+0000", 2024-04-26T03:39:16.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4246e-05 2024-04-26T03:39:16.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:16.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.586422+0000", 2024-04-26T03:39:16.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3919999999999992e-06 2024-04-26T03:39:16.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:16.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.586496+0000", 2024-04-26T03:39:16.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4559000000000002e-05 2024-04-26T03:39:16.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:16.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.586506+0000", 2024-04-26T03:39:16.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0122999999999999e-05 2024-04-26T03:39:16.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:16.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.591034+0000", 2024-04-26T03:39:16.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0045271699999999996 2024-04-26T03:39:16.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:16.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.591066+0000", 2024-04-26T03:39:16.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2289000000000002e-05 2024-04-26T03:39:16.538 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.538 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:16.538 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.538 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.es5 35/32 ECSubWrite(tid=24, reqid=client.4556.0:124, at_version=35'7, trim_to=0'0, roll_forward_to=35'6))", 2024-04-26T03:39:16.538 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.596016+0000", 2024-04-26T03:39:16.538 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 24.911991241999999, 2024-04-26T03:39:16.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0052964839999999997, 2024-04-26T03:39:16.539 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:16.539 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:16.539 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:16.539 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:16.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:16.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.596016+0000", 2024-04-26T03:39:16.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:16.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.596016+0000", 2024-04-26T03:39:16.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:16.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.596022+0000", 2024-04-26T03:39:16.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3530000000000002e-06 2024-04-26T03:39:16.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:16.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.596141+0000", 2024-04-26T03:39:16.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011927 2024-04-26T03:39:16.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:16.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.596154+0000", 2024-04-26T03:39:16.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2903e-05 2024-04-26T03:39:16.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:16.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.596161+0000", 2024-04-26T03:39:16.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1119999999999998e-06 2024-04-26T03:39:16.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:16.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.596246+0000", 2024-04-26T03:39:16.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5056999999999996e-05 2024-04-26T03:39:16.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:16.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.596264+0000", 2024-04-26T03:39:16.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7844000000000001e-05 2024-04-26T03:39:16.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:16.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.601278+0000", 2024-04-26T03:39:16.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0050136 2024-04-26T03:39:16.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:16.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.601313+0000", 2024-04-26T03:39:16.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5345000000000001e-05 2024-04-26T03:39:16.543 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.543 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:16.543 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.543 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.2s3 35/32 ECSubWrite(tid=8, reqid=client.4556.0:139, at_version=35'5, trim_to=0'0, roll_forward_to=35'4))", 2024-04-26T03:39:16.543 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.621036+0000", 2024-04-26T03:39:16.544 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 24.886971886000001, 2024-04-26T03:39:16.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0056652040000000001, 2024-04-26T03:39:16.544 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:16.544 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:16.544 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:16.544 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:16.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:16.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.621036+0000", 2024-04-26T03:39:16.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:16.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.621036+0000", 2024-04-26T03:39:16.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:16.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.621037+0000", 2024-04-26T03:39:16.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.78e-07 2024-04-26T03:39:16.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:16.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.621467+0000", 2024-04-26T03:39:16.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00043015799999999998 2024-04-26T03:39:16.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:16.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.621481+0000", 2024-04-26T03:39:16.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3978999999999999e-05 2024-04-26T03:39:16.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:16.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.621491+0000", 2024-04-26T03:39:16.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7010000000000008e-06 2024-04-26T03:39:16.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:16.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.621571+0000", 2024-04-26T03:39:16.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0408000000000007e-05 2024-04-26T03:39:16.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:16.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.621581+0000", 2024-04-26T03:39:16.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0079e-05 2024-04-26T03:39:16.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:16.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.626670+0000", 2024-04-26T03:39:16.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0050891920000000002 2024-04-26T03:39:16.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:16.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.626701+0000", 2024-04-26T03:39:16.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0809000000000003e-05 2024-04-26T03:39:16.548 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.548 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:16.548 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.548 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.es5 43/32 ECSubWrite(tid=48, reqid=client.4556.0:301, at_version=43'16, trim_to=0'0, roll_forward_to=35'14))", 2024-04-26T03:39:16.548 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:59.531227+0000", 2024-04-26T03:39:16.548 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 16.976780381000001, 2024-04-26T03:39:16.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0047220659999999996, 2024-04-26T03:39:16.548 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:16.548 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:16.548 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:16.548 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:16.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:16.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.531227+0000", 2024-04-26T03:39:16.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:16.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.531227+0000", 2024-04-26T03:39:16.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:16.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.531228+0000", 2024-04-26T03:39:16.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2900000000000004e-07 2024-04-26T03:39:16.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:16.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.531396+0000", 2024-04-26T03:39:16.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000168345 2024-04-26T03:39:16.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:16.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.531421+0000", 2024-04-26T03:39:16.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4882e-05 2024-04-26T03:39:16.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:16.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.531434+0000", 2024-04-26T03:39:16.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2582000000000001e-05 2024-04-26T03:39:16.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:16.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.531518+0000", 2024-04-26T03:39:16.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4658000000000002e-05 2024-04-26T03:39:16.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:16.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.531527+0000", 2024-04-26T03:39:16.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.081e-06 2024-04-26T03:39:16.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:16.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.535915+0000", 2024-04-26T03:39:16.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0043878600000000004 2024-04-26T03:39:16.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:16.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.535949+0000", 2024-04-26T03:39:16.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4128999999999998e-05 2024-04-26T03:39:16.552 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.552 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:16.552 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.552 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.es5 43/32 ECSubWrite(tid=49, reqid=client.4556.0:302, at_version=43'17, trim_to=0'0, roll_forward_to=35'14))", 2024-04-26T03:39:16.552 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:59.537163+0000", 2024-04-26T03:39:16.552 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 16.970844313000001, 2024-04-26T03:39:16.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0055244370000000001, 2024-04-26T03:39:16.553 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:16.553 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:16.553 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:16.553 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:16.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:16.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.537163+0000", 2024-04-26T03:39:16.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:16.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.537163+0000", 2024-04-26T03:39:16.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:16.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.537165+0000", 2024-04-26T03:39:16.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7039999999999999e-06 2024-04-26T03:39:16.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:16.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.537336+0000", 2024-04-26T03:39:16.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000170619 2024-04-26T03:39:16.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:16.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.537352+0000", 2024-04-26T03:39:16.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5974999999999999e-05 2024-04-26T03:39:16.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:16.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.537361+0000", 2024-04-26T03:39:16.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2490000000000007e-06 2024-04-26T03:39:16.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:16.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.537448+0000", 2024-04-26T03:39:16.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7148e-05 2024-04-26T03:39:16.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:16.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.537457+0000", 2024-04-26T03:39:16.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6500000000000002e-06 2024-04-26T03:39:16.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:16.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.542650+0000", 2024-04-26T03:39:16.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0051930580000000004 2024-04-26T03:39:16.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:16.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.542688+0000", 2024-04-26T03:39:16.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8034000000000002e-05 2024-04-26T03:39:16.557 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.557 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:16.557 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.557 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.2s3 45/44 ECSubWrite(tid=22, reqid=client.4556.0:333, at_version=45'12, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:39:16.557 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:01.566819+0000", 2024-04-26T03:39:16.557 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.941188714999999, 2024-04-26T03:39:16.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0049229859999999999, 2024-04-26T03:39:16.558 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:16.558 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:16.558 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:16.558 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:16.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:16.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.566819+0000", 2024-04-26T03:39:16.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:16.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.566819+0000", 2024-04-26T03:39:16.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:16.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.566820+0000", 2024-04-26T03:39:16.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.082e-06 2024-04-26T03:39:16.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:16.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.567205+0000", 2024-04-26T03:39:16.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000384526 2024-04-26T03:39:16.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:16.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.567217+0000", 2024-04-26T03:39:16.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2310000000000001e-05 2024-04-26T03:39:16.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:16.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.567223+0000", 2024-04-26T03:39:16.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5309999999999998e-06 2024-04-26T03:39:16.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:16.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.567284+0000", 2024-04-26T03:39:16.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0446999999999999e-05 2024-04-26T03:39:16.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:16.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.567293+0000", 2024-04-26T03:39:16.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4739999999999994e-06 2024-04-26T03:39:16.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:16.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.571717+0000", 2024-04-26T03:39:16.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0044237579999999999 2024-04-26T03:39:16.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:16.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.571742+0000", 2024-04-26T03:39:16.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4858000000000002e-05 2024-04-26T03:39:16.562 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.562 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:16.562 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.562 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.cs4 45/32 ECSubWrite(tid=53, reqid=client.4556.0:338, at_version=45'21, trim_to=0'0, roll_forward_to=42'19))", 2024-04-26T03:39:16.562 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:01.571984+0000", 2024-04-26T03:39:16.562 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.93602332, 2024-04-26T03:39:16.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0049388119999999999, 2024-04-26T03:39:16.563 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:16.563 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:16.563 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:16.563 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:16.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:16.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.571984+0000", 2024-04-26T03:39:16.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:16.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.571984+0000", 2024-04-26T03:39:16.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:16.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.571984+0000", 2024-04-26T03:39:16.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.37e-07 2024-04-26T03:39:16.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:16.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.572399+0000", 2024-04-26T03:39:16.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00041413600000000001 2024-04-26T03:39:16.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:16.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.572409+0000", 2024-04-26T03:39:16.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0618000000000001e-05 2024-04-26T03:39:16.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:16.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.572417+0000", 2024-04-26T03:39:16.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8790000000000003e-06 2024-04-26T03:39:16.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:16.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.572463+0000", 2024-04-26T03:39:16.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6013000000000001e-05 2024-04-26T03:39:16.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:16.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.572470+0000", 2024-04-26T03:39:16.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.815e-06 2024-04-26T03:39:16.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:16.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.576892+0000", 2024-04-26T03:39:16.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0044224759999999998 2024-04-26T03:39:16.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:16.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.576923+0000", 2024-04-26T03:39:16.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0738000000000003e-05 2024-04-26T03:39:16.566 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.566 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:16.567 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.567 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.2s3 49/44 ECSubWrite(tid=32, reqid=client.4556.0:433, at_version=49'20, trim_to=0'0, roll_forward_to=48'19))", 2024-04-26T03:39:16.567 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:04.700091+0000", 2024-04-26T03:39:16.567 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 11.807916831, 2024-04-26T03:39:16.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0091986069999999993, 2024-04-26T03:39:16.567 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:16.567 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:16.567 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:16.567 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:16.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:16.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.700091+0000", 2024-04-26T03:39:16.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:16.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.700091+0000", 2024-04-26T03:39:16.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:16.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.700091+0000", 2024-04-26T03:39:16.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1600000000000001e-07 2024-04-26T03:39:16.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:16.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.700840+0000", 2024-04-26T03:39:16.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00074861199999999995 2024-04-26T03:39:16.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:16.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.700854+0000", 2024-04-26T03:39:16.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3696999999999999e-05 2024-04-26T03:39:16.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:16.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.700863+0000", 2024-04-26T03:39:16.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2969999999999992e-06 2024-04-26T03:39:16.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:16.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.700940+0000", 2024-04-26T03:39:16.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7356999999999998e-05 2024-04-26T03:39:16.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:16.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.700950+0000", 2024-04-26T03:39:16.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7340000000000003e-06 2024-04-26T03:39:16.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:16.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.709245+0000", 2024-04-26T03:39:16.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.008294582 2024-04-26T03:39:16.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:16.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.709289+0000", 2024-04-26T03:39:16.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4811999999999997e-05 2024-04-26T03:39:16.571 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.571 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:16.571 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.571 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.2s3 49/44 ECSubWrite(tid=34, reqid=client.4556.0:434, at_version=49'23, trim_to=0'0, roll_forward_to=48'19))", 2024-04-26T03:39:16.571 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:04.712922+0000", 2024-04-26T03:39:16.571 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 11.795085887999999, 2024-04-26T03:39:16.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0059780120000000004, 2024-04-26T03:39:16.572 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:16.572 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:16.572 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:16.572 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:16.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:16.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.712922+0000", 2024-04-26T03:39:16.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:16.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.712922+0000", 2024-04-26T03:39:16.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:16.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.712923+0000", 2024-04-26T03:39:16.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.237e-06 2024-04-26T03:39:16.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:16.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.713234+0000", 2024-04-26T03:39:16.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00031075200000000002 2024-04-26T03:39:16.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:16.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.713251+0000", 2024-04-26T03:39:16.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6976000000000001e-05 2024-04-26T03:39:16.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:16.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.713260+0000", 2024-04-26T03:39:16.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0119999999999994e-06 2024-04-26T03:39:16.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:16.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.713314+0000", 2024-04-26T03:39:16.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3947999999999999e-05 2024-04-26T03:39:16.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:16.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.713320+0000", 2024-04-26T03:39:16.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9989999999999997e-06 2024-04-26T03:39:16.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:16.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.718862+0000", 2024-04-26T03:39:16.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0055423080000000001 2024-04-26T03:39:16.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:16.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.718900+0000", 2024-04-26T03:39:16.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7780000000000001e-05 2024-04-26T03:39:16.576 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.576 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:16.576 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.577 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.2s3 50/44 ECSubWrite(tid=42, reqid=client.4556.0:478, at_version=50'28, trim_to=0'0, roll_forward_to=50'27))", 2024-04-26T03:39:16.577 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:05.912137+0000", 2024-04-26T03:39:16.577 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.595870700999999, 2024-04-26T03:39:16.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0056262439999999999, 2024-04-26T03:39:16.577 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:16.577 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:16.577 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:16.577 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:16.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:16.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.912137+0000", 2024-04-26T03:39:16.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:16.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.912137+0000", 2024-04-26T03:39:16.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:16.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.912138+0000", 2024-04-26T03:39:16.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0550000000000001e-06 2024-04-26T03:39:16.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:16.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.912745+0000", 2024-04-26T03:39:16.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000606552 2024-04-26T03:39:16.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:16.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.912756+0000", 2024-04-26T03:39:16.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1019000000000001e-05 2024-04-26T03:39:16.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:16.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.912763+0000", 2024-04-26T03:39:16.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3950000000000003e-06 2024-04-26T03:39:16.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:16.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.912834+0000", 2024-04-26T03:39:16.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1049000000000004e-05 2024-04-26T03:39:16.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:16.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.912846+0000", 2024-04-26T03:39:16.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2340999999999999e-05 2024-04-26T03:39:16.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:16.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.917730+0000", 2024-04-26T03:39:16.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0048834259999999997 2024-04-26T03:39:16.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:16.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.917763+0000", 2024-04-26T03:39:16.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3407e-05 2024-04-26T03:39:16.582 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.582 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:16.582 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.582 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds2 56/32 ECSubWrite(tid=28, reqid=client.4556.0:607, at_version=56'25, trim_to=0'0, roll_forward_to=50'23))", 2024-04-26T03:39:16.582 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:11.724660+0000", 2024-04-26T03:39:16.582 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.7833472400000003, 2024-04-26T03:39:16.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.01307469, 2024-04-26T03:39:16.582 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:16.582 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:16.582 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:16.582 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:16.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:16.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.724660+0000", 2024-04-26T03:39:16.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:16.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.724660+0000", 2024-04-26T03:39:16.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:16.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.724661+0000", 2024-04-26T03:39:16.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9999999999999997e-07 2024-04-26T03:39:16.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:16.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.725284+0000", 2024-04-26T03:39:16.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00062266999999999997 2024-04-26T03:39:16.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:16.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.725299+0000", 2024-04-26T03:39:16.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4849e-05 2024-04-26T03:39:16.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:16.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.725309+0000", 2024-04-26T03:39:16.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9809999999999998e-06 2024-04-26T03:39:16.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:16.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.725390+0000", 2024-04-26T03:39:16.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1000000000000004e-05 2024-04-26T03:39:16.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:16.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.725402+0000", 2024-04-26T03:39:16.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2622e-05 2024-04-26T03:39:16.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:16.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.737637+0000", 2024-04-26T03:39:16.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012234681000000001 2024-04-26T03:39:16.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:16.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.737735+0000", 2024-04-26T03:39:16.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8287000000000003e-05 2024-04-26T03:39:16.586 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.586 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:16.586 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.587 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds2 56/32 ECSubWrite(tid=29, reqid=client.4556.0:608, at_version=56'26, trim_to=0'0, roll_forward_to=50'23))", 2024-04-26T03:39:16.587 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:11.732163+0000", 2024-04-26T03:39:16.587 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.7758446939999999, 2024-04-26T03:39:16.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010518913, 2024-04-26T03:39:16.587 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:16.587 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:16.587 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:16.587 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:16.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:16.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.732163+0000", 2024-04-26T03:39:16.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:16.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.732163+0000", 2024-04-26T03:39:16.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:16.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.732164+0000", 2024-04-26T03:39:16.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7300000000000004e-07 2024-04-26T03:39:16.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:16.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.732418+0000", 2024-04-26T03:39:16.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025363600000000001 2024-04-26T03:39:16.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:16.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.732432+0000", 2024-04-26T03:39:16.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.49e-05 2024-04-26T03:39:16.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:16.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.732443+0000", 2024-04-26T03:39:16.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0351e-05 2024-04-26T03:39:16.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:16.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.733300+0000", 2024-04-26T03:39:16.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00085717599999999997 2024-04-26T03:39:16.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:16.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.733310+0000", 2024-04-26T03:39:16.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0210999999999999e-05 2024-04-26T03:39:16.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:16.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.742622+0000", 2024-04-26T03:39:16.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0093113480000000005 2024-04-26T03:39:16.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:16.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.742682+0000", 2024-04-26T03:39:16.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0318000000000001e-05 2024-04-26T03:39:16.591 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.591 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:16.592 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.592 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds2 56/32 ECSubWrite(tid=30, reqid=client.4556.0:609, at_version=56'27, trim_to=0'0, roll_forward_to=50'23))", 2024-04-26T03:39:16.592 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:11.737685+0000", 2024-04-26T03:39:16.592 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.7703222030000001, 2024-04-26T03:39:16.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0057102849999999998, 2024-04-26T03:39:16.592 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:16.592 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:16.592 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:16.592 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:16.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:16.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.737685+0000", 2024-04-26T03:39:16.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:16.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.737685+0000", 2024-04-26T03:39:16.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:16.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.737686+0000", 2024-04-26T03:39:16.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8400000000000004e-07 2024-04-26T03:39:16.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:16.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.737927+0000", 2024-04-26T03:39:16.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000241215 2024-04-26T03:39:16.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:16.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.737943+0000", 2024-04-26T03:39:16.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5563999999999999e-05 2024-04-26T03:39:16.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:16.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.737952+0000", 2024-04-26T03:39:16.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2909999999999996e-06 2024-04-26T03:39:16.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:16.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.738028+0000", 2024-04-26T03:39:16.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6087000000000005e-05 2024-04-26T03:39:16.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:16.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.738039+0000", 2024-04-26T03:39:16.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0864e-05 2024-04-26T03:39:16.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:16.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.743355+0000", 2024-04-26T03:39:16.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0053161700000000003 2024-04-26T03:39:16.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:16.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.743396+0000", 2024-04-26T03:39:16.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0410000000000001e-05 2024-04-26T03:39:16.596 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.596 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:16.596 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.596 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds2 60/32 ECSubWrite(tid=45, reqid=client.4556.0:691, at_version=60'43, trim_to=0'0, roll_forward_to=60'41))", 2024-04-26T03:39:16.596 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:15.954667+0000", 2024-04-26T03:39:16.596 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.553341164, 2024-04-26T03:39:16.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0056884680000000003, 2024-04-26T03:39:16.597 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:16.597 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:16.597 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:16.597 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:16.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:16.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.954667+0000", 2024-04-26T03:39:16.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:16.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.954667+0000", 2024-04-26T03:39:16.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:16.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.954668+0000", 2024-04-26T03:39:16.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.26e-06 2024-04-26T03:39:16.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:16.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.954882+0000", 2024-04-26T03:39:16.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021437299999999999 2024-04-26T03:39:16.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:16.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.954897+0000", 2024-04-26T03:39:16.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5214999999999999e-05 2024-04-26T03:39:16.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:16.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.954906+0000", 2024-04-26T03:39:16.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7919999999999998e-06 2024-04-26T03:39:16.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:16.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.954968+0000", 2024-04-26T03:39:16.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1661999999999993e-05 2024-04-26T03:39:16.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:16.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.954974+0000", 2024-04-26T03:39:16.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1319999999999999e-06 2024-04-26T03:39:16.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:16.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.960325+0000", 2024-04-26T03:39:16.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0053511990000000001 2024-04-26T03:39:16.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:16.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.960355+0000", 2024-04-26T03:39:16.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9835000000000001e-05 2024-04-26T03:39:16.600 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.600 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:16.601 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.601 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.10s1 60/48 ECSubWrite(tid=272, reqid=client.4556.0:703, at_version=60'20, trim_to=0'0, roll_forward_to=60'18))", 2024-04-26T03:39:16.601 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:15.958198+0000", 2024-04-26T03:39:16.601 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.549809781, 2024-04-26T03:39:16.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0082246890000000003, 2024-04-26T03:39:16.601 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:16.601 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:16.601 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:16.601 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:16.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:16.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.958198+0000", 2024-04-26T03:39:16.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:16.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.958198+0000", 2024-04-26T03:39:16.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:16.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.958199+0000", 2024-04-26T03:39:16.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3e-07 2024-04-26T03:39:16.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:16.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.959083+0000", 2024-04-26T03:39:16.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00088405799999999996 2024-04-26T03:39:16.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:16.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.959097+0000", 2024-04-26T03:39:16.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.456e-05 2024-04-26T03:39:16.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:16.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.959106+0000", 2024-04-26T03:39:16.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3210000000000005e-06 2024-04-26T03:39:16.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:16.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.959164+0000", 2024-04-26T03:39:16.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8700999999999999e-05 2024-04-26T03:39:16.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:16.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.959172+0000", 2024-04-26T03:39:16.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.374e-06 2024-04-26T03:39:16.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:16.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.966393+0000", 2024-04-26T03:39:16.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0072211209999999996 2024-04-26T03:39:16.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:16.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.966423+0000", 2024-04-26T03:39:16.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9824000000000001e-05 2024-04-26T03:39:16.605 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.605 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:16.605 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.605 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.605 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:16.605 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-26T03:39:16.607 INFO:tasks.ceph.osd.10.smithi066.stderr:2024-04-26T03:39:16.606+0000 7f130fcc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 31763) UID: 0 2024-04-26T03:39:16.608 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:39:16.609 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-04-26T03:39:16.609 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-04-26T03:39:16.611 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2024-04-26T03:39:16.056965+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":73171488,"num_objects":50,"num_object_clones":8,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":6,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":1,"num_read":341,"num_read_kb":103619,"num_write":646,"num_write_kb":58820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":827,"ondisk_log_size":827,"up":193,"acting":193,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":203,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":593240,"kb_used_data":140064,"kb_used_omap":24,"kb_used_meta":452903,"kb_avail":1509356200,"statfs":{"total":1546188226560,"available":1545580748800,"internally_reserved":0,"allocated":143425536,"data_stored":150838482,"data_compressed":1060428,"data_compressed_allocated":2224128,"data_compressed_original":5341184,"omap_allocated":25522,"internal_metadata":463772750},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":13131776,"num_objects":4,"num_object_clones":7,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":4,"num_read_kb":50792,"num_write":104,"num_write_kb":5944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000593"},"pg_stats":[{"pgid":"3.18","version":"52'55","reported_seq":202,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T03:39:11.741364+0000","last_change":"2024-04-26T03:39:07.716271+0000","last_active":"2024-04-26T03:39:11.741364+0000","last_peered":"2024-04-26T03:39:11.741364+0000","last_clean":"2024-04-26T03:39:11.741364+0000","last_became_active":"2024-04-26T03:39:01.553702+0000","last_became_peered":"2024-04-26T03:39:01.553702+0000","last_unstale":"2024-04-26T03:39:11.741364+0000","last_undegraded":"2024-04-26T03:39:11.741364+0000","last_fullsized":"2024-04-26T03:39:11.741364+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":5,"last_scrub":"50'43","last_scrub_stamp":"2024-04-26T03:39:06.156322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:06.156322+0000","objects_scrubbed":3,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:25.874703+0000","scrub_duration":3,"objects_trimmed":1,"snaptrim_duration":0.006160736,"stat_sum":{"num_bytes":7798784,"num_objects":3,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":15652,"num_write":9,"num_write_kb":5424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,10,11,15,5],"acting":[6,0,1,10,11,15,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"5","length":"5"}]},{"pgid":"3.17","version":"43'6","reported_seq":141,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-26T03:39:12.723865+0000","last_change":"2024-04-26T03:39:07.700601+0000","last_active":"2024-04-26T03:39:12.723865+0000","last_peered":"2024-04-26T03:39:12.723865+0000","last_clean":"2024-04-26T03:39:12.723865+0000","last_became_active":"2024-04-26T03:39:01.550697+0000","last_became_peered":"2024-04-26T03:39:01.550697+0000","last_unstale":"2024-04-26T03:39:12.723865+0000","last_undegraded":"2024-04-26T03:39:12.723865+0000","last_fullsized":"2024-04-26T03:39:12.723865+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":5,"last_scrub":"43'6","last_scrub_stamp":"2024-04-26T03:39:07.292706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:07.292706+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:27.779428+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00069869099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,13,6,0,14,10,11],"acting":[4,13,6,0,14,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"5","length":"5"}]},{"pgid":"3.16","version":"35'3","reported_seq":139,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-26T03:39:12.727071+0000","last_change":"2024-04-26T03:39:07.715366+0000","last_active":"2024-04-26T03:39:12.727071+0000","last_peered":"2024-04-26T03:39:12.727071+0000","last_clean":"2024-04-26T03:39:12.727071+0000","last_became_active":"2024-04-26T03:39:01.546594+0000","last_became_peered":"2024-04-26T03:39:01.546594+0000","last_unstale":"2024-04-26T03:39:12.727071+0000","last_undegraded":"2024-04-26T03:39:12.727071+0000","last_fullsized":"2024-04-26T03:39:12.727071+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":5,"last_scrub":"35'3","last_scrub_stamp":"2024-04-26T03:39:06.216135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:06.216135+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:09.099934+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0010693810000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,14,6,2,12,7,1],"acting":[0,14,6,2,12,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"5","length":"5"}]},{"pgid":"3.15","version":"42'24","reported_seq":158,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T03:39:11.715836+0000","last_change":"2024-04-26T03:39:07.696106+0000","last_active":"2024-04-26T03:39:11.715836+0000","last_peered":"2024-04-26T03:39:11.715836+0000","last_clean":"2024-04-26T03:39:11.715836+0000","last_became_active":"2024-04-26T03:39:01.543191+0000","last_became_peered":"2024-04-26T03:39:01.543191+0000","last_unstale":"2024-04-26T03:39:11.715836+0000","last_undegraded":"2024-04-26T03:39:11.715836+0000","last_fullsized":"2024-04-26T03:39:11.715836+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":5,"last_scrub":"42'24","last_scrub_stamp":"2024-04-26T03:39:07.185893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:07.185893+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:14.607375+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00091493100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,8,4,2,7,0,5],"acting":[15,8,4,2,7,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"5","length":"5"}]},{"pgid":"3.14","version":"52'35","reported_seq":181,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T03:39:11.726563+0000","last_change":"2024-04-26T03:39:07.714451+0000","last_active":"2024-04-26T03:39:11.726563+0000","last_peered":"2024-04-26T03:39:11.726563+0000","last_clean":"2024-04-26T03:39:11.726563+0000","last_became_active":"2024-04-26T03:39:01.550254+0000","last_became_peered":"2024-04-26T03:39:01.550254+0000","last_unstale":"2024-04-26T03:39:11.726563+0000","last_undegraded":"2024-04-26T03:39:11.726563+0000","last_fullsized":"2024-04-26T03:39:11.726563+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":5,"last_scrub":"50'29","last_scrub_stamp":"2024-04-26T03:39:06.353377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:06.353377+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:24.037510+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.00076574599999999998,"stat_sum":{"num_bytes":6946816,"num_objects":4,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":9459,"num_write":25,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,13,4,2,6,10,5],"acting":[8,13,4,2,6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"5","length":"5"}]},{"pgid":"2.7","version":"0'0","reported_seq":68,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T03:39:11.715105+0000","last_change":"2024-04-26T03:38:37.359252+0000","last_active":"2024-04-26T03:39:11.715105+0000","last_peered":"2024-04-26T03:39:11.715105+0000","last_clean":"2024-04-26T03:39:11.715105+0000","last_became_active":"2024-04-26T03:38:37.358910+0000","last_became_peered":"2024-04-26T03:38:37.358910+0000","last_unstale":"2024-04-26T03:39:11.715105+0000","last_undegraded":"2024-04-26T03:39:11.715105+0000","last_fullsized":"2024-04-26T03:39:11.715105+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:04.186001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"3.6","version":"50'9","reported_seq":145,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-26T03:39:12.727096+0000","last_change":"2024-04-26T03:39:07.715748+0000","last_active":"2024-04-26T03:39:12.727096+0000","last_peered":"2024-04-26T03:39:12.727096+0000","last_clean":"2024-04-26T03:39:12.727096+0000","last_became_active":"2024-04-26T03:39:01.543735+0000","last_became_peered":"2024-04-26T03:39:01.543735+0000","last_unstale":"2024-04-26T03:39:12.727096+0000","last_undegraded":"2024-04-26T03:39:12.727096+0000","last_fullsized":"2024-04-26T03:39:12.727096+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'9","last_scrub_stamp":"2024-04-26T03:39:07.177824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:07.177824+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:28.637664+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0011943170000000001,"stat_sum":{"num_bytes":819200,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,14,6,2,12,7,1],"acting":[0,14,6,2,12,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"5","length":"5"}]},{"pgid":"2.6","version":"0'0","reported_seq":68,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T03:39:11.726660+0000","last_change":"2024-04-26T03:38:36.484782+0000","last_active":"2024-04-26T03:39:11.726660+0000","last_peered":"2024-04-26T03:39:11.726660+0000","last_clean":"2024-04-26T03:39:11.726660+0000","last_became_active":"2024-04-26T03:38:36.484388+0000","last_became_peered":"2024-04-26T03:38:36.484388+0000","last_unstale":"2024-04-26T03:39:11.726660+0000","last_undegraded":"2024-04-26T03:39:11.726660+0000","last_fullsized":"2024-04-26T03:39:11.726660+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:36.318287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,13],"acting":[8,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.7","version":"49'10","reported_seq":146,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-26T03:39:12.723929+0000","last_change":"2024-04-26T03:39:09.248780+0000","last_active":"2024-04-26T03:39:12.723929+0000","last_peered":"2024-04-26T03:39:12.723929+0000","last_clean":"2024-04-26T03:39:12.723929+0000","last_became_active":"2024-04-26T03:39:01.549431+0000","last_became_peered":"2024-04-26T03:39:01.549431+0000","last_unstale":"2024-04-26T03:39:12.723929+0000","last_undegraded":"2024-04-26T03:39:12.723929+0000","last_fullsized":"2024-04-26T03:39:12.723929+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'10","last_scrub_stamp":"2024-04-26T03:39:09.248660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:09.248660+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:25.681455+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.00061997599999999995,"stat_sum":{"num_bytes":131072,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,13,6,0,14,10,11],"acting":[4,13,6,0,14,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"5","length":"5"}]},{"pgid":"2.5","version":"0'0","reported_seq":67,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T03:39:11.709345+0000","last_change":"2024-04-26T03:38:38.187714+0000","last_active":"2024-04-26T03:39:11.709345+0000","last_peered":"2024-04-26T03:39:11.709345+0000","last_clean":"2024-04-26T03:39:11.709345+0000","last_became_active":"2024-04-26T03:38:38.187383+0000","last_became_peered":"2024-04-26T03:38:38.187383+0000","last_unstale":"2024-04-26T03:39:11.709345+0000","last_undegraded":"2024-04-26T03:39:11.709345+0000","last_fullsized":"2024-04-26T03:39:11.709345+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:59.062623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4],"acting":[9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.4","version":"42'18","reported_seq":155,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T03:39:11.726739+0000","last_change":"2024-04-26T03:39:07.714823+0000","last_active":"2024-04-26T03:39:11.726739+0000","last_peered":"2024-04-26T03:39:11.726739+0000","last_clean":"2024-04-26T03:39:11.726739+0000","last_became_active":"2024-04-26T03:39:01.543050+0000","last_became_peered":"2024-04-26T03:39:01.543050+0000","last_unstale":"2024-04-26T03:39:11.726739+0000","last_undegraded":"2024-04-26T03:39:11.726739+0000","last_fullsized":"2024-04-26T03:39:11.726739+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"42'18","last_scrub_stamp":"2024-04-26T03:39:07.342868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:07.342868+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:26.284626+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00084743500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,13,4,2,6,10,5],"acting":[8,13,4,2,6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"5","length":"5"}]},{"pgid":"2.4","version":"0'0","reported_seq":69,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-26T03:39:12.723544+0000","last_change":"2024-04-26T03:38:37.255389+0000","last_active":"2024-04-26T03:39:12.723544+0000","last_peered":"2024-04-26T03:39:12.723544+0000","last_clean":"2024-04-26T03:39:12.723544+0000","last_became_active":"2024-04-26T03:38:37.255066+0000","last_became_peered":"2024-04-26T03:38:37.255066+0000","last_unstale":"2024-04-26T03:39:12.723544+0000","last_undegraded":"2024-04-26T03:39:12.723544+0000","last_fullsized":"2024-04-26T03:39:12.723544+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:43.860480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.5","version":"56'57","reported_seq":195,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T03:39:11.745963+0000","last_change":"2024-04-26T03:39:07.696235+0000","last_active":"2024-04-26T03:39:11.745963+0000","last_peered":"2024-04-26T03:39:11.745963+0000","last_clean":"2024-04-26T03:39:11.745963+0000","last_became_active":"2024-04-26T03:39:01.540394+0000","last_became_peered":"2024-04-26T03:39:01.540394+0000","last_unstale":"2024-04-26T03:39:11.745963+0000","last_undegraded":"2024-04-26T03:39:11.745963+0000","last_fullsized":"2024-04-26T03:39:11.745963+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'51","last_scrub_stamp":"2024-04-26T03:39:06.149763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:06.149763+0000","objects_scrubbed":1,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:10.126071+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00097181500000000001,"stat_sum":{"num_bytes":4341760,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":10,"num_read_kb":12724,"num_write":24,"num_write_kb":1696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,8,4,2,7,0,5],"acting":[15,8,4,2,7,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"5","length":"5"}]},{"pgid":"3.3","version":"52'33","reported_seq":177,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T03:39:11.709034+0000","last_change":"2024-04-26T03:39:07.699126+0000","last_active":"2024-04-26T03:39:11.709034+0000","last_peered":"2024-04-26T03:39:11.709034+0000","last_clean":"2024-04-26T03:39:11.709034+0000","last_became_active":"2024-04-26T03:39:01.550287+0000","last_became_peered":"2024-04-26T03:39:01.550287+0000","last_unstale":"2024-04-26T03:39:11.709034+0000","last_undegraded":"2024-04-26T03:39:11.709034+0000","last_fullsized":"2024-04-26T03:39:11.709034+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'32","last_scrub_stamp":"2024-04-26T03:39:07.451867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:07.451867+0000","objects_scrubbed":1,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:26.660052+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0019103900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,7,4,15,1,2,3],"acting":[10,7,4,15,1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"5","length":"5"}]},{"pgid":"2.2","version":"0'0","reported_seq":56,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-26T03:39:10.710968+0000","last_change":"2024-04-26T03:38:48.475569+0000","last_active":"2024-04-26T03:39:10.710968+0000","last_peered":"2024-04-26T03:39:10.710968+0000","last_clean":"2024-04-26T03:39:10.710968+0000","last_became_active":"2024-04-26T03:38:48.475155+0000","last_became_peered":"2024-04-26T03:38:48.475155+0000","last_unstale":"2024-04-26T03:39:10.710968+0000","last_undegraded":"2024-04-26T03:39:10.710968+0000","last_fullsized":"2024-04-26T03:39:10.710968+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:38.214686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,12],"acting":[11,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.0","version":"35'9","reported_seq":147,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-26T03:39:12.727417+0000","last_change":"2024-04-26T03:39:07.695314+0000","last_active":"2024-04-26T03:39:12.727417+0000","last_peered":"2024-04-26T03:39:12.727417+0000","last_clean":"2024-04-26T03:39:12.727417+0000","last_became_active":"2024-04-26T03:39:01.540712+0000","last_became_peered":"2024-04-26T03:39:01.540712+0000","last_unstale":"2024-04-26T03:39:12.727417+0000","last_undegraded":"2024-04-26T03:39:12.727417+0000","last_fullsized":"2024-04-26T03:39:12.727417+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'9","last_scrub_stamp":"2024-04-26T03:39:06.119088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:06.119088+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:24.387299+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00075975399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,9,15,2,6,14,4],"acting":[1,9,15,2,6,14,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"5","length":"5"}]},{"pgid":"2.1","version":"0'0","reported_seq":68,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T03:39:11.711909+0000","last_change":"2024-04-26T03:38:37.255450+0000","last_active":"2024-04-26T03:39:11.711909+0000","last_peered":"2024-04-26T03:39:11.711909+0000","last_clean":"2024-04-26T03:39:11.711909+0000","last_became_active":"2024-04-26T03:38:37.255181+0000","last_became_peered":"2024-04-26T03:38:37.255181+0000","last_unstale":"2024-04-26T03:39:11.711909+0000","last_undegraded":"2024-04-26T03:39:11.711909+0000","last_fullsized":"2024-04-26T03:39:11.711909+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:00.449388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0],"acting":[14,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.1","version":"42'8","reported_seq":143,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-26T03:39:12.723946+0000","last_change":"2024-04-26T03:39:08.263995+0000","last_active":"2024-04-26T03:39:12.723946+0000","last_peered":"2024-04-26T03:39:12.723946+0000","last_clean":"2024-04-26T03:39:12.723946+0000","last_became_active":"2024-04-26T03:39:01.547451+0000","last_became_peered":"2024-04-26T03:39:01.547451+0000","last_unstale":"2024-04-26T03:39:12.723946+0000","last_undegraded":"2024-04-26T03:39:12.723946+0000","last_fullsized":"2024-04-26T03:39:12.723946+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"42'8","last_scrub_stamp":"2024-04-26T03:39:08.263903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:08.263903+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:27.836763+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00049398599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,14,10,9,1,2],"acting":[4,11,14,10,9,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"5","length":"5"}]},{"pgid":"2.0","version":"0'0","reported_seq":71,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T03:39:13.751973+0000","last_change":"2024-04-26T03:38:36.431325+0000","last_active":"2024-04-26T03:39:13.751973+0000","last_peered":"2024-04-26T03:39:13.751973+0000","last_clean":"2024-04-26T03:39:13.751973+0000","last_became_active":"2024-04-26T03:38:36.431121+0000","last_became_peered":"2024-04-26T03:38:36.431121+0000","last_unstale":"2024-04-26T03:39:13.751973+0000","last_undegraded":"2024-04-26T03:39:13.751973+0000","last_fullsized":"2024-04-26T03:39:13.751973+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:53.294619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"56'41","reported_seq":187,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T03:39:11.712030+0000","last_change":"2024-04-26T03:39:07.703716+0000","last_active":"2024-04-26T03:39:11.712030+0000","last_peered":"2024-04-26T03:39:11.712030+0000","last_clean":"2024-04-26T03:39:11.712030+0000","last_became_active":"2024-04-26T03:39:01.543967+0000","last_became_peered":"2024-04-26T03:39:01.543967+0000","last_unstale":"2024-04-26T03:39:11.712030+0000","last_undegraded":"2024-04-26T03:39:11.712030+0000","last_fullsized":"2024-04-26T03:39:11.712030+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'31","last_scrub_stamp":"2024-04-26T03:39:07.479325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:07.479325+0000","objects_scrubbed":3,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:10.744722+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.00061490099999999995,"stat_sum":{"num_bytes":9142272,"num_objects":4,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":7,"num_read_kb":4258,"num_write":9,"num_write_kb":3344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,5,3,9,15,10,11],"acting":[14,5,3,9,15,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"5","length":"5"}]},{"pgid":"2.3","version":"0'0","reported_seq":68,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T03:39:11.712013+0000","last_change":"2024-04-26T03:38:36.431667+0000","last_active":"2024-04-26T03:39:11.712013+0000","last_peered":"2024-04-26T03:39:11.712013+0000","last_clean":"2024-04-26T03:39:11.712013+0000","last_became_active":"2024-04-26T03:38:36.431449+0000","last_became_peered":"2024-04-26T03:38:36.431449+0000","last_unstale":"2024-04-26T03:39:11.712013+0000","last_undegraded":"2024-04-26T03:39:11.712013+0000","last_fullsized":"2024-04-26T03:39:11.712013+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:47.562648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"27'157","reported_seq":269,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T03:39:11.733942+0000","last_change":"2024-04-26T03:38:34.390206+0000","last_active":"2024-04-26T03:39:11.733942+0000","last_peered":"2024-04-26T03:39:11.733942+0000","last_clean":"2024-04-26T03:39:11.733942+0000","last_became_active":"2024-04-26T03:38:34.389099+0000","last_became_peered":"2024-04-26T03:38:34.389099+0000","last_unstale":"2024-04-26T03:39:11.733942+0000","last_undegraded":"2024-04-26T03:39:11.733942+0000","last_fullsized":"2024-04-26T03:39:11.733942+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:32.306810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:32.306810+0000","last_clean_scrub_stamp":"2024-04-26T03:38:32.306810+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:40.216975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.8","version":"42'12","reported_seq":156,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T03:39:11.733810+0000","last_change":"2024-04-26T03:39:08.136725+0000","last_active":"2024-04-26T03:39:11.733810+0000","last_peered":"2024-04-26T03:39:11.733810+0000","last_clean":"2024-04-26T03:39:11.733810+0000","last_became_active":"2024-04-26T03:39:01.546346+0000","last_became_peered":"2024-04-26T03:39:01.546346+0000","last_unstale":"2024-04-26T03:39:11.733810+0000","last_undegraded":"2024-04-26T03:39:11.733810+0000","last_fullsized":"2024-04-26T03:39:11.733810+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"42'12","last_scrub_stamp":"2024-04-26T03:39:08.136579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:08.136579+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:13.276207+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.001125816,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,10,11,15,5],"acting":[6,0,1,10,11,15,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"5","length":"5"}]},{"pgid":"3.9","version":"52'11","reported_seq":157,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T03:39:11.940886+0000","last_change":"2024-04-26T03:39:07.728514+0000","last_active":"2024-04-26T03:39:11.940886+0000","last_peered":"2024-04-26T03:39:11.940886+0000","last_clean":"2024-04-26T03:39:11.940886+0000","last_became_active":"2024-04-26T03:39:01.541605+0000","last_became_peered":"2024-04-26T03:39:01.541605+0000","last_unstale":"2024-04-26T03:39:11.940886+0000","last_undegraded":"2024-04-26T03:39:11.940886+0000","last_fullsized":"2024-04-26T03:39:11.940886+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'6","last_scrub_stamp":"2024-04-26T03:39:07.345704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:07.345704+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:30.892500+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.018295509000000001,"stat_sum":{"num_bytes":2244608,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":1586,"num_write":5,"num_write_kb":1664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2,10,9,15,8,4],"acting":[13,2,10,9,15,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"5","length":"5"}]},{"pgid":"3.a","version":"58'52","reported_seq":205,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T03:39:13.752225+0000","last_change":"2024-04-26T03:39:07.703520+0000","last_active":"2024-04-26T03:39:13.752225+0000","last_peered":"2024-04-26T03:39:13.752225+0000","last_clean":"2024-04-26T03:39:13.752225+0000","last_became_active":"2024-04-26T03:38:49.486365+0000","last_became_peered":"2024-04-26T03:38:49.486365+0000","last_unstale":"2024-04-26T03:39:13.752225+0000","last_undegraded":"2024-04-26T03:39:13.752225+0000","last_fullsized":"2024-04-26T03:39:13.752225+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:55.308889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067771699999999999,"stat_sum":{"num_bytes":8978432,"num_objects":8,"num_object_clones":2,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":55,"num_read_kb":22436,"num_write":73,"num_write_kb":10544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,10,13,1,11],"acting":[3,7,6,10,13,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"5","length":"5"}]},{"pgid":"3.b","version":"57'63","reported_seq":194,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-26T03:39:12.781177+0000","last_change":"2024-04-26T03:39:07.700277+0000","last_active":"2024-04-26T03:39:12.781177+0000","last_peered":"2024-04-26T03:39:12.781177+0000","last_clean":"2024-04-26T03:39:12.781177+0000","last_became_active":"2024-04-26T03:38:49.940561+0000","last_became_peered":"2024-04-26T03:38:49.940561+0000","last_unstale":"2024-04-26T03:39:12.781177+0000","last_undegraded":"2024-04-26T03:39:12.781177+0000","last_fullsized":"2024-04-26T03:39:12.781177+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:13.530246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000510053,"stat_sum":{"num_bytes":7897088,"num_objects":5,"num_object_clones":1,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":6543,"num_write":100,"num_write_kb":10864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,10,12,1,7,14],"acting":[4,0,10,12,1,7,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"5","length":"5"}]},{"pgid":"3.c","version":"52'40","reported_seq":153,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-26T03:39:10.711250+0000","last_change":"2024-04-26T03:39:07.695340+0000","last_active":"2024-04-26T03:39:10.711250+0000","last_peered":"2024-04-26T03:39:10.711250+0000","last_clean":"2024-04-26T03:39:10.711250+0000","last_became_active":"2024-04-26T03:38:49.941017+0000","last_became_peered":"2024-04-26T03:38:49.941017+0000","last_unstale":"2024-04-26T03:39:10.711250+0000","last_undegraded":"2024-04-26T03:39:10.711250+0000","last_fullsized":"2024-04-26T03:39:10.711250+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:17.328263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000834028,"stat_sum":{"num_bytes":1720320,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":5136,"num_write":38,"num_write_kb":5696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,0,1,9,12,14],"acting":[11,8,0,1,9,12,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"5","length":"5"}]},{"pgid":"3.d","version":"56'28","reported_seq":142,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T03:39:11.746709+0000","last_change":"2024-04-26T03:39:07.704287+0000","last_active":"2024-04-26T03:39:11.746709+0000","last_peered":"2024-04-26T03:39:11.746709+0000","last_clean":"2024-04-26T03:39:11.746709+0000","last_became_active":"2024-04-26T03:38:49.511783+0000","last_became_peered":"2024-04-26T03:38:49.511783+0000","last_unstale":"2024-04-26T03:39:11.746709+0000","last_undegraded":"2024-04-26T03:39:11.746709+0000","last_fullsized":"2024-04-26T03:39:11.746709+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:07.502456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091924699999999999,"stat_sum":{"num_bytes":2506752,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":8,"num_read_kb":4,"num_write":33,"num_write_kb":2736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,9,1,5,8,4],"acting":[7,3,9,1,5,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"5","length":"5"}]},{"pgid":"3.e","version":"52'42","reported_seq":174,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T03:39:11.940929+0000","last_change":"2024-04-26T03:39:07.716455+0000","last_active":"2024-04-26T03:39:11.940929+0000","last_peered":"2024-04-26T03:39:11.940929+0000","last_clean":"2024-04-26T03:39:11.940929+0000","last_became_active":"2024-04-26T03:38:49.941260+0000","last_became_peered":"2024-04-26T03:38:49.941260+0000","last_unstale":"2024-04-26T03:39:11.940929+0000","last_undegraded":"2024-04-26T03:39:11.940929+0000","last_fullsized":"2024-04-26T03:39:11.940929+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:57.035864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061539480000000002,"stat_sum":{"num_bytes":6029312,"num_objects":4,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":33,"num_read_kb":20221,"num_write":43,"num_write_kb":6208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,14,6,11,8,9,12],"acting":[13,14,6,11,8,9,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"5","length":"5"}]},{"pgid":"3.f","version":"52'34","reported_seq":157,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-26T03:39:10.711331+0000","last_change":"2024-04-26T03:39:07.695404+0000","last_active":"2024-04-26T03:39:10.711331+0000","last_peered":"2024-04-26T03:39:10.711331+0000","last_clean":"2024-04-26T03:39:10.711331+0000","last_became_active":"2024-04-26T03:38:49.486850+0000","last_became_peered":"2024-04-26T03:38:49.486850+0000","last_unstale":"2024-04-26T03:39:10.711331+0000","last_undegraded":"2024-04-26T03:39:10.711331+0000","last_fullsized":"2024-04-26T03:39:10.711331+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:14.313849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070240999999999999,"stat_sum":{"num_bytes":2342912,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":5480,"num_write":36,"num_write_kb":3680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,13,5,14,2,1,0],"acting":[11,13,5,14,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"5","length":"5"}]},{"pgid":"3.10","version":"56'14","reported_seq":81,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T03:39:11.741688+0000","last_change":"2024-04-26T03:39:07.710846+0000","last_active":"2024-04-26T03:39:11.741688+0000","last_peered":"2024-04-26T03:39:11.741688+0000","last_clean":"2024-04-26T03:39:11.741688+0000","last_became_active":"2024-04-26T03:39:04.704276+0000","last_became_peered":"2024-04-26T03:39:04.704276+0000","last_unstale":"2024-04-26T03:39:11.741688+0000","last_undegraded":"2024-04-26T03:39:11.741688+0000","last_fullsized":"2024-04-26T03:39:11.741688+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":5,"last_scrub":"35'9","last_scrub_stamp":"2024-04-26T03:39:07.110098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:07.110098+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:29.316949+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00083012699999999995,"stat_sum":{"num_bytes":1015808,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,7,4,8,0,11],"acting":[6,9,7,4,8,0,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"5","length":"5"}]},{"pgid":"3.11","version":"58'10","reported_seq":59,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T03:39:13.738702+0000","last_change":"2024-04-26T03:39:08.233857+0000","last_active":"2024-04-26T03:39:13.738702+0000","last_peered":"2024-04-26T03:39:13.738702+0000","last_clean":"2024-04-26T03:39:13.738702+0000","last_became_active":"2024-04-26T03:39:07.719054+0000","last_became_peered":"2024-04-26T03:39:07.719054+0000","last_unstale":"2024-04-26T03:39:13.738702+0000","last_undegraded":"2024-04-26T03:39:13.738702+0000","last_fullsized":"2024-04-26T03:39:13.738702+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":5,"last_scrub":"42'8","last_scrub_stamp":"2024-04-26T03:39:08.233750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:08.233750+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:35.106997+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.000434886,"stat_sum":{"num_bytes":2621440,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,15,10,0,14,2],"acting":[5,11,15,10,0,14,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"5","length":"5"}]},{"pgid":"3.12","version":"35'11","reported_seq":155,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T03:39:11.711937+0000","last_change":"2024-04-26T03:39:09.918260+0000","last_active":"2024-04-26T03:39:11.711937+0000","last_peered":"2024-04-26T03:39:11.711937+0000","last_clean":"2024-04-26T03:39:11.711937+0000","last_became_active":"2024-04-26T03:39:09.917254+0000","last_became_peered":"2024-04-26T03:39:09.917254+0000","last_unstale":"2024-04-26T03:39:11.711937+0000","last_undegraded":"2024-04-26T03:39:11.711937+0000","last_fullsized":"2024-04-26T03:39:11.711937+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":5,"last_scrub":"35'11","last_scrub_stamp":"2024-04-26T03:39:06.494007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:06.494007+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:09.259949+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.00066597499999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,7,8,9,4,3,15],"acting":[14,7,8,9,4,3,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"5","length":"5"}]},{"pgid":"3.13","version":"52'45","reported_seq":2,"reported_epoch":55,"state":"peering","last_fresh":"2024-04-26T03:39:08.010510+0000","last_change":"2024-04-26T03:39:11.289971+0000","last_active":"2024-04-26T03:39:11.289971+0000","last_peered":"2024-04-26T03:39:08.010510+0000","last_clean":"2024-04-26T03:39:08.010510+0000","last_became_active":"2024-04-26T03:39:01.561931+0000","last_became_peered":"2024-04-26T03:39:01.561931+0000","last_unstale":"2024-04-26T03:39:08.010510+0000","last_undegraded":"2024-04-26T03:39:08.010510+0000","last_fullsized":"2024-04-26T03:39:08.010510+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":5,"last_scrub":"50'40","last_scrub_stamp":"2024-04-26T03:39:06.466060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:06.466060+0000","objects_scrubbed":4,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.00080123,"stat_sum":{"num_bytes":8044544,"num_objects":3,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":6,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,7,2,6,0,4,1],"acting":[11,7,2,6,0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"5","length":"5"}]}],"pool_stats":[{"poolid":3,"num_pg":25,"stat_sum":{"num_bytes":72581120,"num_objects":48,"num_object_clones":8,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":6,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":1,"num_read":205,"num_read_kb":103501,"num_write":443,"num_write_kb":55360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":134733824,"data_stored":146477056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":670,"ondisk_log_size":670,"up":175,"acting":175,"num_store_stats":16},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":13852,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":24,"seq":103079215117,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42192,"kb_used_data":11008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94329648,"statfs":{"total":96636764160,"available":96593559552,"internally_reserved":0,"allocated":11272192,"data_stored":13456971,"data_compressed":74490,"data_compressed_allocated":172032,"data_compressed_original":790528,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,4,6,7,8,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215117,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32672,"kb_used_data":5644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339168,"statfs":{"total":96636764160,"available":96603308032,"internally_reserved":0,"allocated":5779456,"data_stored":6206507,"data_compressed":67564,"data_compressed_allocated":139264,"data_compressed_original":278528,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":23,"seq":98784247822,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40536,"kb_used_data":9348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331304,"statfs":{"total":96636764160,"available":96595255296,"internally_reserved":0,"allocated":9572352,"data_stored":9862157,"data_compressed":65536,"data_compressed_allocated":135168,"data_compressed_original":270336,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280523,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37544,"kb_used_data":10524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94334296,"statfs":{"total":96636764160,"available":96598319104,"internally_reserved":0,"allocated":10776576,"data_stored":11046445,"data_compressed":72462,"data_compressed_allocated":167936,"data_compressed_original":782336,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,4,5,7,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280523,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35520,"kb_used_data":8496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94336320,"statfs":{"total":96636764160,"available":96600391680,"internally_reserved":0,"allocated":8699904,"data_stored":8989709,"data_compressed":65536,"data_compressed_allocated":135168,"data_compressed_original":270336,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,3,4,5,6,8,9,10,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":8,"up_from":22,"seq":94489280524,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34176,"kb_used_data":7152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337664,"statfs":{"total":96636764160,"available":96601767936,"internally_reserved":0,"allocated":7323648,"data_stored":7613453,"data_compressed":65536,"data_compressed_allocated":135168,"data_compressed_original":270336,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[2,4,5,6,7,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280522,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33188,"kb_used_data":6164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338652,"statfs":{"total":96636764160,"available":96602779648,"internally_reserved":0,"allocated":6311936,"data_stored":6106125,"data_compressed":65536,"data_compressed_allocated":135168,"data_compressed_original":270336,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,2,4,7,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280523,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40300,"kb_used_data":13280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94331540,"statfs":{"total":96636764160,"available":96595496960,"internally_reserved":0,"allocated":13598720,"data_stored":13394919,"data_compressed":63514,"data_compressed_allocated":131072,"data_compressed_original":262144,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[1,2,3,4,7,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313228,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36844,"kb_used_data":9816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94334996,"statfs":{"total":96636764160,"available":96599035904,"internally_reserved":0,"allocated":10051584,"data_stored":9849801,"data_compressed":61490,"data_compressed_allocated":126976,"data_compressed_original":253952,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,5,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":12,"up_from":22,"seq":94489280522,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31168,"kb_used_data":4144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340672,"statfs":{"total":96636764160,"available":96604848128,"internally_reserved":0,"allocated":4243456,"data_stored":4037645,"data_compressed":65536,"data_compressed_allocated":135168,"data_compressed_original":270336,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,7,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280522,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34148,"kb_used_data":7128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337692,"statfs":{"total":96636764160,"available":96601796608,"internally_reserved":0,"allocated":7299072,"data_stored":7095271,"data_compressed":63514,"data_compressed_allocated":131072,"data_compressed_original":262144,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[2,4,6,7,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247820,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43024,"kb_used_data":11844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94328816,"statfs":{"total":96636764160,"available":96592707584,"internally_reserved":0,"allocated":12128256,"data_stored":14242859,"data_compressed":67564,"data_compressed_allocated":139264,"data_compressed_original":278528,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,6,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280522,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34628,"kb_used_data":7608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337212,"statfs":{"total":96636764160,"available":96601305088,"internally_reserved":0,"allocated":7790592,"data_stored":7586791,"data_compressed":63514,"data_compressed_allocated":131072,"data_compressed_original":262144,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,3,5,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":23,"seq":98784247822,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43196,"kb_used_data":12016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94328644,"statfs":{"total":96636764160,"available":96592531456,"internally_reserved":0,"allocated":12304384,"data_stored":14619691,"data_compressed":67564,"data_compressed_allocated":139264,"data_compressed_original":278528,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,4,6,7,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280522,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34504,"kb_used_data":7480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337336,"statfs":{"total":96636764160,"available":96601432064,"internally_reserved":0,"allocated":7659520,"data_stored":7887885,"data_compressed":65536,"data_compressed_allocated":135168,"data_compressed_original":270336,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,4,5,7,8,9,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247820,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39600,"kb_used_data":8412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332240,"statfs":{"total":96636764160,"available":96596213760,"internally_reserved":0,"allocated":8613888,"data_stored":8842253,"data_compressed":65536,"data_compressed_allocated":135168,"data_compressed_original":270336,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":11505664,"data_stored":13828096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":11681792,"data_stored":14204928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8671232,"data_stored":9105408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5345280,"data_stored":6356992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":10113024,"data_stored":11075584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":11292672,"data_stored":14295040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":10113024,"data_stored":10113024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8089600,"data_stored":8585216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":6713344,"data_stored":7208960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":6176768,"data_stored":6238208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":10989568,"data_stored":10989568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":9465856,"data_stored":9465856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":3633152,"data_stored":3633152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":6701056,"data_stored":6701056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":7192576,"data_stored":7192576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":7049216,"data_stored":7483392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T03:39:16.613 INFO:tasks.thrashosds.thrasher:Setting 3.f to [2, 9, 4, 13, 7, 1, 15, 10, 3, 11, 5, 14, 8, 0] 2024-04-26T03:39:16.613 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.f', '2', '9', '4', '13', '7', '1', '15', '10', '3', '11', '5', '14', '8', '0'] 2024-04-26T03:39:16.613 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.f 2 9 4 13 7 1 15 10 3 11 5 14 8 0 2024-04-26T03:39:16.707 INFO:tasks.ceph.osd.10.smithi066.stderr:2024-04-26T03:39:16.706+0000 7f130fcc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 31763) UID: 0 2024-04-26T03:39:16.709 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-26T03:39:16.721 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-26T03:39:16.721 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-04-26T03:39:16.721 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-04-26T03:39:16.721 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-26T03:39:16.731 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-04-26T03:39:16.735 INFO:tasks.rados.rados.0.smithi092.stdout:264: done (7 left) 2024-04-26T03:39:16.736 INFO:tasks.rados.rados.0.smithi092.stdout:265: done (6 left) 2024-04-26T03:39:16.736 INFO:tasks.rados.rados.0.smithi092.stdout:266: done (5 left) 2024-04-26T03:39:16.736 INFO:tasks.rados.rados.0.smithi092.stdout:267: done (4 left) 2024-04-26T03:39:16.736 INFO:tasks.rados.rados.0.smithi092.stdout:268: done (3 left) 2024-04-26T03:39:16.736 INFO:tasks.rados.rados.0.smithi092.stdout:269: done (2 left) 2024-04-26T03:39:16.736 INFO:tasks.rados.rados.0.smithi092.stdout:270: done (1 left) 2024-04-26T03:39:16.736 INFO:tasks.rados.rados.0.smithi092.stdout:271: done (0 left) 2024-04-26T03:39:16.736 INFO:tasks.rados.rados.0.smithi092.stdout:272: snap_create 2024-04-26T03:39:16.809 INFO:tasks.ceph.osd.14.smithi066.stderr:2024-04-26T03:39:16.808+0000 7f5395f08640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 31765) UID: 0 2024-04-26T03:39:16.838 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-26T03:39:16.839 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-04-26T03:39:16.839 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-04-26T03:39:16.839 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-04-26T03:39:16.839 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-26T03:39:16.847 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-04-26T03:39:16.910 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:39:16.908+0000 7fba0d2ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31961) UID: 0 2024-04-26T03:39:16.955 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-26T03:39:16.955 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-04-26T03:39:16.956 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-04-26T03:39:16.956 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-04-26T03:39:16.956 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.956 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:47 3.9s0 3:9d1726ae:::smithi09235267-16:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:16.956 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.504120+0000", 2024-04-26T03:39:16.956 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.448864268000001, 2024-04-26T03:39:16.956 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011964519999999999, 2024-04-26T03:39:16.956 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:16.956 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:16.956 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:16.956 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:16.956 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:16.956 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:16.957 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47 2024-04-26T03:39:16.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.957 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:16.957 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.957 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:16.957 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504120+0000", 2024-04-26T03:39:16.957 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.957 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.957 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:16.957 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504120+0000", 2024-04-26T03:39:16.957 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.958 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.958 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:16.958 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504121+0000", 2024-04-26T03:39:16.958 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0359999999999999e-06 2024-04-26T03:39:16.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.958 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.958 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:16.958 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504127+0000", 2024-04-26T03:39:16.958 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1750000000000002e-06 2024-04-26T03:39:16.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.958 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.958 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:16.958 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504138+0000", 2024-04-26T03:39:16.959 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1045e-05 2024-04-26T03:39:16.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.959 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.959 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:16.959 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504145+0000", 2024-04-26T03:39:16.959 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5919999999999997e-06 2024-04-26T03:39:16.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.959 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.959 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:16.959 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.508573+0000", 2024-04-26T03:39:16.959 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.004427762 2024-04-26T03:39:16.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.959 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.959 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:16.960 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.508680+0000", 2024-04-26T03:39:16.960 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000107106 2024-04-26T03:39:16.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.960 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:16.960 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.509206+0000", 2024-04-26T03:39:16.960 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00052654999999999998 2024-04-26T03:39:16.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.960 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:16.960 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.510170+0000", 2024-04-26T03:39:16.960 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00096335099999999999 2024-04-26T03:39:16.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.961 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:39:16.961 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.515986+0000", 2024-04-26T03:39:16.961 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0058160449999999997 2024-04-26T03:39:16.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.961 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.961 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:16.961 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.516084+0000", 2024-04-26T03:39:16.961 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8857999999999994e-05 2024-04-26T03:39:16.961 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.961 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:16.961 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.961 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.961 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:48 3.9s0 3:9d1726ae:::smithi09235267-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:16.962 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.504164+0000", 2024-04-26T03:39:16.962 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.448820347000002, 2024-04-26T03:39:16.962 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.014750030000000001, 2024-04-26T03:39:16.962 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:16.962 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:16.962 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-04-26T03:39:16.962 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:16.962 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:16.962 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:16.962 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 48 2024-04-26T03:39:16.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.962 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:16.962 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.962 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:16.963 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504164+0000", 2024-04-26T03:39:16.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:16.963 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504164+0000", 2024-04-26T03:39:16.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:16.964 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504165+0000", 2024-04-26T03:39:16.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8999999999999996e-07 2024-04-26T03:39:16.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.964 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.964 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:16.964 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504169+0000", 2024-04-26T03:39:16.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0239999999999999e-06 2024-04-26T03:39:16.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.964 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.964 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:16.965 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504178+0000", 2024-04-26T03:39:16.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7610000000000002e-06 2024-04-26T03:39:16.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.965 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:16.965 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504187+0000", 2024-04-26T03:39:16.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1799999999999996e-06 2024-04-26T03:39:16.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.965 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:16.965 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.509439+0000", 2024-04-26T03:39:16.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0052526150000000004 2024-04-26T03:39:16.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:16.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.509517+0000", 2024-04-26T03:39:16.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7417999999999996e-05 2024-04-26T03:39:16.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:16.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.516457+0000", 2024-04-26T03:39:16.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0069402259999999999 2024-04-26T03:39:16.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:16.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.516564+0000", 2024-04-26T03:39:16.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010746799999999999 2024-04-26T03:39:16.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:16.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.518914+0000", 2024-04-26T03:39:16.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023496480000000002 2024-04-26T03:39:16.968 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.968 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:16.968 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.968 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:60 3.es0 3:7db045cf:::smithi09235267-20:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:16.968 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.512353+0000", 2024-04-26T03:39:16.968 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.440630897999998, 2024-04-26T03:39:16.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015276305, 2024-04-26T03:39:16.968 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:16.968 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:16.968 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-04-26T03:39:16.968 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:16.968 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:16.969 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:16.969 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 60 2024-04-26T03:39:16.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.969 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:16.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.969 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:16.969 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.512353+0000", 2024-04-26T03:39:16.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.969 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:16.969 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.512353+0000", 2024-04-26T03:39:16.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:16.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.512354+0000", 2024-04-26T03:39:16.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3600000000000003e-07 2024-04-26T03:39:16.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:16.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.512357+0000", 2024-04-26T03:39:16.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3050000000000001e-06 2024-04-26T03:39:16.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:16.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.512365+0000", 2024-04-26T03:39:16.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1950000000000003e-06 2024-04-26T03:39:16.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:16.971 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.512372+0000", 2024-04-26T03:39:16.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1589999999999998e-06 2024-04-26T03:39:16.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:16.971 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.517366+0000", 2024-04-26T03:39:16.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0049946269999999997 2024-04-26T03:39:16.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:16.971 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.517461+0000", 2024-04-26T03:39:16.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4529000000000002e-05 2024-04-26T03:39:16.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:16.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.521520+0000", 2024-04-26T03:39:16.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0040588519999999999 2024-04-26T03:39:16.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:16.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.521609+0000", 2024-04-26T03:39:16.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8967000000000004e-05 2024-04-26T03:39:16.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:16.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.527630+0000", 2024-04-26T03:39:16.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0060210350000000001 2024-04-26T03:39:16.973 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.973 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:16.973 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.973 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.973 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:72 3.es0 3:75fafc97:::smithi09235267-24:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:16.973 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.521677+0000", 2024-04-26T03:39:16.973 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.431307255, 2024-04-26T03:39:16.973 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012551157, 2024-04-26T03:39:16.973 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:16.973 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:16.973 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-04-26T03:39:16.973 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:16.973 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:16.973 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:16.974 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 72 2024-04-26T03:39:16.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.974 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:16.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.974 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:16.974 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.521677+0000", 2024-04-26T03:39:16.974 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.974 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:16.974 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.521677+0000", 2024-04-26T03:39:16.974 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.975 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:16.975 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.521677+0000", 2024-04-26T03:39:16.975 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.15e-07 2024-04-26T03:39:16.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.975 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:16.975 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.521679+0000", 2024-04-26T03:39:16.975 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9429999999999999e-06 2024-04-26T03:39:16.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.975 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:16.975 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.521684+0000", 2024-04-26T03:39:16.975 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3610000000000003e-06 2024-04-26T03:39:16.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.976 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:16.976 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.521688+0000", 2024-04-26T03:39:16.976 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.138e-06 2024-04-26T03:39:16.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.976 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:16.976 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528505+0000", 2024-04-26T03:39:16.976 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0068170679999999999 2024-04-26T03:39:16.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.976 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:16.976 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528576+0000", 2024-04-26T03:39:16.976 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1507000000000005e-05 2024-04-26T03:39:16.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.977 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:16.977 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.531827+0000", 2024-04-26T03:39:16.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003250883 2024-04-26T03:39:16.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.977 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:16.977 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.531903+0000", 2024-04-26T03:39:16.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5957999999999993e-05 2024-04-26T03:39:16.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.977 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:16.977 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.534228+0000", 2024-04-26T03:39:16.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023248840000000002 2024-04-26T03:39:16.977 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.978 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:16.978 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.978 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.978 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:78 3.9s0 3:9da60d2b:::smithi09235267-26:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:16.978 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.528454+0000", 2024-04-26T03:39:16.978 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.424530111999999, 2024-04-26T03:39:16.978 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011500992, 2024-04-26T03:39:16.978 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:16.978 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:16.978 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-04-26T03:39:16.978 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:16.978 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:16.978 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:16.978 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 78 2024-04-26T03:39:16.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.979 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:16.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:16.979 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528454+0000", 2024-04-26T03:39:16.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:16.979 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528454+0000", 2024-04-26T03:39:16.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:16.979 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528455+0000", 2024-04-26T03:39:16.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9699999999999995e-07 2024-04-26T03:39:16.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.980 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:16.980 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528459+0000", 2024-04-26T03:39:16.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1620000000000001e-06 2024-04-26T03:39:16.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.980 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:16.980 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528468+0000", 2024-04-26T03:39:16.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7010000000000002e-06 2024-04-26T03:39:16.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.980 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:16.980 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528474+0000", 2024-04-26T03:39:16.981 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7039999999999997e-06 2024-04-26T03:39:16.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.981 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:16.981 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.536776+0000", 2024-04-26T03:39:16.981 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0083020709999999994 2024-04-26T03:39:16.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.981 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:16.981 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.536858+0000", 2024-04-26T03:39:16.981 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1130000000000004e-05 2024-04-26T03:39:16.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.981 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:16.982 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.538545+0000", 2024-04-26T03:39:16.982 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016869610000000001 2024-04-26T03:39:16.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.982 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.982 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:16.982 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.538629+0000", 2024-04-26T03:39:16.982 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4257000000000003e-05 2024-04-26T03:39:16.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.982 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.982 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:16.982 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.539955+0000", 2024-04-26T03:39:16.982 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013263089999999999 2024-04-26T03:39:16.982 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.982 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:16.982 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.983 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.983 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:87 3.es0 3:799cf17d:::smithi09235267-29:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:16.983 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.540988+0000", 2024-04-26T03:39:16.983 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.411995784999998, 2024-04-26T03:39:16.983 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010406014, 2024-04-26T03:39:16.983 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:16.983 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:16.983 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-04-26T03:39:16.983 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:16.983 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:16.983 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:16.983 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 87 2024-04-26T03:39:16.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.984 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:16.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.984 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:16.984 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.540988+0000", 2024-04-26T03:39:16.984 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.984 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:16.984 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.540988+0000", 2024-04-26T03:39:16.984 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.985 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:16.985 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.540989+0000", 2024-04-26T03:39:16.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4400000000000001e-07 2024-04-26T03:39:16.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.985 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:16.985 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.540991+0000", 2024-04-26T03:39:16.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5299999999999999e-06 2024-04-26T03:39:16.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.985 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:16.985 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.540998+0000", 2024-04-26T03:39:16.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0190000000000001e-06 2024-04-26T03:39:16.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.986 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:16.986 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.541005+0000", 2024-04-26T03:39:16.986 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2149999999999998e-06 2024-04-26T03:39:16.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.986 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:16.986 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.546779+0000", 2024-04-26T03:39:16.986 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0057747659999999998 2024-04-26T03:39:16.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.986 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:16.986 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.546880+0000", 2024-04-26T03:39:16.986 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100657 2024-04-26T03:39:16.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.987 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:16.987 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.550192+0000", 2024-04-26T03:39:16.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033114310000000001 2024-04-26T03:39:16.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.987 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:16.987 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.550270+0000", 2024-04-26T03:39:16.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.805e-05 2024-04-26T03:39:16.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.987 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:16.987 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.551394+0000", 2024-04-26T03:39:16.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011249020000000001 2024-04-26T03:39:16.988 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.988 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:16.988 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.988 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.988 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:99 3.9s0 3:91c00773:::smithi09235267-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:16.988 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.555207+0000", 2024-04-26T03:39:16.988 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.397777569999999, 2024-04-26T03:39:16.988 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010397744, 2024-04-26T03:39:16.988 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:16.988 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:16.988 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-04-26T03:39:16.988 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:16.988 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:16.989 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:16.989 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 99 2024-04-26T03:39:16.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.989 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:16.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.989 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:16.989 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.555207+0000", 2024-04-26T03:39:16.989 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.989 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:16.989 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.555207+0000", 2024-04-26T03:39:16.989 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.990 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:16.990 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.555207+0000", 2024-04-26T03:39:16.990 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0800000000000005e-07 2024-04-26T03:39:16.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.990 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.990 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:16.990 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.555209+0000", 2024-04-26T03:39:16.990 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3020000000000002e-06 2024-04-26T03:39:16.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.990 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.990 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:16.990 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.555216+0000", 2024-04-26T03:39:16.990 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2349999999999996e-06 2024-04-26T03:39:16.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.991 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.991 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:16.991 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.555221+0000", 2024-04-26T03:39:16.991 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1399999999999999e-06 2024-04-26T03:39:16.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.991 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.991 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:16.991 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.560263+0000", 2024-04-26T03:39:16.991 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0050417839999999997 2024-04-26T03:39:16.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.991 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.991 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:16.991 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.560356+0000", 2024-04-26T03:39:16.991 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2987999999999998e-05 2024-04-26T03:39:16.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.992 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:16.992 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.563913+0000", 2024-04-26T03:39:16.992 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035570010000000002 2024-04-26T03:39:16.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.992 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:16.992 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.563988+0000", 2024-04-26T03:39:16.992 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5463000000000006e-05 2024-04-26T03:39:16.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.992 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:16.992 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.565604+0000", 2024-04-26T03:39:16.992 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001616323 2024-04-26T03:39:16.992 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.993 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:16.993 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.993 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.993 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:207 3.es0 3:718e77da:::smithi09235267-42:head [append 0~638976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:16.993 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.806150+0000", 2024-04-26T03:39:16.993 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.146833786999998, 2024-04-26T03:39:16.993 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010521529, 2024-04-26T03:39:16.993 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:16.993 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:16.993 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:16.993 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:16.993 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:16.993 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:16.994 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 207 2024-04-26T03:39:16.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.994 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:16.994 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.994 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:16.994 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.806150+0000", 2024-04-26T03:39:16.994 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.994 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.994 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:16.994 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.806150+0000", 2024-04-26T03:39:16.994 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:16.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.995 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:16.995 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.806152+0000", 2024-04-26T03:39:16.995 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.274e-06 2024-04-26T03:39:16.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.995 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:16.995 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.806688+0000", 2024-04-26T03:39:16.995 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00053636700000000005 2024-04-26T03:39:16.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.995 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:16.995 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.806700+0000", 2024-04-26T03:39:16.995 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1969e-05 2024-04-26T03:39:16.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.996 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:16.996 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.806705+0000", 2024-04-26T03:39:16.996 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2780000000000001e-06 2024-04-26T03:39:16.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.996 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:16.996 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.806771+0000", 2024-04-26T03:39:16.996 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5748999999999997e-05 2024-04-26T03:39:16.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.996 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:16.996 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.806837+0000", 2024-04-26T03:39:16.996 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5445999999999996e-05 2024-04-26T03:39:16.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.997 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.997 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:16.997 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.807429+0000", 2024-04-26T03:39:16.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00059259600000000003 2024-04-26T03:39:16.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.997 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.997 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:16.997 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.814358+0000", 2024-04-26T03:39:16.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0069287949999999997 2024-04-26T03:39:16.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.997 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.997 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:39:16.997 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.816619+0000", 2024-04-26T03:39:16.998 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022611710000000002 2024-04-26T03:39:16.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.998 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.998 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:16.998 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.816672+0000", 2024-04-26T03:39:16.998 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2883999999999997e-05 2024-04-26T03:39:16.998 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.998 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:16.998 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:16.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.998 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.998 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:210 3.es0 3:718e77da:::smithi09235267-42:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:16.998 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.808586+0000", 2024-04-26T03:39:16.999 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.144398441, 2024-04-26T03:39:16.999 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010859396, 2024-04-26T03:39:16.999 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:16.999 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:16.999 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-04-26T03:39:16.999 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:16.999 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:16.999 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:16.999 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 210 2024-04-26T03:39:16.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:16.999 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:16.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:16.999 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:16.999 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.808586+0000", 2024-04-26T03:39:17.000 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:17.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.000 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.000 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:17.000 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.808586+0000", 2024-04-26T03:39:17.000 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:17.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.000 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.000 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:17.000 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.808586+0000", 2024-04-26T03:39:17.000 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7000000000000001e-07 2024-04-26T03:39:17.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.000 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.001 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:17.001 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.808587+0000", 2024-04-26T03:39:17.001 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.251e-06 2024-04-26T03:39:17.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.001 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:17.001 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.808591+0000", 2024-04-26T03:39:17.001 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6849999999999999e-06 2024-04-26T03:39:17.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.001 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:17.001 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.808595+0000", 2024-04-26T03:39:17.001 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6430000000000001e-06 2024-04-26T03:39:17.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.002 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.002 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:17.002 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.814250+0000", 2024-04-26T03:39:17.002 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.005655116 2024-04-26T03:39:17.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.002 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.002 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:17.002 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.814313+0000", 2024-04-26T03:39:17.002 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3075000000000002e-05 2024-04-26T03:39:17.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.002 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.002 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:17.002 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.817583+0000", 2024-04-26T03:39:17.003 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0032703300000000001 2024-04-26T03:39:17.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.003 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.003 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:17.003 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.817668+0000", 2024-04-26T03:39:17.003 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5222000000000005e-05 2024-04-26T03:39:17.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.003 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.003 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:17.003 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.819445+0000", 2024-04-26T03:39:17.003 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0017768040000000001 2024-04-26T03:39:17.003 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:17.003 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:17.003 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:17.004 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.004 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.004 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:301 3.es0 3:75fafc97:::smithi09235267-24:head [append 0~442368] snapc 6=[6,5] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-26T03:39:17.004 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:59.527343+0000", 2024-04-26T03:39:17.004 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.425640816000001, 2024-04-26T03:39:17.004 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018417728000000001, 2024-04-26T03:39:17.004 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:17.004 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:17.004 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:17.004 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:17.004 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:17.004 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:17.004 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 301 2024-04-26T03:39:17.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.005 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:17.005 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.005 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:17.005 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.527343+0000", 2024-04-26T03:39:17.005 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:17.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.005 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.005 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:17.005 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.527343+0000", 2024-04-26T03:39:17.005 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:17.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.005 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.005 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:17.006 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.527346+0000", 2024-04-26T03:39:17.006 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3420000000000002e-06 2024-04-26T03:39:17.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.006 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.006 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:17.006 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.527805+0000", 2024-04-26T03:39:17.006 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00045928899999999999 2024-04-26T03:39:17.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.006 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.006 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:17.006 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.527825+0000", 2024-04-26T03:39:17.006 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9825000000000001e-05 2024-04-26T03:39:17.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.006 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.007 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:17.007 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.527838+0000", 2024-04-26T03:39:17.007 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.31e-05 2024-04-26T03:39:17.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.007 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.007 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:17.007 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.527930+0000", 2024-04-26T03:39:17.007 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2060999999999998e-05 2024-04-26T03:39:17.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.007 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.007 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-04-26T03:39:17.007 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.527937+0000", 2024-04-26T03:39:17.007 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3150000000000003e-06 2024-04-26T03:39:17.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.007 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.007 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:17.008 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.530067+0000", 2024-04-26T03:39:17.008 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0021300989999999999 2024-04-26T03:39:17.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.008 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.008 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:17.008 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.530147+0000", 2024-04-26T03:39:17.008 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9178999999999995e-05 2024-04-26T03:39:17.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.008 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.008 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:17.008 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.531062+0000", 2024-04-26T03:39:17.008 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00091551399999999998 2024-04-26T03:39:17.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.008 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.008 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:17.009 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.542352+0000", 2024-04-26T03:39:17.009 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011290117000000001 2024-04-26T03:39:17.009 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.009 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.009 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:39:17.009 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.545702+0000", 2024-04-26T03:39:17.009 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033499290000000002 2024-04-26T03:39:17.009 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.009 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.009 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:17.009 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.545761+0000", 2024-04-26T03:39:17.009 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8958e-05 2024-04-26T03:39:17.009 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:17.009 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:17.009 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:17.009 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.010 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.010 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:302 3.es0 3:75fafc97:::smithi09235267-24:head [append 0~720896] snapc 6=[6,5] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-26T03:39:17.010 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:59.531209+0000", 2024-04-26T03:39:17.010 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.421774783, 2024-04-26T03:39:17.010 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015580979, 2024-04-26T03:39:17.010 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:17.010 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:17.010 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:17.010 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:17.010 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:17.010 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:17.010 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 302 2024-04-26T03:39:17.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.010 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:17.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.011 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:17.011 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.531209+0000", 2024-04-26T03:39:17.011 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:17.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.011 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:17.011 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.531209+0000", 2024-04-26T03:39:17.011 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:17.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.011 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:17.011 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.531211+0000", 2024-04-26T03:39:17.011 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.37e-06 2024-04-26T03:39:17.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.012 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:17.012 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.531886+0000", 2024-04-26T03:39:17.012 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00067507799999999996 2024-04-26T03:39:17.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.012 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.012 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:17.012 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.531901+0000", 2024-04-26T03:39:17.012 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5335000000000001e-05 2024-04-26T03:39:17.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.012 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.012 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:17.012 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.531911+0000", 2024-04-26T03:39:17.012 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6569999999999992e-06 2024-04-26T03:39:17.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.012 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.012 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:17.013 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.535914+0000", 2024-04-26T03:39:17.013 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0040035410000000002 2024-04-26T03:39:17.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.013 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.013 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:17.013 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.536006+0000", 2024-04-26T03:39:17.013 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1390000000000004e-05 2024-04-26T03:39:17.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.013 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.013 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:17.013 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.536984+0000", 2024-04-26T03:39:17.013 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0009780609999999999 2024-04-26T03:39:17.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.013 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.013 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:17.014 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.544537+0000", 2024-04-26T03:39:17.014 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.007553087 2024-04-26T03:39:17.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.014 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.014 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:39:17.014 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.546737+0000", 2024-04-26T03:39:17.014 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002199613 2024-04-26T03:39:17.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.014 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.014 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:17.014 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.546790+0000", 2024-04-26T03:39:17.014 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3847000000000001e-05 2024-04-26T03:39:17.014 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:17.015 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:17.015 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:17.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.015 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:303 3.es0 3:75fafc97:::smithi09235267-24:head [append 0~49152] snapc 6=[6,5] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-26T03:39:17.015 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:59.531927+0000", 2024-04-26T03:39:17.015 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.421057241, 2024-04-26T03:39:17.015 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017418454999999999, 2024-04-26T03:39:17.015 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:17.015 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:17.015 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:17.015 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:17.015 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:17.015 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:17.016 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 303 2024-04-26T03:39:17.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.016 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:17.016 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.016 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:17.016 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.531927+0000", 2024-04-26T03:39:17.016 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:17.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.016 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.016 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:17.016 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.531927+0000", 2024-04-26T03:39:17.016 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:17.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.016 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.016 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:17.017 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.531928+0000", 2024-04-26T03:39:17.017 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7699999999999992e-07 2024-04-26T03:39:17.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.017 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.017 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:17.017 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.531984+0000", 2024-04-26T03:39:17.017 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5787e-05 2024-04-26T03:39:17.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.017 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.017 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:17.017 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.531993+0000", 2024-04-26T03:39:17.017 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5349999999999993e-06 2024-04-26T03:39:17.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.017 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.017 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:17.018 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.531998+0000", 2024-04-26T03:39:17.018 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8040000000000001e-06 2024-04-26T03:39:17.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.018 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:17.018 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.542497+0000", 2024-04-26T03:39:17.018 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010499163000000001 2024-04-26T03:39:17.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.018 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:17.018 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.542593+0000", 2024-04-26T03:39:17.018 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6026999999999994e-05 2024-04-26T03:39:17.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:17.019 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.543050+0000", 2024-04-26T03:39:17.019 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00045626999999999999 2024-04-26T03:39:17.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:17.019 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.544538+0000", 2024-04-26T03:39:17.019 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014883299999999999 2024-04-26T03:39:17.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:39:17.019 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.549296+0000", 2024-04-26T03:39:17.019 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0047584369999999999 2024-04-26T03:39:17.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:17.020 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.549345+0000", 2024-04-26T03:39:17.020 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9125000000000001e-05 2024-04-26T03:39:17.020 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:17.020 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:17.020 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:17.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.020 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.020 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:304 3.es0 3:75fafc97:::smithi09235267-24:head [setxattr _header (56) in=7b] snapc 6=[6,5] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-26T03:39:17.020 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:59.532007+0000", 2024-04-26T03:39:17.020 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.420977192999999, 2024-04-26T03:39:17.020 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017496230000000002, 2024-04-26T03:39:17.020 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:17.020 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:17.020 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:17.021 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:17.021 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:17.021 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:17.021 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 304 2024-04-26T03:39:17.021 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.021 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:17.021 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.021 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:17.021 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.532007+0000", 2024-04-26T03:39:17.021 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:17.021 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.021 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.021 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:17.021 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.532007+0000", 2024-04-26T03:39:17.022 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:17.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.022 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.022 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:17.022 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.532008+0000", 2024-04-26T03:39:17.022 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1600000000000001e-07 2024-04-26T03:39:17.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.022 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.022 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:17.022 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.532011+0000", 2024-04-26T03:39:17.022 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5039999999999998e-06 2024-04-26T03:39:17.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.022 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.022 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:17.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.532015+0000", 2024-04-26T03:39:17.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0520000000000003e-06 2024-04-26T03:39:17.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:17.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.532029+0000", 2024-04-26T03:39:17.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4353e-05 2024-04-26T03:39:17.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:17.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.543638+0000", 2024-04-26T03:39:17.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011608095000000001 2024-04-26T03:39:17.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.024 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:17.024 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.543725+0000", 2024-04-26T03:39:17.024 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7379999999999993e-05 2024-04-26T03:39:17.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.024 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:17.024 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.544159+0000", 2024-04-26T03:39:17.024 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000433602 2024-04-26T03:39:17.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.024 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:17.024 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.544744+0000", 2024-04-26T03:39:17.024 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00058520399999999998 2024-04-26T03:39:17.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.025 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.025 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:39:17.025 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.549431+0000", 2024-04-26T03:39:17.025 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0046875110000000001 2024-04-26T03:39:17.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.025 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.025 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:17.025 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.549503+0000", 2024-04-26T03:39:17.025 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2013000000000003e-05 2024-04-26T03:39:17.025 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:17.025 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:17.025 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:17.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.025 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.026 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:305 3.es0 3:75fafc97:::smithi09235267-24:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e43)", 2024-04-26T03:39:17.026 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:59.532037+0000", 2024-04-26T03:39:17.026 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.420947282, 2024-04-26T03:39:17.026 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019157461000000001, 2024-04-26T03:39:17.026 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:17.026 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:17.026 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-04-26T03:39:17.026 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:17.026 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:17.026 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:17.026 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 305 2024-04-26T03:39:17.026 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.026 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:17.026 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.027 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:17.027 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.532037+0000", 2024-04-26T03:39:17.027 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:17.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.027 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:17.027 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.532037+0000", 2024-04-26T03:39:17.027 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:17.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.027 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:17.027 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.532037+0000", 2024-04-26T03:39:17.027 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0900000000000002e-07 2024-04-26T03:39:17.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.028 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:17.028 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.532053+0000", 2024-04-26T03:39:17.028 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5586999999999998e-05 2024-04-26T03:39:17.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.028 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:17.028 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.532060+0000", 2024-04-26T03:39:17.028 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3610000000000003e-06 2024-04-26T03:39:17.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.028 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:17.028 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.532065+0000", 2024-04-26T03:39:17.028 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4989999999999996e-06 2024-04-26T03:39:17.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:17.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.544785+0000", 2024-04-26T03:39:17.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012719988 2024-04-26T03:39:17.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:17.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.544890+0000", 2024-04-26T03:39:17.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010498 2024-04-26T03:39:17.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:17.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.549851+0000", 2024-04-26T03:39:17.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0049613050000000001 2024-04-26T03:39:17.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.030 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.030 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:17.030 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.549944+0000", 2024-04-26T03:39:17.030 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2950999999999998e-05 2024-04-26T03:39:17.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.030 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.030 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:17.030 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.551194+0000", 2024-04-26T03:39:17.030 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012503810000000001 2024-04-26T03:39:17.030 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:17.030 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:17.030 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:17.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.030 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.030 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:349 3.es0 3:75fafc97:::smithi09235267-24:head [append 0~557056] snapc 7=[7,6,5] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-26T03:39:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:01.631025+0000", 2024-04-26T03:39:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.321958906000001, 2024-04-26T03:39:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012493898, 2024-04-26T03:39:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 349 2024-04-26T03:39:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:17.032 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.631025+0000", 2024-04-26T03:39:17.032 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:17.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.032 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.032 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:17.032 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.631025+0000", 2024-04-26T03:39:17.032 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:17.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.032 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.032 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:17.032 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.631027+0000", 2024-04-26T03:39:17.032 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2359999999999999e-06 2024-04-26T03:39:17.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.032 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.033 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:17.033 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.631581+0000", 2024-04-26T03:39:17.033 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00055440199999999995 2024-04-26T03:39:17.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.033 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.033 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:17.033 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.631594+0000", 2024-04-26T03:39:17.033 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2978e-05 2024-04-26T03:39:17.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.033 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.033 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:17.033 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.631602+0000", 2024-04-26T03:39:17.033 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9340000000000006e-06 2024-04-26T03:39:17.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.034 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:17.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.631664+0000", 2024-04-26T03:39:17.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2119999999999995e-05 2024-04-26T03:39:17.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.034 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:17.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.631781+0000", 2024-04-26T03:39:17.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011652 2024-04-26T03:39:17.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.034 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:17.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.632790+0000", 2024-04-26T03:39:17.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010097019999999999 2024-04-26T03:39:17.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.035 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:17.035 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.643222+0000", 2024-04-26T03:39:17.035 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010431753 2024-04-26T03:39:17.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.035 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:39:17.035 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.643459+0000", 2024-04-26T03:39:17.035 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00023718900000000001 2024-04-26T03:39:17.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.035 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:17.035 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.643519+0000", 2024-04-26T03:39:17.035 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0064e-05 2024-04-26T03:39:17.036 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:17.036 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:17.036 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:17.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.036 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:350 3.es0 3:75fafc97:::smithi09235267-24:head [append 0~671744] snapc 7=[7,6,5] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-26T03:39:17.036 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:01.634732+0000", 2024-04-26T03:39:17.036 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.318252294000001, 2024-04-26T03:39:17.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012281976, 2024-04-26T03:39:17.036 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:17.036 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:17.036 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:17.036 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:17.036 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:17.037 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:17.037 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 350 2024-04-26T03:39:17.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.037 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:17.037 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.037 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:17.037 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.634732+0000", 2024-04-26T03:39:17.037 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:17.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.037 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.037 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:17.037 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.634732+0000", 2024-04-26T03:39:17.037 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:17.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.037 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.037 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:17.037 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.634734+0000", 2024-04-26T03:39:17.038 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.595e-06 2024-04-26T03:39:17.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.038 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:17.038 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.635461+0000", 2024-04-26T03:39:17.038 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00072744499999999996 2024-04-26T03:39:17.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.038 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:17.038 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.635478+0000", 2024-04-26T03:39:17.038 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7098000000000001e-05 2024-04-26T03:39:17.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.038 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:17.039 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.635487+0000", 2024-04-26T03:39:17.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7420000000000003e-06 2024-04-26T03:39:17.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:17.039 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.636529+0000", 2024-04-26T03:39:17.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010418910000000001 2024-04-26T03:39:17.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:17.039 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.636658+0000", 2024-04-26T03:39:17.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012908099999999999 2024-04-26T03:39:17.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:17.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.637438+0000", 2024-04-26T03:39:17.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000779953 2024-04-26T03:39:17.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:17.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.643223+0000", 2024-04-26T03:39:17.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0057856280000000001 2024-04-26T03:39:17.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:39:17.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.646969+0000", 2024-04-26T03:39:17.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003745211 2024-04-26T03:39:17.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:17.041 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.647014+0000", 2024-04-26T03:39:17.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5331999999999999e-05 2024-04-26T03:39:17.041 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:17.041 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:17.041 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:17.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.041 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.041 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:351 3.es0 3:75fafc97:::smithi09235267-24:head [append 0~98304] snapc 7=[7,6,5] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-26T03:39:17.041 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:01.635505+0000", 2024-04-26T03:39:17.041 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.317478942999999, 2024-04-26T03:39:17.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012808014, 2024-04-26T03:39:17.041 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:17.041 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:17.041 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:17.041 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:17.041 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:17.042 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:17.042 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 351 2024-04-26T03:39:17.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.042 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:17.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:17.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.635505+0000", 2024-04-26T03:39:17.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:17.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:17.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.635505+0000", 2024-04-26T03:39:17.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:17.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.043 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:17.043 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.635506+0000", 2024-04-26T03:39:17.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2099999999999995e-07 2024-04-26T03:39:17.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.043 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:17.043 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.635555+0000", 2024-04-26T03:39:17.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8658000000000001e-05 2024-04-26T03:39:17.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.043 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:17.043 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.635567+0000", 2024-04-26T03:39:17.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1809e-05 2024-04-26T03:39:17.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.044 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:17.044 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.635575+0000", 2024-04-26T03:39:17.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3319999999999992e-06 2024-04-26T03:39:17.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.044 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:17.044 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.641450+0000", 2024-04-26T03:39:17.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00587558 2024-04-26T03:39:17.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.044 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:17.044 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.641561+0000", 2024-04-26T03:39:17.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000110183 2024-04-26T03:39:17.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.045 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:17.045 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.642098+0000", 2024-04-26T03:39:17.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00053782199999999997 2024-04-26T03:39:17.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.045 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:17.045 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.643585+0000", 2024-04-26T03:39:17.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001486802 2024-04-26T03:39:17.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.045 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:39:17.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.648271+0000", 2024-04-26T03:39:17.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0046858309999999997 2024-04-26T03:39:17.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:17.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.648313+0000", 2024-04-26T03:39:17.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2175999999999997e-05 2024-04-26T03:39:17.046 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:17.046 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:17.046 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:17.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.046 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:352 3.es0 3:75fafc97:::smithi09235267-24:head [setxattr _header (56) in=7b] snapc 7=[7,6,5] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-26T03:39:17.046 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:01.635588+0000", 2024-04-26T03:39:17.047 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.3173963, 2024-04-26T03:39:17.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012886458, 2024-04-26T03:39:17.047 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:17.047 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:17.047 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:17.047 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:17.047 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:17.047 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:17.047 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 352 2024-04-26T03:39:17.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.047 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:17.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:17.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.635588+0000", 2024-04-26T03:39:17.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:17.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:17.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.635588+0000", 2024-04-26T03:39:17.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:17.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:17.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.635589+0000", 2024-04-26T03:39:17.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1699999999999998e-07 2024-04-26T03:39:17.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:17.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.635593+0000", 2024-04-26T03:39:17.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2769999999999999e-06 2024-04-26T03:39:17.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:17.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.635601+0000", 2024-04-26T03:39:17.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6249999999999996e-06 2024-04-26T03:39:17.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:17.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.635608+0000", 2024-04-26T03:39:17.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6900000000000003e-06 2024-04-26T03:39:17.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:17.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.643620+0000", 2024-04-26T03:39:17.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0080122619999999992 2024-04-26T03:39:17.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:17.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.643681+0000", 2024-04-26T03:39:17.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0593000000000001e-05 2024-04-26T03:39:17.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:17.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.643953+0000", 2024-04-26T03:39:17.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00027240400000000002 2024-04-26T03:39:17.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:17.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.644544+0000", 2024-04-26T03:39:17.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00059055000000000002 2024-04-26T03:39:17.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:39:17.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.648420+0000", 2024-04-26T03:39:17.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038758120000000002 2024-04-26T03:39:17.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:17.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.648474+0000", 2024-04-26T03:39:17.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4627999999999999e-05 2024-04-26T03:39:17.051 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:17.051 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:17.052 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:17.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.052 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:353 3.es0 3:75fafc97:::smithi09235267-24:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e45)", 2024-04-26T03:39:17.052 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:01.635616+0000", 2024-04-26T03:39:17.052 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.317368120999999, 2024-04-26T03:39:17.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013944994, 2024-04-26T03:39:17.052 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:17.052 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:17.052 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-04-26T03:39:17.052 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:17.052 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:17.052 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:17.053 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 353 2024-04-26T03:39:17.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.053 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:17.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:17.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.635616+0000", 2024-04-26T03:39:17.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:17.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:17.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.635616+0000", 2024-04-26T03:39:17.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:17.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:17.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.635617+0000", 2024-04-26T03:39:17.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9900000000000001e-07 2024-04-26T03:39:17.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:17.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.635620+0000", 2024-04-26T03:39:17.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.365e-06 2024-04-26T03:39:17.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:17.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.635628+0000", 2024-04-26T03:39:17.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3809999999999999e-06 2024-04-26T03:39:17.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:17.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.635635+0000", 2024-04-26T03:39:17.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.934e-06 2024-04-26T03:39:17.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:17.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.644429+0000", 2024-04-26T03:39:17.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.008794049 2024-04-26T03:39:17.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:17.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.644532+0000", 2024-04-26T03:39:17.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000102494 2024-04-26T03:39:17.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:17.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.648687+0000", 2024-04-26T03:39:17.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0041551770000000003 2024-04-26T03:39:17.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:17.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.648760+0000", 2024-04-26T03:39:17.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2794999999999997e-05 2024-04-26T03:39:17.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:17.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.649561+0000", 2024-04-26T03:39:17.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00080130000000000002 2024-04-26T03:39:17.056 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:17.056 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:17.056 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:17.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.057 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:437 3.19s0 3:9d1726ae:::smithi09235267-16:head [delete] snapc 8=[8] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e49)", 2024-04-26T03:39:17.057 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:04.697979+0000", 2024-04-26T03:39:17.057 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 12.255005504, 2024-04-26T03:39:17.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012570316999999999, 2024-04-26T03:39:17.057 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:17.057 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:17.057 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:17.057 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:17.057 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:17.057 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:17.057 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 437 2024-04-26T03:39:17.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.057 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:17.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:17.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.697979+0000", 2024-04-26T03:39:17.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:17.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:17.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.697979+0000", 2024-04-26T03:39:17.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:17.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:17.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.697981+0000", 2024-04-26T03:39:17.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.306e-06 2024-04-26T03:39:17.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:17.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.697989+0000", 2024-04-26T03:39:17.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0830000000000004e-06 2024-04-26T03:39:17.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:17.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.698011+0000", 2024-04-26T03:39:17.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1920999999999999e-05 2024-04-26T03:39:17.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:17.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.698030+0000", 2024-04-26T03:39:17.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8589000000000001e-05 2024-04-26T03:39:17.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:17.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.698117+0000", 2024-04-26T03:39:17.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7769000000000006e-05 2024-04-26T03:39:17.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:17.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.698309+0000", 2024-04-26T03:39:17.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019131700000000001 2024-04-26T03:39:17.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:17.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.699096+0000", 2024-04-26T03:39:17.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00078738300000000001 2024-04-26T03:39:17.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:17.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.700150+0000", 2024-04-26T03:39:17.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010535340000000001 2024-04-26T03:39:17.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:39:17.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.710460+0000", 2024-04-26T03:39:17.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.01031017 2024-04-26T03:39:17.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:17.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:17.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:17.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.710549+0000", 2024-04-26T03:39:17.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9245000000000004e-05 2024-04-26T03:39:17.062 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:17.062 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:17.062 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:17.062 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:17.062 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:17.062 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-26T03:39:17.064 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-26T03:39:17.064 INFO:tasks.ceph.osd.14.smithi066.stderr:2024-04-26T03:39:17.062+0000 7f5395f08640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 31765) UID: 0 2024-04-26T03:39:17.091 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-26T03:39:17.164 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-04-26T03:39:17.163+0000 7fe9223c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31954) UID: 0 2024-04-26T03:39:17.184 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:39:17.184 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-04-26T03:39:17.184 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 0 2024-04-26T03:39:17.184 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:39:17.193 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-26T03:39:17.216 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0: {} 2024-04-26T03:39:17.216 INFO:teuthology.orchestra.run.smithi028.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T03:39:17.234 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1: {} 2024-04-26T03:39:17.235 INFO:teuthology.orchestra.run.smithi028.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T03:39:17.252 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2: {} 2024-04-26T03:39:17.253 INFO:teuthology.orchestra.run.smithi028.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T03:39:17.265 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:39:17.264+0000 7f60d3c4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31757) UID: 0 2024-04-26T03:39:17.271 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3: {} 2024-04-26T03:39:17.271 INFO:teuthology.orchestra.run.smithi028.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T03:39:17.288 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4: {} 2024-04-26T03:39:17.289 INFO:teuthology.orchestra.run.smithi028.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T03:39:17.301 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:17.301 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:17.306 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5: {} 2024-04-26T03:39:17.306 INFO:teuthology.orchestra.run.smithi028.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T03:39:17.310 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-26T03:39:17.323 INFO:teuthology.orchestra.run.smithi028.stdout:osd.6: {} 2024-04-26T03:39:17.324 INFO:teuthology.orchestra.run.smithi028.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T03:39:17.341 INFO:teuthology.orchestra.run.smithi028.stdout:osd.7: {} 2024-04-26T03:39:17.342 INFO:teuthology.orchestra.run.smithi028.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T03:39:17.358 INFO:teuthology.orchestra.run.smithi028.stdout:osd.8: {} 2024-04-26T03:39:17.358 INFO:teuthology.orchestra.run.smithi028.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T03:39:17.365 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:39:17.364+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:39:17.376 INFO:teuthology.orchestra.run.smithi028.stdout:osd.9: {} 2024-04-26T03:39:17.376 INFO:teuthology.orchestra.run.smithi028.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T03:39:17.395 INFO:teuthology.orchestra.run.smithi028.stdout:osd.10: {} 2024-04-26T03:39:17.395 INFO:teuthology.orchestra.run.smithi028.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T03:39:17.413 INFO:teuthology.orchestra.run.smithi028.stdout:osd.11: {} 2024-04-26T03:39:17.413 INFO:teuthology.orchestra.run.smithi028.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T03:39:17.430 INFO:teuthology.orchestra.run.smithi028.stdout:osd.12: {} 2024-04-26T03:39:17.430 INFO:teuthology.orchestra.run.smithi028.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T03:39:17.447 INFO:teuthology.orchestra.run.smithi028.stdout:osd.13: {} 2024-04-26T03:39:17.448 INFO:teuthology.orchestra.run.smithi028.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T03:39:17.458 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:17.458 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:17.465 INFO:teuthology.orchestra.run.smithi028.stdout:osd.14: {} 2024-04-26T03:39:17.465 INFO:teuthology.orchestra.run.smithi028.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T03:39:17.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:39:17.464+0000 7f8680feb640 0 -- 172.21.15.28:0/4186125780 >> [v2:172.21.15.92:6826/3982634323,v1:172.21.15.92:6827/3982634323] conn(0x7f867c222ac0 msgr2=0x7f867c224eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-26T03:39:17.466 INFO:tasks.ceph.osd.14.smithi066.stderr:2024-04-26T03:39:17.465+0000 7f5395f08640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 31765) UID: 0 2024-04-26T03:39:17.467 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-26T03:39:17.567 INFO:tasks.ceph.osd.8.smithi028.stderr:2024-04-26T03:39:17.565+0000 7f5bb75d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 31968) UID: 0 2024-04-26T03:39:17.617 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:17.617 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:17.625 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-26T03:39:17.669 INFO:tasks.ceph.osd.10.smithi066.stderr:2024-04-26T03:39:17.668+0000 7f130fcc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 31763) UID: 0 2024-04-26T03:39:17.684 INFO:teuthology.orchestra.run.smithi028.stdout:osd.15: {} 2024-04-26T03:39:17.685 INFO:teuthology.orchestra.run.smithi028.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T03:39:17.732 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:17.732 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:17.733 INFO:teuthology.orchestra.run.smithi028.stderr:set 3.f pg_upmap_items mapping to [2->9,4->13,7->1,15->10,3->11,5->14,8->0] 2024-04-26T03:39:17.733 INFO:teuthology.orchestra.run.smithi028.stderr:noscrub is unset 2024-04-26T03:39:17.742 INFO:tasks.rados.rados.0.smithi092.stdout:272: done (0 left) 2024-04-26T03:39:17.742 INFO:tasks.rados.rados.0.smithi092.stdout:273: snap_remove snap 13 2024-04-26T03:39:17.743 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-26T03:39:17.785 INFO:tasks.ceph.osd.14.smithi066.stderr:2024-04-26T03:39:17.784+0000 7f5395f08640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 31765) UID: 0 2024-04-26T03:39:17.870 INFO:tasks.ceph.osd.12.smithi028.stderr:2024-04-26T03:39:17.868+0000 7f114fe14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 31967) UID: 0 2024-04-26T03:39:17.891 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:17.891 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:17.900 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-04-26T03:39:17.972 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:39:17.970+0000 7fba0d2ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31961) UID: 0 2024-04-26T03:39:18.007 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:18.008 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:18.016 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-04-26T03:39:18.072 INFO:tasks.ceph.osd.10.smithi066.stderr:2024-04-26T03:39:18.071+0000 7f130fcc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 31763) UID: 0 2024-04-26T03:39:18.123 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:18.123 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:18.131 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-04-26T03:39:18.173 INFO:tasks.ceph.osd.9.smithi042.stderr:2024-04-26T03:39:18.172+0000 7fdac63b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 31960) UID: 0 2024-04-26T03:39:18.274 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:39:18.273+0000 7f60d3c4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31757) UID: 0 2024-04-26T03:39:18.278 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:18.278 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:18.288 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-04-26T03:39:18.375 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-26T03:39:18.374+0000 7f610b3a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31816) UID: 0 2024-04-26T03:39:18.437 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:18.437 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:18.446 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-04-26T03:39:18.476 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-04-26T03:39:18.475+0000 7f745118d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 31821) UID: 0 2024-04-26T03:39:18.551 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:18.552 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:18.560 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-04-26T03:39:18.577 INFO:tasks.ceph.osd.12.smithi028.stderr:2024-04-26T03:39:18.575+0000 7f114fe14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 31967) UID: 0 2024-04-26T03:39:18.667 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:18.667 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:18.676 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-26T03:39:18.679 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-04-26T03:39:18.677+0000 7fdc2d21c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31819) UID: 0 2024-04-26T03:39:18.744 INFO:tasks.rados.rados.0.smithi092.stdout:273: done (0 left) 2024-04-26T03:39:18.744 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 4 current snap is 16 2024-04-26T03:39:18.744 INFO:tasks.rados.rados.0.smithi092.stdout:274: seq_num 127 ranges {1720320=589824,2310144=753664,3063808=475136,3538944=327680} 2024-04-26T03:39:18.759 INFO:tasks.rados.rados.0.smithi092.stdout:274: writing smithi09235267-4 from 1720320 to 2310144 tid 1 2024-04-26T03:39:18.766 INFO:tasks.rados.rados.0.smithi092.stdout:274: writing smithi09235267-4 from 2310144 to 3063808 tid 2 2024-04-26T03:39:18.767 INFO:tasks.rados.rados.0.smithi092.stdout:274: writing smithi09235267-4 from 3063808 to 3538944 tid 3 2024-04-26T03:39:18.769 INFO:tasks.rados.rados.0.smithi092.stdout:274: writing smithi09235267-4 from 3538944 to 3866624 tid 4 2024-04-26T03:39:18.769 INFO:tasks.rados.rados.0.smithi092.stdout:275: setattr oid 48 current snap is 16 2024-04-26T03:39:18.771 INFO:tasks.rados.rados.0.smithi092.stdout:276: snap_remove snap 15 2024-04-26T03:39:18.779 INFO:tasks.ceph.osd.9.smithi042.stderr:2024-04-26T03:39:18.777+0000 7fdac63b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 31960) UID: 0 2024-04-26T03:39:18.803 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:18.803 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:18.812 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-26T03:39:18.880 INFO:tasks.ceph.osd.14.smithi066.stderr:2024-04-26T03:39:18.879+0000 7f5395f08640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 31765) UID: 0 2024-04-26T03:39:18.915 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:18.915 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:18.923 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-26T03:39:18.981 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-04-26T03:39:18.979+0000 7fdc2d21c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31819) UID: 0 2024-04-26T03:39:19.026 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:19.026 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:19.035 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-26T03:39:19.120 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-04-26T03:39:19.118+0000 7fdc2d21c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31819) UID: 0 2024-04-26T03:39:19.178 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:19.178 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:19.183 INFO:tasks.ceph.osd.14.smithi066.stderr:2024-04-26T03:39:19.182+0000 7f5395f08640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 31765) UID: 0 2024-04-26T03:39:19.187 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-26T03:39:19.284 INFO:tasks.ceph.osd.4.smithi028.stderr:2024-04-26T03:39:19.282+0000 7fd9511b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31963) UID: 0 2024-04-26T03:39:19.328 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:19.329 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:19.337 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-26T03:39:19.385 INFO:tasks.ceph.osd.8.smithi028.stderr:2024-04-26T03:39:19.383+0000 7f5bb75d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 31968) UID: 0 2024-04-26T03:39:19.439 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:19.440 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:19.448 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-04-26T03:39:19.486 INFO:tasks.ceph.osd.4.smithi028.stderr:2024-04-26T03:39:19.484+0000 7fd9511b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31963) UID: 0 2024-04-26T03:39:19.551 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:19.551 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:19.561 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-04-26T03:39:19.586 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-04-26T03:39:19.585+0000 7fee562fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31759) UID: 0 2024-04-26T03:39:19.663 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:19.663 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:19.672 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-04-26T03:39:19.687 INFO:tasks.rados.rados.0.smithi092.stdout:276: done (2 left) 2024-04-26T03:39:19.687 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 38 current snap is 16 2024-04-26T03:39:19.687 INFO:tasks.rados.rados.0.smithi092.stdout:277: seq_num 128 ranges {0=606208,606208=507904,1114112=573440,1687552=475136} 2024-04-26T03:39:19.688 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-26T03:39:19.686+0000 7f610b3a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31816) UID: 0 2024-04-26T03:39:19.691 INFO:tasks.rados.rados.0.smithi092.stdout:277: writing smithi09235267-38 from 0 to 606208 tid 1 2024-04-26T03:39:19.694 INFO:tasks.rados.rados.0.smithi092.stdout:277: writing smithi09235267-38 from 606208 to 1114112 tid 2 2024-04-26T03:39:19.697 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-26T03:39:19.698 INFO:tasks.rados.rados.0.smithi092.stdout:277: writing smithi09235267-38 from 1114112 to 1687552 tid 3 2024-04-26T03:39:19.702 INFO:tasks.rados.rados.0.smithi092.stdout:277: writing smithi09235267-38 from 1687552 to 2162688 tid 4 2024-04-26T03:39:19.702 INFO:tasks.rados.rados.0.smithi092.stdout:278: read oid 15 snap -1 2024-04-26T03:39:19.702 INFO:tasks.rados.rados.0.smithi092.stdout:278: expect deleted 2024-04-26T03:39:19.702 INFO:tasks.rados.rados.0.smithi092.stdout:279: delete oid 49 current snap is 16 2024-04-26T03:39:19.703 INFO:tasks.rados.rados.0.smithi092.stdout:274: finishing write tid 1 to smithi09235267-4 2024-04-26T03:39:19.703 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 48 v 34 (ObjNum 65 snap 0 seq_num 11) dirty exists 2024-04-26T03:39:19.703 INFO:tasks.rados.rados.0.smithi092.stdout:274: finishing write tid 2 to smithi09235267-4 2024-04-26T03:39:19.703 INFO:tasks.rados.rados.0.smithi092.stdout:274: finishing write tid 3 to smithi09235267-4 2024-04-26T03:39:19.703 INFO:tasks.rados.rados.0.smithi092.stdout:274: finishing write tid 4 to smithi09235267-4 2024-04-26T03:39:19.703 INFO:tasks.rados.rados.0.smithi092.stdout:274: finishing write tid 5 to smithi09235267-4 2024-04-26T03:39:19.703 INFO:tasks.rados.rados.0.smithi092.stdout:274: finishing write tid 6 to smithi09235267-4 2024-04-26T03:39:19.703 INFO:tasks.rados.rados.0.smithi092.stdout:274: oid 4 updating version 0 to 50 2024-04-26T03:39:19.703 INFO:tasks.rados.rados.0.smithi092.stdout:274: oid 4 version 50 is already newer than 48 2024-04-26T03:39:19.703 INFO:tasks.rados.rados.0.smithi092.stdout:274: oid 4 version 50 is already newer than 47 2024-04-26T03:39:19.703 INFO:tasks.rados.rados.0.smithi092.stdout:274: oid 4 version 50 is already newer than 46 2024-04-26T03:39:19.703 INFO:tasks.rados.rados.0.smithi092.stdout:274: oid 4 version 50 is already newer than 49 2024-04-26T03:39:19.703 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 4 v 50 (ObjNum 127 snap 16 seq_num 127) dirty exists 2024-04-26T03:39:19.704 INFO:tasks.rados.rados.0.smithi092.stdout:274: left oid 4 (ObjNum 127 snap 16 seq_num 127) 2024-04-26T03:39:19.705 INFO:tasks.rados.rados.0.smithi092.stdout:274: done (4 left) 2024-04-26T03:39:19.705 INFO:tasks.rados.rados.0.smithi092.stdout:275: done (3 left) 2024-04-26T03:39:19.706 INFO:tasks.rados.rados.0.smithi092.stdout:278: done (2 left) 2024-04-26T03:39:19.706 INFO:tasks.rados.rados.0.smithi092.stdout:279: done (1 left) 2024-04-26T03:39:19.706 INFO:tasks.rados.rados.0.smithi092.stdout:280: delete oid 43 current snap is 16 2024-04-26T03:39:19.709 INFO:tasks.rados.rados.0.smithi092.stdout:280: done (1 left) 2024-04-26T03:39:19.709 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 25 current snap is 16 2024-04-26T03:39:19.710 INFO:tasks.rados.rados.0.smithi092.stdout:281: seq_num 131 ranges {819200=655360,1474560=425984,1900544=638976,2539520=491520} 2024-04-26T03:39:19.718 INFO:tasks.rados.rados.0.smithi092.stdout:281: writing smithi09235267-25 from 819200 to 1474560 tid 1 2024-04-26T03:39:19.721 INFO:tasks.rados.rados.0.smithi092.stdout:281: writing smithi09235267-25 from 1474560 to 1900544 tid 2 2024-04-26T03:39:19.725 INFO:tasks.rados.rados.0.smithi092.stdout:281: writing smithi09235267-25 from 1900544 to 2539520 tid 3 2024-04-26T03:39:19.728 INFO:tasks.rados.rados.0.smithi092.stdout:281: writing smithi09235267-25 from 2539520 to 3031040 tid 4 2024-04-26T03:39:19.728 INFO:tasks.rados.rados.0.smithi092.stdout:282: snap_create 2024-04-26T03:39:19.728 INFO:tasks.rados.rados.0.smithi092.stdout:277: finishing write tid 1 to smithi09235267-38 2024-04-26T03:39:19.729 INFO:tasks.rados.rados.0.smithi092.stdout:277: finishing write tid 2 to smithi09235267-38 2024-04-26T03:39:19.729 INFO:tasks.rados.rados.0.smithi092.stdout:277: finishing write tid 3 to smithi09235267-38 2024-04-26T03:39:19.729 INFO:tasks.rados.rados.0.smithi092.stdout:277: finishing write tid 4 to smithi09235267-38 2024-04-26T03:39:19.729 INFO:tasks.rados.rados.0.smithi092.stdout:277: finishing write tid 5 to smithi09235267-38 2024-04-26T03:39:19.729 INFO:tasks.rados.rados.0.smithi092.stdout:277: finishing write tid 6 to smithi09235267-38 2024-04-26T03:39:19.729 INFO:tasks.rados.rados.0.smithi092.stdout:277: oid 38 updating version 0 to 54 2024-04-26T03:39:19.729 INFO:tasks.rados.rados.0.smithi092.stdout:277: oid 38 updating version 54 to 57 2024-04-26T03:39:19.729 INFO:tasks.rados.rados.0.smithi092.stdout:277: oid 38 version 57 is already newer than 55 2024-04-26T03:39:19.729 INFO:tasks.rados.rados.0.smithi092.stdout:277: oid 38 version 57 is already newer than 53 2024-04-26T03:39:19.729 INFO:tasks.rados.rados.0.smithi092.stdout:277: oid 38 version 57 is already newer than 56 2024-04-26T03:39:19.729 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 38 v 57 (ObjNum 128 snap 16 seq_num 128) dirty exists 2024-04-26T03:39:19.729 INFO:tasks.rados.rados.0.smithi092.stdout:277: left oid 38 (ObjNum 128 snap 16 seq_num 128) 2024-04-26T03:39:19.732 INFO:tasks.rados.rados.0.smithi092.stdout:281: finishing write tid 1 to smithi09235267-25 2024-04-26T03:39:19.744 INFO:tasks.rados.rados.0.smithi092.stdout:281: finishing write tid 2 to smithi09235267-25 2024-04-26T03:39:19.744 INFO:tasks.rados.rados.0.smithi092.stdout:281: finishing write tid 3 to smithi09235267-25 2024-04-26T03:39:19.746 INFO:tasks.rados.rados.0.smithi092.stdout:281: finishing write tid 4 to smithi09235267-25 2024-04-26T03:39:19.746 INFO:tasks.rados.rados.0.smithi092.stdout:281: finishing write tid 5 to smithi09235267-25 2024-04-26T03:39:19.747 INFO:tasks.rados.rados.0.smithi092.stdout:281: finishing write tid 6 to smithi09235267-25 2024-04-26T03:39:19.747 INFO:tasks.rados.rados.0.smithi092.stdout:281: oid 25 updating version 0 to 68 2024-04-26T03:39:19.748 INFO:tasks.rados.rados.0.smithi092.stdout:281: oid 25 version 68 is already newer than 64 2024-04-26T03:39:19.748 INFO:tasks.rados.rados.0.smithi092.stdout:281: oid 25 version 68 is already newer than 65 2024-04-26T03:39:19.748 INFO:tasks.rados.rados.0.smithi092.stdout:281: oid 25 version 68 is already newer than 67 2024-04-26T03:39:19.748 INFO:tasks.rados.rados.0.smithi092.stdout:281: oid 25 version 68 is already newer than 66 2024-04-26T03:39:19.748 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 25 v 68 (ObjNum 131 snap 16 seq_num 131) dirty exists 2024-04-26T03:39:19.748 INFO:tasks.rados.rados.0.smithi092.stdout:281: left oid 25 (ObjNum 131 snap 16 seq_num 131) 2024-04-26T03:39:19.750 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-26T03:39:19.790 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:19.790 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:19.794 INFO:tasks.ceph.osd.4.smithi028.stderr:2024-04-26T03:39:19.792+0000 7fd9511b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31963) UID: 0 2024-04-26T03:39:19.798 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-04-26T03:39:19.834 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0: {} 2024-04-26T03:39:19.835 INFO:teuthology.orchestra.run.smithi028.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T03:39:19.853 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1: {} 2024-04-26T03:39:19.853 INFO:teuthology.orchestra.run.smithi028.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T03:39:19.872 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2: {} 2024-04-26T03:39:19.872 INFO:teuthology.orchestra.run.smithi028.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T03:39:19.888 INFO:tasks.ceph.osd.12.smithi028.stderr:2024-04-26T03:39:19.887+0000 7f114fe14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 31967) UID: 0 2024-04-26T03:39:19.890 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3: {} 2024-04-26T03:39:19.890 INFO:teuthology.orchestra.run.smithi028.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T03:39:19.907 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4: {} 2024-04-26T03:39:19.907 INFO:teuthology.orchestra.run.smithi028.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T03:39:19.925 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5: {} 2024-04-26T03:39:19.925 INFO:teuthology.orchestra.run.smithi028.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T03:39:19.941 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:19.941 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:19.943 INFO:teuthology.orchestra.run.smithi028.stdout:osd.6: {} 2024-04-26T03:39:19.943 INFO:teuthology.orchestra.run.smithi028.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T03:39:19.950 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-04-26T03:39:19.960 INFO:teuthology.orchestra.run.smithi028.stdout:osd.7: {} 2024-04-26T03:39:19.960 INFO:teuthology.orchestra.run.smithi028.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T03:39:19.978 INFO:teuthology.orchestra.run.smithi028.stdout:osd.8: {} 2024-04-26T03:39:19.978 INFO:teuthology.orchestra.run.smithi028.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T03:39:19.989 INFO:tasks.ceph.osd.8.smithi028.stderr:2024-04-26T03:39:19.987+0000 7f5bb75d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 31968) UID: 0 2024-04-26T03:39:19.996 INFO:teuthology.orchestra.run.smithi028.stdout:osd.9: {} 2024-04-26T03:39:19.996 INFO:teuthology.orchestra.run.smithi028.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T03:39:20.014 INFO:teuthology.orchestra.run.smithi028.stdout:osd.10: {} 2024-04-26T03:39:20.015 INFO:teuthology.orchestra.run.smithi028.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T03:39:20.033 INFO:teuthology.orchestra.run.smithi028.stdout:osd.11: {} 2024-04-26T03:39:20.033 INFO:teuthology.orchestra.run.smithi028.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T03:39:20.050 INFO:teuthology.orchestra.run.smithi028.stdout:osd.12: {} 2024-04-26T03:39:20.050 INFO:teuthology.orchestra.run.smithi028.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T03:39:20.052 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:20.052 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:20.061 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-04-26T03:39:20.068 INFO:teuthology.orchestra.run.smithi028.stdout:osd.13: {} 2024-04-26T03:39:20.068 INFO:teuthology.orchestra.run.smithi028.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T03:39:20.086 INFO:teuthology.orchestra.run.smithi028.stdout:osd.14: {} 2024-04-26T03:39:20.086 INFO:teuthology.orchestra.run.smithi028.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T03:39:20.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:39:20.084+0000 7f1f76d9d640 0 -- 172.21.15.28:0/4169066942 >> [v2:172.21.15.92:6826/3982634323,v1:172.21.15.92:6827/3982634323] conn(0x7f1f78222a60 msgr2=0x7f1f78224e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-26T03:39:20.091 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:39:20.090+0000 7f60d3c4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31757) UID: 0 2024-04-26T03:39:20.163 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:20.163 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:20.171 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-26T03:39:20.192 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:39:20.191+0000 7f60d3c4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31757) UID: 0 2024-04-26T03:39:20.279 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:39:20.279 INFO:teuthology.orchestra.run.smithi028.stdout: "ops": [], 2024-04-26T03:39:20.279 INFO:teuthology.orchestra.run.smithi028.stdout: "num_ops": 0 2024-04-26T03:39:20.279 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:39:20.287 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-26T03:39:20.293 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-04-26T03:39:20.292+0000 7fee562fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31759) UID: 0 2024-04-26T03:39:20.305 INFO:teuthology.orchestra.run.smithi028.stdout:osd.15: {} 2024-04-26T03:39:20.305 INFO:teuthology.orchestra.run.smithi028.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T03:39:20.394 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:39:20.393+0000 7fb78c544640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31956) UID: 0 2024-04-26T03:39:20.395 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:39:20.396 INFO:teuthology.orchestra.run.smithi028.stdout: "ops": [], 2024-04-26T03:39:20.396 INFO:teuthology.orchestra.run.smithi028.stdout: "complaint_time": 30, 2024-04-26T03:39:20.396 INFO:teuthology.orchestra.run.smithi028.stdout: "num_blocked_ops": 0 2024-04-26T03:39:20.396 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:39:20.404 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-26T03:39:20.494 INFO:tasks.ceph.osd.9.smithi042.stderr:2024-04-26T03:39:20.493+0000 7fdac63b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 31960) UID: 0 2024-04-26T03:39:20.513 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:39:20.513 INFO:teuthology.orchestra.run.smithi028.stdout: "size": 20, 2024-04-26T03:39:20.513 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 600, 2024-04-26T03:39:20.513 INFO:teuthology.orchestra.run.smithi028.stdout: "ops": [ 2024-04-26T03:39:20.513 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.513 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(osd.3.24:1 3.6s0 3:64627763:::smithi09235267-49:head [assert-version v3,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-04-26T03:39:20.513 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:59.539567+0000", 2024-04-26T03:39:20.513 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 20.970794006999999, 2024-04-26T03:39:20.513 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0073782170000000003, 2024-04-26T03:39:20.513 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:20.513 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:20.513 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "started", 2024-04-26T03:39:20.514 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:20.514 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.514 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:20.514 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.539567+0000", 2024-04-26T03:39:20.514 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:20.514 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.514 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.514 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:20.514 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.539567+0000", 2024-04-26T03:39:20.514 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:20.514 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.514 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.514 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:20.514 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.539569+0000", 2024-04-26T03:39:20.515 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.9240000000000001e-06 2024-04-26T03:39:20.515 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.515 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.515 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:20.515 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.539575+0000", 2024-04-26T03:39:20.515 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.4949999999999999e-06 2024-04-26T03:39:20.515 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.515 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.515 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:20.515 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.539585+0000", 2024-04-26T03:39:20.515 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0084e-05 2024-04-26T03:39:20.515 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.515 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.515 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:20.516 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.539592+0000", 2024-04-26T03:39:20.516 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.3350000000000001e-06 2024-04-26T03:39:20.516 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.516 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.516 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:20.516 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.539651+0000", 2024-04-26T03:39:20.516 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.8789000000000002e-05 2024-04-26T03:39:20.516 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.516 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.516 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:20.516 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.539738+0000", 2024-04-26T03:39:20.516 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.6734e-05 2024-04-26T03:39:20.516 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.516 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.516 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:20.517 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.546946+0000", 2024-04-26T03:39:20.517 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0072078560000000003 2024-04-26T03:39:20.517 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:20.517 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:20.517 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:20.517 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.517 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.517 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.bs1 45/32 ECSubWrite(tid=68, reqid=client.4556.0:361, at_version=45'31, trim_to=0'0, roll_forward_to=43'30))", 2024-04-26T03:39:20.517 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:01.715207+0000", 2024-04-26T03:39:20.517 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 18.795154409999999, 2024-04-26T03:39:20.517 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0081499150000000006, 2024-04-26T03:39:20.517 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:20.517 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:20.517 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:20.517 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:20.518 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.518 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:20.518 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.715207+0000", 2024-04-26T03:39:20.518 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:20.518 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.518 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.518 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:20.518 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.715207+0000", 2024-04-26T03:39:20.518 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:20.518 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.518 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.518 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:20.518 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.715208+0000", 2024-04-26T03:39:20.519 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.3e-06 2024-04-26T03:39:20.519 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.519 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.519 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:20.519 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.715385+0000", 2024-04-26T03:39:20.519 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00017650000000000001 2024-04-26T03:39:20.519 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.520 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.520 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:20.520 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.715407+0000", 2024-04-26T03:39:20.520 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.2334999999999999e-05 2024-04-26T03:39:20.520 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.520 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.520 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:20.520 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.715418+0000", 2024-04-26T03:39:20.520 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1134999999999999e-05 2024-04-26T03:39:20.520 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.521 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.521 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:20.521 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.715534+0000", 2024-04-26T03:39:20.521 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000115322 2024-04-26T03:39:20.521 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.521 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.521 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:20.521 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.715544+0000", 2024-04-26T03:39:20.521 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0815000000000001e-05 2024-04-26T03:39:20.521 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.521 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.521 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:20.521 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.723316+0000", 2024-04-26T03:39:20.521 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0077715550000000003 2024-04-26T03:39:20.522 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.522 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.522 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:20.522 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.723357+0000", 2024-04-26T03:39:20.522 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.0952999999999999e-05 2024-04-26T03:39:20.522 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:20.522 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:20.522 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:20.522 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.522 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.522 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:430 3.6s0 3:64627763:::smithi09235267-49:head [append 0~0] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-04-26T03:39:20.522 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:04.683807+0000", 2024-04-26T03:39:20.522 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 15.826554623, 2024-04-26T03:39:20.522 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.017357337, 2024-04-26T03:39:20.523 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:20.523 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:20.523 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:20.523 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:20.523 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:20.523 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:20.523 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 430 2024-04-26T03:39:20.523 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.523 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:20.523 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.523 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:20.523 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.683807+0000", 2024-04-26T03:39:20.523 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:20.523 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.524 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.524 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:20.524 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.683807+0000", 2024-04-26T03:39:20.524 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:20.524 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.524 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.524 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:20.524 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.683809+0000", 2024-04-26T03:39:20.524 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.8610000000000001e-06 2024-04-26T03:39:20.524 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.524 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.524 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:20.524 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.683818+0000", 2024-04-26T03:39:20.524 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.4250000000000004e-06 2024-04-26T03:39:20.525 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.525 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.525 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:20.525 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.683834+0000", 2024-04-26T03:39:20.525 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.6228000000000001e-05 2024-04-26T03:39:20.525 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.525 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.525 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:20.525 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.683843+0000", 2024-04-26T03:39:20.525 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.2910000000000007e-06 2024-04-26T03:39:20.525 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.525 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.525 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:20.525 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.683899+0000", 2024-04-26T03:39:20.526 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.6681999999999997e-05 2024-04-26T03:39:20.526 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.526 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.526 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "op must wait for map", 2024-04-26T03:39:20.526 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.683904+0000", 2024-04-26T03:39:20.526 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.0719999999999999e-06 2024-04-26T03:39:20.526 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.526 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.526 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:20.526 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.688041+0000", 2024-04-26T03:39:20.526 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0041362339999999999 2024-04-26T03:39:20.526 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.526 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.526 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:20.527 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.688204+0000", 2024-04-26T03:39:20.527 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00016366800000000001 2024-04-26T03:39:20.527 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.527 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.527 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:20.527 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.689177+0000", 2024-04-26T03:39:20.527 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00097255499999999995 2024-04-26T03:39:20.527 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.527 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.527 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:20.527 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.692572+0000", 2024-04-26T03:39:20.527 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0033949879999999998 2024-04-26T03:39:20.527 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.527 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.528 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:20.528 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.701106+0000", 2024-04-26T03:39:20.528 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0085337069999999998 2024-04-26T03:39:20.528 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.528 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.528 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:20.528 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.701164+0000", 2024-04-26T03:39:20.528 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.8625999999999997e-05 2024-04-26T03:39:20.528 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:20.528 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:20.528 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:20.528 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.528 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.528 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:431 3.6s0 3:64627763:::smithi09235267-49:head [rollback 8] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-04-26T03:39:20.529 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:04.683867+0000", 2024-04-26T03:39:20.529 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 15.826494214, 2024-04-26T03:39:20.529 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.017754329999999999, 2024-04-26T03:39:20.529 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:20.529 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:20.529 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:20.529 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:20.529 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:20.530 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:20.530 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 431 2024-04-26T03:39:20.530 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.530 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:20.530 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.530 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:20.530 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.683867+0000", 2024-04-26T03:39:20.530 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:20.530 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.530 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.530 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:20.530 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.683867+0000", 2024-04-26T03:39:20.530 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:20.531 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.531 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.531 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:20.531 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.683868+0000", 2024-04-26T03:39:20.531 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.8599999999999999e-07 2024-04-26T03:39:20.531 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.531 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.531 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:20.531 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.683870+0000", 2024-04-26T03:39:20.531 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.2910000000000002e-06 2024-04-26T03:39:20.531 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.531 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.531 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:20.531 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.683875+0000", 2024-04-26T03:39:20.532 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.6829999999999999e-06 2024-04-26T03:39:20.532 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.532 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.532 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:20.532 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.683878+0000", 2024-04-26T03:39:20.532 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.4690000000000002e-06 2024-04-26T03:39:20.532 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.532 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.532 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:20.532 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.683965+0000", 2024-04-26T03:39:20.532 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.7071999999999996e-05 2024-04-26T03:39:20.532 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.532 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.532 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "waiting_for_map not empty", 2024-04-26T03:39:20.533 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.683974+0000", 2024-04-26T03:39:20.533 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.7159999999999992e-06 2024-04-26T03:39:20.533 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.533 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.533 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:20.533 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.689846+0000", 2024-04-26T03:39:20.533 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0058718399999999997 2024-04-26T03:39:20.533 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.533 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.533 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:20.533 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.689941+0000", 2024-04-26T03:39:20.533 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.5580000000000003e-05 2024-04-26T03:39:20.533 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.534 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.534 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:20.534 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.690497+0000", 2024-04-26T03:39:20.534 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00055555299999999997 2024-04-26T03:39:20.534 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.534 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.534 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:20.534 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.693675+0000", 2024-04-26T03:39:20.534 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0031778269999999998 2024-04-26T03:39:20.534 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.534 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.534 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:20.534 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.701555+0000", 2024-04-26T03:39:20.534 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0078804089999999997 2024-04-26T03:39:20.535 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.535 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.535 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:20.535 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.701622+0000", 2024-04-26T03:39:20.535 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.6503999999999999e-05 2024-04-26T03:39:20.535 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:20.535 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:20.535 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:20.535 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.535 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.535 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:432 3.6s0 3:64627763:::smithi09235267-49:head [append 0~0] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-04-26T03:39:20.535 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:04.683885+0000", 2024-04-26T03:39:20.535 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 15.826476270000001, 2024-04-26T03:39:20.535 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.017911924999999999, 2024-04-26T03:39:20.536 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:20.536 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:20.536 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:20.536 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:20.536 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:20.536 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:20.536 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 432 2024-04-26T03:39:20.536 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.536 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:20.536 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.536 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:20.536 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.683885+0000", 2024-04-26T03:39:20.536 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:20.536 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.536 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.537 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:20.537 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.683885+0000", 2024-04-26T03:39:20.537 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:20.537 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.537 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.537 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:20.537 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.683885+0000", 2024-04-26T03:39:20.537 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.1e-07 2024-04-26T03:39:20.537 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.537 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.537 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:20.537 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.683887+0000", 2024-04-26T03:39:20.537 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.863e-06 2024-04-26T03:39:20.537 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.537 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.538 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:20.538 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.683891+0000", 2024-04-26T03:39:20.538 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.3289999999999998e-06 2024-04-26T03:39:20.538 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.538 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.538 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:20.538 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.683893+0000", 2024-04-26T03:39:20.538 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.8169999999999999e-06 2024-04-26T03:39:20.538 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.538 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.538 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:20.538 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.683999+0000", 2024-04-26T03:39:20.538 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00010592200000000001 2024-04-26T03:39:20.538 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.539 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.539 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "waiting_for_map not empty", 2024-04-26T03:39:20.539 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.684007+0000", 2024-04-26T03:39:20.539 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.3119999999999996e-06 2024-04-26T03:39:20.539 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.539 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.539 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:20.539 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.690818+0000", 2024-04-26T03:39:20.539 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0068111589999999998 2024-04-26T03:39:20.539 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.539 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.539 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:20.539 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.690910+0000", 2024-04-26T03:39:20.539 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.1916999999999994e-05 2024-04-26T03:39:20.539 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.540 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.540 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:20.540 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.691360+0000", 2024-04-26T03:39:20.540 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00045033600000000002 2024-04-26T03:39:20.540 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.540 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.540 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:20.540 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.694888+0000", 2024-04-26T03:39:20.540 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0035276880000000002 2024-04-26T03:39:20.540 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.540 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.540 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:20.540 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.701741+0000", 2024-04-26T03:39:20.540 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0068528529999999999 2024-04-26T03:39:20.541 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.541 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.541 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:20.541 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.701797+0000", 2024-04-26T03:39:20.541 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.6518999999999999e-05 2024-04-26T03:39:20.541 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:20.541 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:20.541 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:20.541 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.541 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.541 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDPGPush(3.10s5 49/48 [PushOp(3:0e9b8628:::smithi09235267-14:head, version: 35'8, data_included: [0~253952], data_size: 253952, omap_header_size: 0, omap_entries_size: 0, attrset_size: 4, recovery_info: ObjectRecoveryInfo(3:0e9b8628:::smithi09235267-14:head@35'8, size: 1015808, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1015808, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-04-26T03:39:20.541 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:04.717140+0000", 2024-04-26T03:39:20.541 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 15.793221811, 2024-04-26T03:39:20.541 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0061295170000000001, 2024-04-26T03:39:20.542 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:20.542 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:20.542 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:20.542 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:20.542 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.542 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:20.542 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.717140+0000", 2024-04-26T03:39:20.542 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:20.542 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.542 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.542 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:20.542 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.717140+0000", 2024-04-26T03:39:20.542 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:20.543 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.543 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.543 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:20.543 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.717140+0000", 2024-04-26T03:39:20.543 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.1099999999999995e-07 2024-04-26T03:39:20.543 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.543 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.543 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:20.543 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.717608+0000", 2024-04-26T03:39:20.543 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000467935 2024-04-26T03:39:20.543 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.543 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.543 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:20.543 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.717670+0000", 2024-04-26T03:39:20.544 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.1774999999999997e-05 2024-04-26T03:39:20.544 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.544 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.544 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:20.544 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.717714+0000", 2024-04-26T03:39:20.544 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.4353e-05 2024-04-26T03:39:20.544 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.544 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.544 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:20.544 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.717815+0000", 2024-04-26T03:39:20.544 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00010097700000000001 2024-04-26T03:39:20.544 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.544 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.544 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:20.545 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.723269+0000", 2024-04-26T03:39:20.545 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0054538659999999999 2024-04-26T03:39:20.545 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:20.545 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:20.545 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:20.545 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.545 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.545 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.18s1 49/44 ECSubWrite(tid=206, reqid=client.4556.0:443, at_version=49'36, trim_to=0'0, roll_forward_to=49'34))", 2024-04-26T03:39:20.545 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:04.734356+0000", 2024-04-26T03:39:20.545 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 15.776005805, 2024-04-26T03:39:20.545 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0072519239999999999, 2024-04-26T03:39:20.545 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:20.545 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:20.545 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:20.546 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:20.546 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.546 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:20.546 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.734356+0000", 2024-04-26T03:39:20.546 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:20.546 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.546 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.546 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:20.546 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.734356+0000", 2024-04-26T03:39:20.546 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:20.546 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.546 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.546 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:20.546 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.734356+0000", 2024-04-26T03:39:20.547 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.2900000000000003e-07 2024-04-26T03:39:20.547 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.547 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.547 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:20.547 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.734606+0000", 2024-04-26T03:39:20.547 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00024913799999999999 2024-04-26T03:39:20.547 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.547 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.547 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:20.547 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.734622+0000", 2024-04-26T03:39:20.547 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.6453999999999999e-05 2024-04-26T03:39:20.547 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.547 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.548 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:20.548 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.734631+0000", 2024-04-26T03:39:20.548 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.6929999999999996e-06 2024-04-26T03:39:20.548 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.548 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.548 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:20.548 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.734713+0000", 2024-04-26T03:39:20.548 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.2408000000000001e-05 2024-04-26T03:39:20.548 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.548 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.548 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:20.548 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.734745+0000", 2024-04-26T03:39:20.548 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.1866999999999998e-05 2024-04-26T03:39:20.548 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.549 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.549 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:20.549 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.741565+0000", 2024-04-26T03:39:20.549 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0068200869999999999 2024-04-26T03:39:20.549 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.549 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.549 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:20.549 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.741608+0000", 2024-04-26T03:39:20.549 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.2548e-05 2024-04-26T03:39:20.549 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:20.549 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:20.549 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:20.549 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.549 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.550 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDPGPush(3.13s4 56/55 [PushOp(3:c944ba3d:::smithi09235267-6:head, version: 52'44, data_included: [0~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 4, recovery_info: ObjectRecoveryInfo(3:c944ba3d:::smithi09235267-6:head@52'44, size: 4669440, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-04-26T03:39:20.550 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:11.761882+0000", 2024-04-26T03:39:20.550 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 8.7484795149999997, 2024-04-26T03:39:20.550 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0061161080000000003, 2024-04-26T03:39:20.550 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:20.550 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:20.550 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:20.550 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:20.550 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.550 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:20.550 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.761882+0000", 2024-04-26T03:39:20.550 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:20.550 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.551 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.551 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:20.551 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.761882+0000", 2024-04-26T03:39:20.551 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:20.551 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.551 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.551 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:20.551 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.761883+0000", 2024-04-26T03:39:20.551 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.7400000000000003e-07 2024-04-26T03:39:20.551 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.551 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.551 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:20.551 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.762439+0000", 2024-04-26T03:39:20.551 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00055661700000000003 2024-04-26T03:39:20.551 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.552 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.552 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:20.552 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.762458+0000", 2024-04-26T03:39:20.552 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.8845e-05 2024-04-26T03:39:20.552 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.552 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.552 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:20.552 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.762479+0000", 2024-04-26T03:39:20.552 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.1399e-05 2024-04-26T03:39:20.552 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.552 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.552 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:20.552 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.762553+0000", 2024-04-26T03:39:20.552 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.3860000000000001e-05 2024-04-26T03:39:20.552 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.553 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.553 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:20.553 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.767998+0000", 2024-04-26T03:39:20.553 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0054447130000000003 2024-04-26T03:39:20.553 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:20.553 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:20.553 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:20.553 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.553 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.553 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.13s4 56/55 ECSubWrite(tid=104, reqid=client.4556.0:604, at_version=56'47, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:39:20.553 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:11.774415+0000", 2024-04-26T03:39:20.553 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 8.7359468239999991, 2024-04-26T03:39:20.553 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.016022932, 2024-04-26T03:39:20.553 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:20.554 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:20.554 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:20.554 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:20.554 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.554 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:20.554 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.774415+0000", 2024-04-26T03:39:20.554 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:20.554 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.554 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.554 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:20.554 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.774415+0000", 2024-04-26T03:39:20.554 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:20.554 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.554 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.555 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:20.555 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.774416+0000", 2024-04-26T03:39:20.555 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.497e-06 2024-04-26T03:39:20.555 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.555 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.555 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:20.555 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.775996+0000", 2024-04-26T03:39:20.555 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.001579558 2024-04-26T03:39:20.555 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.555 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.555 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:20.555 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.776007+0000", 2024-04-26T03:39:20.555 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1355000000000001e-05 2024-04-26T03:39:20.555 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.556 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.556 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:20.556 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.776013+0000", 2024-04-26T03:39:20.556 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.7579999999999998e-06 2024-04-26T03:39:20.556 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.556 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.556 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:20.556 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.776070+0000", 2024-04-26T03:39:20.556 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.6818999999999999e-05 2024-04-26T03:39:20.556 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.556 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.556 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:20.556 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.776076+0000", 2024-04-26T03:39:20.556 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.6460000000000004e-06 2024-04-26T03:39:20.557 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.557 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.557 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:20.557 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.790407+0000", 2024-04-26T03:39:20.557 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.014330489 2024-04-26T03:39:20.557 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.557 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.557 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:20.557 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.790438+0000", 2024-04-26T03:39:20.557 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.0809999999999998e-05 2024-04-26T03:39:20.557 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:20.557 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:20.557 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:20.557 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.557 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.558 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDPGPush(3.13s4 56/55 [PushOp(3:c944ba3d:::smithi09235267-6:head, version: 52'44, data_included: [262144~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:c944ba3d:::smithi09235267-6:head@52'44, size: 4669440, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-04-26T03:39:20.558 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:11.795216+0000", 2024-04-26T03:39:20.558 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 8.7151452939999992, 2024-04-26T03:39:20.558 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0063283059999999997, 2024-04-26T03:39:20.558 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:20.558 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:20.558 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:20.558 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:20.558 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.558 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:20.558 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.795216+0000", 2024-04-26T03:39:20.558 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:20.558 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.559 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.559 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:20.559 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.795216+0000", 2024-04-26T03:39:20.559 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:20.559 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.559 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.559 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:20.559 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.795217+0000", 2024-04-26T03:39:20.559 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.9800000000000003e-07 2024-04-26T03:39:20.559 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.559 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.559 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:20.559 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.795721+0000", 2024-04-26T03:39:20.559 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00050459599999999995 2024-04-26T03:39:20.560 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.560 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.560 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:20.560 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.795739+0000", 2024-04-26T03:39:20.560 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.7703e-05 2024-04-26T03:39:20.560 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.560 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.560 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:20.560 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.795753+0000", 2024-04-26T03:39:20.560 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.4313e-05 2024-04-26T03:39:20.560 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.560 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.560 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:20.560 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.795820+0000", 2024-04-26T03:39:20.561 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.6482999999999993e-05 2024-04-26T03:39:20.561 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.561 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.561 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:20.561 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.801544+0000", 2024-04-26T03:39:20.561 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0057246129999999999 2024-04-26T03:39:20.561 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:20.561 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:20.561 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:20.561 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.561 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.561 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDPGPush(3.13s4 56/55 [PushOp(3:c944ba3d:::smithi09235267-6:head, version: 52'44, data_included: [524288~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:c944ba3d:::smithi09235267-6:head@52'44, size: 4669440, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-04-26T03:39:20.561 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:11.810224+0000", 2024-04-26T03:39:20.561 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 8.7001373320000006, 2024-04-26T03:39:20.562 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0063452109999999999, 2024-04-26T03:39:20.562 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:20.562 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:20.562 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:20.562 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:20.562 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.562 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:20.562 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.810224+0000", 2024-04-26T03:39:20.562 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:20.562 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.562 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.562 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:20.562 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.810224+0000", 2024-04-26T03:39:20.562 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:20.563 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.563 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.563 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:20.563 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.810226+0000", 2024-04-26T03:39:20.563 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.424e-06 2024-04-26T03:39:20.563 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.563 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.563 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:20.563 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.810750+0000", 2024-04-26T03:39:20.563 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00052469099999999998 2024-04-26T03:39:20.563 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.563 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.563 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:20.563 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.810772+0000", 2024-04-26T03:39:20.563 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.2087e-05 2024-04-26T03:39:20.564 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.564 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.564 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:20.564 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.810785+0000", 2024-04-26T03:39:20.564 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.2449000000000001e-05 2024-04-26T03:39:20.564 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.564 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.564 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:20.564 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.810848+0000", 2024-04-26T03:39:20.564 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.3021000000000005e-05 2024-04-26T03:39:20.564 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.564 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.564 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:20.564 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.816569+0000", 2024-04-26T03:39:20.565 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0057215390000000003 2024-04-26T03:39:20.565 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:20.565 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:20.565 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:20.565 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.565 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.565 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDPGPush(3.13s4 56/55 [PushOp(3:c8a9e28b:::smithi09235267-30:head, version: 49'32, data_included: [0~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 4, recovery_info: ObjectRecoveryInfo(3:c8a9e28b:::smithi09235267-30:head@49'32, size: 2768896, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-04-26T03:39:20.565 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:11.811344+0000", 2024-04-26T03:39:20.565 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 8.6990177870000007, 2024-04-26T03:39:20.565 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.010863571000000001, 2024-04-26T03:39:20.565 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:20.565 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:20.565 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:20.566 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:20.566 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.566 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:20.566 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.811344+0000", 2024-04-26T03:39:20.566 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:20.566 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.566 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.566 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:20.566 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.811344+0000", 2024-04-26T03:39:20.566 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:20.566 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.566 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.566 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:20.566 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.811344+0000", 2024-04-26T03:39:20.567 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.4899999999999995e-07 2024-04-26T03:39:20.567 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.567 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.567 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:20.567 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.811943+0000", 2024-04-26T03:39:20.567 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00059826900000000001 2024-04-26T03:39:20.567 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.567 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.567 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:20.567 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.811957+0000", 2024-04-26T03:39:20.567 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.4831000000000001e-05 2024-04-26T03:39:20.567 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.567 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.567 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:20.568 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.811966+0000", 2024-04-26T03:39:20.568 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.0280000000000006e-06 2024-04-26T03:39:20.568 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.568 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.568 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:20.568 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.816605+0000", 2024-04-26T03:39:20.568 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0046387889999999999 2024-04-26T03:39:20.568 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.568 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.568 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:20.568 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.822207+0000", 2024-04-26T03:39:20.568 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0056020050000000002 2024-04-26T03:39:20.568 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:20.568 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:20.569 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:20.569 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.569 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.569 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDPGPush(3.13s4 56/55 [PushOp(3:c8a9e28b:::smithi09235267-30:head, version: 49'32, data_included: [262144~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:c8a9e28b:::smithi09235267-30:head@49'32, size: 2768896, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-04-26T03:39:20.569 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:11.833127+0000", 2024-04-26T03:39:20.569 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 8.6772343119999995, 2024-04-26T03:39:20.569 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.006043578, 2024-04-26T03:39:20.569 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:20.569 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:20.569 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:20.569 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:20.569 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.569 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:20.569 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.833127+0000", 2024-04-26T03:39:20.569 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:20.570 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.570 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.570 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:20.570 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.833127+0000", 2024-04-26T03:39:20.570 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:20.570 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.570 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.570 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:20.570 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.833128+0000", 2024-04-26T03:39:20.570 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.8999999999999996e-07 2024-04-26T03:39:20.570 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.570 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.570 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:20.570 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.833602+0000", 2024-04-26T03:39:20.571 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00047453699999999997 2024-04-26T03:39:20.571 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.571 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.571 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:20.571 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.833618+0000", 2024-04-26T03:39:20.571 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.5400000000000002e-05 2024-04-26T03:39:20.571 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.571 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.571 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:20.571 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.833628+0000", 2024-04-26T03:39:20.571 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0287e-05 2024-04-26T03:39:20.571 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.571 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.571 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:20.571 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.833688+0000", 2024-04-26T03:39:20.572 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.9741000000000003e-05 2024-04-26T03:39:20.572 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.572 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.572 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:20.572 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.839171+0000", 2024-04-26T03:39:20.572 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0054830230000000001 2024-04-26T03:39:20.572 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:20.572 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:20.572 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:20.572 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.572 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.572 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDPGPush(3.13s4 56/55 [PushOp(3:c8a9e28b:::smithi09235267-30:head, version: 49'32, data_included: [524288~167936], data_size: 167936, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:c8a9e28b:::smithi09235267-30:head@49'32, size: 2768896, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2768896, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-04-26T03:39:20.572 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:11.842440+0000", 2024-04-26T03:39:20.572 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 8.6679213239999999, 2024-04-26T03:39:20.573 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0079054569999999994, 2024-04-26T03:39:20.573 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:20.573 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:20.573 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:20.573 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:20.573 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.573 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:20.573 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.842440+0000", 2024-04-26T03:39:20.573 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:20.573 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.573 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.573 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:20.573 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.842440+0000", 2024-04-26T03:39:20.573 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:20.573 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.574 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.574 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:20.574 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.842441+0000", 2024-04-26T03:39:20.574 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.75e-07 2024-04-26T03:39:20.574 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.574 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.574 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:20.574 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.842876+0000", 2024-04-26T03:39:20.574 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000435757 2024-04-26T03:39:20.574 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.574 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.575 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:20.575 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.842896+0000", 2024-04-26T03:39:20.575 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.9202e-05 2024-04-26T03:39:20.575 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.575 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.575 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:20.575 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.842905+0000", 2024-04-26T03:39:20.575 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.4080000000000004e-06 2024-04-26T03:39:20.575 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.575 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.575 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:20.575 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.844849+0000", 2024-04-26T03:39:20.575 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.001944028 2024-04-26T03:39:20.575 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.575 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.576 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:20.576 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.850346+0000", 2024-04-26T03:39:20.576 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0054965869999999998 2024-04-26T03:39:20.576 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:20.576 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:20.576 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:20.576 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.576 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.576 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.10s5 60/48 ECSubWrite(tid=272, reqid=client.4556.0:703, at_version=60'20, trim_to=0'0, roll_forward_to=60'18))", 2024-04-26T03:39:20.576 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:15.958104+0000", 2024-04-26T03:39:20.576 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 4.5522574579999997, 2024-04-26T03:39:20.576 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0092449460000000004, 2024-04-26T03:39:20.576 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:20.576 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:20.576 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:20.577 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:20.577 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.577 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:20.577 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.958104+0000", 2024-04-26T03:39:20.577 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:20.577 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.577 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.577 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:20.577 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.958104+0000", 2024-04-26T03:39:20.577 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:20.577 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.577 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.577 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:20.577 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.958105+0000", 2024-04-26T03:39:20.578 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.2500000000000005e-07 2024-04-26T03:39:20.578 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.578 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.578 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:20.578 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.958846+0000", 2024-04-26T03:39:20.578 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00074102 2024-04-26T03:39:20.578 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.578 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.578 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:20.578 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.958857+0000", 2024-04-26T03:39:20.578 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1601e-05 2024-04-26T03:39:20.578 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.578 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.578 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:20.579 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.958865+0000", 2024-04-26T03:39:20.579 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.0260000000000007e-06 2024-04-26T03:39:20.579 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.579 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.579 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:20.579 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.958947+0000", 2024-04-26T03:39:20.579 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.1675999999999995e-05 2024-04-26T03:39:20.579 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.579 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.579 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:20.579 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.958956+0000", 2024-04-26T03:39:20.579 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.0710000000000001e-06 2024-04-26T03:39:20.579 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.579 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.580 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:20.580 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.967309+0000", 2024-04-26T03:39:20.580 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0083527810000000001 2024-04-26T03:39:20.580 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.580 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.580 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:20.580 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.967349+0000", 2024-04-26T03:39:20.580 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.0145999999999998e-05 2024-04-26T03:39:20.580 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:20.580 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:20.580 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:20.580 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.580 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.580 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.10s5 60/48 ECSubWrite(tid=273, reqid=client.4556.0:704, at_version=60'21, trim_to=0'0, roll_forward_to=60'18))", 2024-04-26T03:39:20.581 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:15.962785+0000", 2024-04-26T03:39:20.581 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 4.5475761400000003, 2024-04-26T03:39:20.581 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0092490620000000006, 2024-04-26T03:39:20.581 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:20.581 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:20.581 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:20.581 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:20.581 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.581 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:20.581 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.962785+0000", 2024-04-26T03:39:20.581 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:20.581 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.581 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.581 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:20.581 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.962785+0000", 2024-04-26T03:39:20.582 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:20.582 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.582 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.582 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:20.582 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.962786+0000", 2024-04-26T03:39:20.582 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.8999999999999995e-07 2024-04-26T03:39:20.582 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.582 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.582 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:20.582 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.962919+0000", 2024-04-26T03:39:20.582 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000132538 2024-04-26T03:39:20.582 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.583 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.583 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:20.583 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.962938+0000", 2024-04-26T03:39:20.583 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.9210000000000001e-05 2024-04-26T03:39:20.583 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.583 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.583 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:20.583 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.962946+0000", 2024-04-26T03:39:20.583 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.4570000000000005e-06 2024-04-26T03:39:20.583 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.583 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.583 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:20.583 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.962998+0000", 2024-04-26T03:39:20.583 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.1811000000000003e-05 2024-04-26T03:39:20.583 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.584 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.584 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:20.584 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.963007+0000", 2024-04-26T03:39:20.584 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.8550000000000008e-06 2024-04-26T03:39:20.584 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.584 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.584 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:20.584 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.972000+0000", 2024-04-26T03:39:20.584 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0089924989999999993 2024-04-26T03:39:20.584 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.584 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.584 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:20.584 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.972034+0000", 2024-04-26T03:39:20.584 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.4802000000000003e-05 2024-04-26T03:39:20.585 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:20.585 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:20.585 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:20.585 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.585 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.585 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.10s5 60/48 ECSubWrite(tid=274, reqid=client.4556.0:705, at_version=60'22, trim_to=0'0, roll_forward_to=60'18))", 2024-04-26T03:39:20.585 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:15.966729+0000", 2024-04-26T03:39:20.585 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 4.5436325770000003, 2024-04-26T03:39:20.585 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0095163020000000008, 2024-04-26T03:39:20.585 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:20.585 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:20.585 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:20.585 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:20.586 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.586 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:20.586 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.966729+0000", 2024-04-26T03:39:20.586 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:20.586 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.586 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.586 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:20.587 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.966729+0000", 2024-04-26T03:39:20.587 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:20.587 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.587 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.587 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:20.587 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.966730+0000", 2024-04-26T03:39:20.587 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1510000000000001e-06 2024-04-26T03:39:20.587 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.587 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.587 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:20.587 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.966955+0000", 2024-04-26T03:39:20.587 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000224897 2024-04-26T03:39:20.587 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.587 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.587 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:20.588 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.966981+0000", 2024-04-26T03:39:20.588 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.6290000000000001e-05 2024-04-26T03:39:20.588 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.588 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.588 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:20.588 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.966992+0000", 2024-04-26T03:39:20.588 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0740999999999999e-05 2024-04-26T03:39:20.588 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.588 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.588 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:20.588 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.967384+0000", 2024-04-26T03:39:20.588 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00039248700000000001 2024-04-26T03:39:20.588 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.588 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.589 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:20.589 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.967395+0000", 2024-04-26T03:39:20.589 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0168e-05 2024-04-26T03:39:20.589 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.589 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.589 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:20.589 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.976218+0000", 2024-04-26T03:39:20.589 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0088229169999999996 2024-04-26T03:39:20.589 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.589 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.589 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:20.589 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.976245+0000", 2024-04-26T03:39:20.589 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.7651e-05 2024-04-26T03:39:20.589 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:20.589 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:20.590 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:20.590 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.590 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.590 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.10s5 60/48 ECSubWrite(tid=275, reqid=client.4556.0:706, at_version=60'23, trim_to=0'0, roll_forward_to=60'18))", 2024-04-26T03:39:20.590 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:15.970750+0000", 2024-04-26T03:39:20.590 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 4.5396109970000005, 2024-04-26T03:39:20.590 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0072075180000000004, 2024-04-26T03:39:20.590 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:20.590 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:20.590 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:20.590 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:20.590 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.590 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:20.591 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.970750+0000", 2024-04-26T03:39:20.591 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:20.591 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.591 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.591 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:20.591 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.970750+0000", 2024-04-26T03:39:20.591 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:20.591 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.591 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.591 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:20.591 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.970751+0000", 2024-04-26T03:39:20.591 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.7699999999999992e-07 2024-04-26T03:39:20.591 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.591 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.592 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:20.592 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.970952+0000", 2024-04-26T03:39:20.592 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00020024499999999999 2024-04-26T03:39:20.592 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.592 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.592 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:20.592 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.970965+0000", 2024-04-26T03:39:20.592 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.3179999999999999e-05 2024-04-26T03:39:20.592 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.592 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.592 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:20.592 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.970974+0000", 2024-04-26T03:39:20.592 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.5570000000000001e-06 2024-04-26T03:39:20.592 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.592 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.593 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:20.593 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.972106+0000", 2024-04-26T03:39:20.593 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0011319279999999999 2024-04-26T03:39:20.593 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.593 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.593 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:20.593 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.972117+0000", 2024-04-26T03:39:20.593 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0181e-05 2024-04-26T03:39:20.593 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.593 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.593 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:20.593 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.977907+0000", 2024-04-26T03:39:20.593 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0057901180000000003 2024-04-26T03:39:20.593 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.594 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.594 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:20.594 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.977958+0000", 2024-04-26T03:39:20.594 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.1332000000000003e-05 2024-04-26T03:39:20.594 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:20.594 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:20.594 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:20.594 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.594 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.594 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.13s4 60/55 ECSubWrite(tid=141, reqid=client.4556.0:711, at_version=60'53, trim_to=0'0, roll_forward_to=60'51))", 2024-04-26T03:39:20.594 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:15.977943+0000", 2024-04-26T03:39:20.594 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 4.532418099, 2024-04-26T03:39:20.594 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.010870046, 2024-04-26T03:39:20.594 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:20.595 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:20.595 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:20.595 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:20.595 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.595 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:20.595 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.977943+0000", 2024-04-26T03:39:20.595 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:20.595 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.595 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.595 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:20.595 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.977943+0000", 2024-04-26T03:39:20.595 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:20.595 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.595 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.596 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:20.596 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.977944+0000", 2024-04-26T03:39:20.596 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0499999999999999e-06 2024-04-26T03:39:20.596 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.596 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.596 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:20.596 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.978162+0000", 2024-04-26T03:39:20.596 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000217238 2024-04-26T03:39:20.596 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.596 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.596 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:20.596 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.978177+0000", 2024-04-26T03:39:20.596 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.4895999999999999e-05 2024-04-26T03:39:20.596 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.596 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.597 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:20.597 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.978186+0000", 2024-04-26T03:39:20.597 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.9129999999999998e-06 2024-04-26T03:39:20.597 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.597 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.597 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:20.597 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.978255+0000", 2024-04-26T03:39:20.597 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.8915e-05 2024-04-26T03:39:20.597 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.597 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.597 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:20.597 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.978261+0000", 2024-04-26T03:39:20.597 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.0680000000000002e-06 2024-04-26T03:39:20.597 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.598 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.598 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:20.598 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.988781+0000", 2024-04-26T03:39:20.598 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.010519153 2024-04-26T03:39:20.598 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.598 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.598 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:20.598 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.988813+0000", 2024-04-26T03:39:20.598 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.2812999999999999e-05 2024-04-26T03:39:20.598 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:20.598 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:20.598 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:20.598 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.598 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.599 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.13s4 60/55 ECSubWrite(tid=142, reqid=client.4556.0:712, at_version=60'54, trim_to=0'0, roll_forward_to=60'51))", 2024-04-26T03:39:20.599 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:15.983346+0000", 2024-04-26T03:39:20.599 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 4.527015263, 2024-04-26T03:39:20.599 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0075977960000000004, 2024-04-26T03:39:20.599 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:20.599 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:20.599 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:20.599 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:20.599 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.599 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:20.599 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.983346+0000", 2024-04-26T03:39:20.599 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:20.599 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.599 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.599 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:20.600 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.983346+0000", 2024-04-26T03:39:20.600 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:20.600 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.600 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.600 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:20.600 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.983347+0000", 2024-04-26T03:39:20.600 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.1900000000000002e-07 2024-04-26T03:39:20.600 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.600 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.600 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:20.600 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.983538+0000", 2024-04-26T03:39:20.600 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000190807 2024-04-26T03:39:20.600 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.600 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.601 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:20.601 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.983548+0000", 2024-04-26T03:39:20.601 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0566e-05 2024-04-26T03:39:20.601 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.601 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.601 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:20.601 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.983557+0000", 2024-04-26T03:39:20.601 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.9700000000000005e-06 2024-04-26T03:39:20.601 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.601 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.601 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:20.601 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.983621+0000", 2024-04-26T03:39:20.601 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.3316000000000002e-05 2024-04-26T03:39:20.601 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.601 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.602 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:20.602 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.983630+0000", 2024-04-26T03:39:20.602 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.9119999999999993e-06 2024-04-26T03:39:20.602 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.602 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.602 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:20.602 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.990910+0000", 2024-04-26T03:39:20.602 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00727986 2024-04-26T03:39:20.602 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.602 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.602 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:20.602 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.990944+0000", 2024-04-26T03:39:20.602 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.3646000000000003e-05 2024-04-26T03:39:20.602 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:20.603 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:20.603 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:20.603 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:20.603 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:20.603 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:39:20.605 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-26T03:39:20.605 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:39:20.603+0000 7f60d3c4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31757) UID: 0 2024-04-26T03:39:20.686 INFO:teuthology.orchestra.run.smithi028.stderr:nodeep-scrub is unset 2024-04-26T03:39:20.689 INFO:tasks.rados.rados.0.smithi092.stdout:277: done (2 left) 2024-04-26T03:39:20.689 INFO:tasks.rados.rados.0.smithi092.stdout:281: done (1 left) 2024-04-26T03:39:20.689 INFO:tasks.rados.rados.0.smithi092.stdout:282: done (0 left) 2024-04-26T03:39:20.689 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 13 current snap is 17 2024-04-26T03:39:20.689 INFO:tasks.rados.rados.0.smithi092.stdout:283: seq_num 132 ranges {4128768=671744,4800512=573440,5373952=753664,6127616=229376} 2024-04-26T03:39:20.705 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:39:20.703+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:39:20.714 INFO:tasks.rados.rados.0.smithi092.stdout:283: writing smithi09235267-13 from 4128768 to 4800512 tid 1 2024-04-26T03:39:20.716 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:39:20.716 INFO:teuthology.orchestra.run.smithi028.stdout: "ops": [ 2024-04-26T03:39:20.716 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.716 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:745 3.bs0 3:d42e9299:::smithi09235267-13:head [append 0~671744 in=671744b] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:20.716 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:20.714217+0000", 2024-04-26T03:39:20.716 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 0.0011638359999999999, 2024-04-26T03:39:20.716 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0011853020000000001, 2024-04-26T03:39:20.716 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:20.716 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:20.716 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "started", 2024-04-26T03:39:20.717 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:20.717 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:20.717 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:20.717 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 745 2024-04-26T03:39:20.717 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.717 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:20.717 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.717 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:20.717 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.714217+0000", 2024-04-26T03:39:20.717 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:20.717 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.717 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.717 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:20.717 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.714217+0000", 2024-04-26T03:39:20.717 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:20.718 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.718 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.718 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:20.718 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.714219+0000", 2024-04-26T03:39:20.718 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.426e-06 2024-04-26T03:39:20.718 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.718 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.718 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:20.718 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.714924+0000", 2024-04-26T03:39:20.718 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00070503899999999997 2024-04-26T03:39:20.718 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.718 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.718 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:20.719 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.714937+0000", 2024-04-26T03:39:20.719 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.2741999999999999e-05 2024-04-26T03:39:20.719 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.719 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.719 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:20.719 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.714944+0000", 2024-04-26T03:39:20.719 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.5039999999999996e-06 2024-04-26T03:39:20.719 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.719 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.719 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:20.719 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.715015+0000", 2024-04-26T03:39:20.719 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.1143000000000006e-05 2024-04-26T03:39:20.719 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:20.719 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:20.720 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:20.720 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.715092+0000", 2024-04-26T03:39:20.720 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.7068000000000001e-05 2024-04-26T03:39:20.720 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:20.720 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:20.720 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:20.720 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:20.720 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-04-26T03:39:20.720 INFO:teuthology.orchestra.run.smithi028.stdout: "num_ops": 1 2024-04-26T03:39:20.720 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:39:20.721 INFO:tasks.rados.rados.0.smithi092.stdout:283: writing smithi09235267-13 from 4800512 to 5373952 tid 2 2024-04-26T03:39:20.722 INFO:tasks.rados.rados.0.smithi092.stdout:283: writing smithi09235267-13 from 5373952 to 6127616 tid 3 2024-04-26T03:39:20.723 INFO:tasks.rados.rados.0.smithi092.stdout:283: writing smithi09235267-13 from 6127616 to 6356992 tid 4 2024-04-26T03:39:20.723 INFO:tasks.rados.rados.0.smithi092.stdout:284: read oid 15 snap -1 2024-04-26T03:39:20.723 INFO:tasks.rados.rados.0.smithi092.stdout:284: expect deleted 2024-04-26T03:39:20.724 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 12 current snap is 17 2024-04-26T03:39:20.724 INFO:tasks.rados.rados.0.smithi092.stdout:285: seq_num 133 ranges {196608=688128,884736=720896,1605632=475136,2080768=360448} 2024-04-26T03:39:20.727 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-26T03:39:20.729 INFO:tasks.rados.rados.0.smithi092.stdout:285: writing smithi09235267-12 from 196608 to 884736 tid 1 2024-04-26T03:39:20.733 INFO:tasks.rados.rados.0.smithi092.stdout:285: writing smithi09235267-12 from 884736 to 1605632 tid 2 2024-04-26T03:39:20.736 INFO:tasks.rados.rados.0.smithi092.stdout:285: writing smithi09235267-12 from 1605632 to 2080768 tid 3 2024-04-26T03:39:20.738 INFO:tasks.rados.rados.0.smithi092.stdout:285: writing smithi09235267-12 from 2080768 to 2441216 tid 4 2024-04-26T03:39:20.738 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 39 current snap is 17 2024-04-26T03:39:20.738 INFO:tasks.rados.rados.0.smithi092.stdout:286: seq_num 134 ranges {638976=704512,1343488=475136,1818624=573440,2392064=507904} 2024-04-26T03:39:20.745 INFO:tasks.rados.rados.0.smithi092.stdout:286: writing smithi09235267-39 from 638976 to 1343488 tid 1 2024-04-26T03:39:20.748 INFO:tasks.rados.rados.0.smithi092.stdout:286: writing smithi09235267-39 from 1343488 to 1818624 tid 2 2024-04-26T03:39:20.752 INFO:tasks.rados.rados.0.smithi092.stdout:286: writing smithi09235267-39 from 1818624 to 2392064 tid 3 2024-04-26T03:39:20.755 INFO:tasks.rados.rados.0.smithi092.stdout:286: writing smithi09235267-39 from 2392064 to 2899968 tid 4 2024-04-26T03:39:20.755 INFO:tasks.rados.rados.0.smithi092.stdout:287: copy_from oid 6 from oid 48 current snap is 17 2024-04-26T03:39:20.755 INFO:tasks.rados.rados.0.smithi092.stdout:283: finishing write tid 1 to smithi09235267-13 2024-04-26T03:39:20.755 INFO:tasks.rados.rados.0.smithi092.stdout:283: finishing write tid 2 to smithi09235267-13 2024-04-26T03:39:20.755 INFO:tasks.rados.rados.0.smithi092.stdout:284: done (4 left) 2024-04-26T03:39:20.755 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 29 current snap is 17 2024-04-26T03:39:20.755 INFO:tasks.rados.rados.0.smithi092.stdout:283: finishing write tid 3 to smithi09235267-13 2024-04-26T03:39:20.755 INFO:tasks.rados.rados.0.smithi092.stdout:283: finishing write tid 4 to smithi09235267-13 2024-04-26T03:39:20.756 INFO:tasks.rados.rados.0.smithi092.stdout:288: seq_num 135 ranges {475136=720896,1196032=622592,1818624=688128,2506752=245760} 2024-04-26T03:39:20.762 INFO:tasks.rados.rados.0.smithi092.stdout:288: writing smithi09235267-29 from 475136 to 1196032 tid 1 2024-04-26T03:39:20.766 INFO:tasks.rados.rados.0.smithi092.stdout:288: writing smithi09235267-29 from 1196032 to 1818624 tid 2 2024-04-26T03:39:20.770 INFO:tasks.rados.rados.0.smithi092.stdout:288: writing smithi09235267-29 from 1818624 to 2506752 tid 3 2024-04-26T03:39:20.772 INFO:tasks.rados.rados.0.smithi092.stdout:288: writing smithi09235267-29 from 2506752 to 2752512 tid 4 2024-04-26T03:39:20.772 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 22 current snap is 17 2024-04-26T03:39:20.773 INFO:tasks.rados.rados.0.smithi092.stdout:283: finishing write tid 5 to smithi09235267-13 2024-04-26T03:39:20.773 INFO:tasks.rados.rados.0.smithi092.stdout:283: finishing write tid 6 to smithi09235267-13 2024-04-26T03:39:20.773 INFO:tasks.rados.rados.0.smithi092.stdout:283: oid 13 updating version 0 to 82 2024-04-26T03:39:20.773 INFO:tasks.rados.rados.0.smithi092.stdout:283: oid 13 version 82 is already newer than 81 2024-04-26T03:39:20.773 INFO:tasks.rados.rados.0.smithi092.stdout:283: oid 13 updating version 82 to 85 2024-04-26T03:39:20.773 INFO:tasks.rados.rados.0.smithi092.stdout:283: oid 13 version 85 is already newer than 83 2024-04-26T03:39:20.773 INFO:tasks.rados.rados.0.smithi092.stdout:283: oid 13 version 85 is already newer than 84 2024-04-26T03:39:20.773 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 13 v 85 (ObjNum 132 snap 17 seq_num 132) dirty exists 2024-04-26T03:39:20.773 INFO:tasks.rados.rados.0.smithi092.stdout:283: left oid 13 (ObjNum 132 snap 17 seq_num 132) 2024-04-26T03:39:20.773 INFO:tasks.rados.rados.0.smithi092.stdout:285: finishing write tid 1 to smithi09235267-12 2024-04-26T03:39:20.773 INFO:tasks.rados.rados.0.smithi092.stdout:285: finishing write tid 2 to smithi09235267-12 2024-04-26T03:39:20.773 INFO:tasks.rados.rados.0.smithi092.stdout:289: seq_num 136 ranges {753664=737280,1490944=770048,2260992=540672,2801664=245760} 2024-04-26T03:39:20.780 INFO:tasks.rados.rados.0.smithi092.stdout:289: writing smithi09235267-22 from 753664 to 1490944 tid 1 2024-04-26T03:39:20.785 INFO:tasks.rados.rados.0.smithi092.stdout:289: writing smithi09235267-22 from 1490944 to 2260992 tid 2 2024-04-26T03:39:20.788 INFO:tasks.rados.rados.0.smithi092.stdout:289: writing smithi09235267-22 from 2260992 to 2801664 tid 3 2024-04-26T03:39:20.791 INFO:tasks.rados.rados.0.smithi092.stdout:289: writing smithi09235267-22 from 2801664 to 3047424 tid 4 2024-04-26T03:39:20.791 INFO:tasks.rados.rados.0.smithi092.stdout:283: done (5 left) 2024-04-26T03:39:20.791 INFO:tasks.rados.rados.0.smithi092.stdout:290: delete oid 26 current snap is 17 2024-04-26T03:39:20.791 INFO:tasks.rados.rados.0.smithi092.stdout:285: finishing write tid 3 to smithi09235267-12 2024-04-26T03:39:20.791 INFO:tasks.rados.rados.0.smithi092.stdout:285: finishing write tid 4 to smithi09235267-12 2024-04-26T03:39:20.791 INFO:tasks.rados.rados.0.smithi092.stdout:285: finishing write tid 5 to smithi09235267-12 2024-04-26T03:39:20.791 INFO:tasks.rados.rados.0.smithi092.stdout:285: finishing write tid 6 to smithi09235267-12 2024-04-26T03:39:20.791 INFO:tasks.rados.rados.0.smithi092.stdout:285: oid 12 updating version 0 to 37 2024-04-26T03:39:20.791 INFO:tasks.rados.rados.0.smithi092.stdout:285: oid 12 updating version 37 to 41 2024-04-26T03:39:20.792 INFO:tasks.rados.rados.0.smithi092.stdout:285: oid 12 version 41 is already newer than 39 2024-04-26T03:39:20.792 INFO:tasks.rados.rados.0.smithi092.stdout:285: oid 12 version 41 is already newer than 38 2024-04-26T03:39:20.792 INFO:tasks.rados.rados.0.smithi092.stdout:285: oid 12 version 41 is already newer than 40 2024-04-26T03:39:20.792 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 12 v 41 (ObjNum 133 snap 17 seq_num 133) dirty exists 2024-04-26T03:39:20.792 INFO:tasks.rados.rados.0.smithi092.stdout:285: left oid 12 (ObjNum 133 snap 17 seq_num 133) 2024-04-26T03:39:20.792 INFO:tasks.rados.rados.0.smithi092.stdout:287: finishing copy_from racing read to smithi09235267-6 2024-04-26T03:39:20.792 INFO:tasks.rados.rados.0.smithi092.stdout:287: finishing copy_from to smithi09235267-6 2024-04-26T03:39:20.792 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 6 v 34 (ObjNum 161 snap 0 seq_num 1) dirty exists 2024-04-26T03:39:20.792 INFO:tasks.rados.rados.0.smithi092.stdout:286: finishing write tid 1 to smithi09235267-39 2024-04-26T03:39:20.792 INFO:tasks.rados.rados.0.smithi092.stdout:286: finishing write tid 2 to smithi09235267-39 2024-04-26T03:39:20.792 INFO:tasks.rados.rados.0.smithi092.stdout:286: finishing write tid 3 to smithi09235267-39 2024-04-26T03:39:20.792 INFO:tasks.rados.rados.0.smithi092.stdout:286: finishing write tid 4 to smithi09235267-39 2024-04-26T03:39:20.792 INFO:tasks.rados.rados.0.smithi092.stdout:286: finishing write tid 5 to smithi09235267-39 2024-04-26T03:39:20.792 INFO:tasks.rados.rados.0.smithi092.stdout:286: finishing write tid 6 to smithi09235267-39 2024-04-26T03:39:20.792 INFO:tasks.rados.rados.0.smithi092.stdout:286: oid 39 updating version 0 to 71 2024-04-26T03:39:20.793 INFO:tasks.rados.rados.0.smithi092.stdout:286: oid 39 updating version 71 to 73 2024-04-26T03:39:20.793 INFO:tasks.rados.rados.0.smithi092.stdout:286: oid 39 version 73 is already newer than 72 2024-04-26T03:39:20.793 INFO:tasks.rados.rados.0.smithi092.stdout:286: oid 39 updating version 73 to 74 2024-04-26T03:39:20.793 INFO:tasks.rados.rados.0.smithi092.stdout:286: oid 39 version 74 is already newer than 70 2024-04-26T03:39:20.793 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 39 v 74 (ObjNum 134 snap 17 seq_num 134) dirty exists 2024-04-26T03:39:20.793 INFO:tasks.rados.rados.0.smithi092.stdout:286: left oid 39 (ObjNum 134 snap 17 seq_num 134) 2024-04-26T03:39:20.793 INFO:tasks.rados.rados.0.smithi092.stdout:288: finishing write tid 1 to smithi09235267-29 2024-04-26T03:39:20.793 INFO:tasks.rados.rados.0.smithi092.stdout:288: finishing write tid 2 to smithi09235267-29 2024-04-26T03:39:20.793 INFO:tasks.rados.rados.0.smithi092.stdout:288: finishing write tid 3 to smithi09235267-29 2024-04-26T03:39:20.793 INFO:tasks.rados.rados.0.smithi092.stdout:288: finishing write tid 4 to smithi09235267-29 2024-04-26T03:39:20.793 INFO:tasks.rados.rados.0.smithi092.stdout:288: finishing write tid 5 to smithi09235267-29 2024-04-26T03:39:20.794 INFO:tasks.rados.rados.0.smithi092.stdout:288: finishing write tid 6 to smithi09235267-29 2024-04-26T03:39:20.794 INFO:tasks.rados.rados.0.smithi092.stdout:288: oid 29 updating version 0 to 54 2024-04-26T03:39:20.794 INFO:tasks.rados.rados.0.smithi092.stdout:288: oid 29 version 54 is already newer than 53 2024-04-26T03:39:20.794 INFO:tasks.rados.rados.0.smithi092.stdout:288: oid 29 version 54 is already newer than 50 2024-04-26T03:39:20.794 INFO:tasks.rados.rados.0.smithi092.stdout:288: oid 29 version 54 is already newer than 52 2024-04-26T03:39:20.794 INFO:tasks.rados.rados.0.smithi092.stdout:288: oid 29 version 54 is already newer than 51 2024-04-26T03:39:20.795 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 29 v 54 (ObjNum 135 snap 17 seq_num 135) dirty exists 2024-04-26T03:39:20.795 INFO:tasks.rados.rados.0.smithi092.stdout:288: left oid 29 (ObjNum 135 snap 17 seq_num 135) 2024-04-26T03:39:20.795 INFO:tasks.rados.rados.0.smithi092.stdout:285: done (5 left) 2024-04-26T03:39:20.795 INFO:tasks.rados.rados.0.smithi092.stdout:286: done (4 left) 2024-04-26T03:39:20.795 INFO:tasks.rados.rados.0.smithi092.stdout:287: done (3 left) 2024-04-26T03:39:20.795 INFO:tasks.rados.rados.0.smithi092.stdout:288: done (2 left) 2024-04-26T03:39:20.795 INFO:tasks.rados.rados.0.smithi092.stdout:290: done (1 left) 2024-04-26T03:39:20.795 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 46 current snap is 17 2024-04-26T03:39:20.795 INFO:tasks.rados.rados.0.smithi092.stdout:291: seq_num 138 ranges {3751936=770048,4521984=540672,5062656=753664,5816320=262144} 2024-04-26T03:39:20.805 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:39:20.804+0000 7fb78c544640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31956) UID: 0 2024-04-26T03:39:20.818 INFO:tasks.rados.rados.0.smithi092.stdout:291: writing smithi09235267-46 from 3751936 to 4521984 tid 1 2024-04-26T03:39:20.822 INFO:tasks.rados.rados.0.smithi092.stdout:291: writing smithi09235267-46 from 4521984 to 5062656 tid 2 2024-04-26T03:39:20.827 INFO:tasks.rados.rados.0.smithi092.stdout:291: writing smithi09235267-46 from 5062656 to 5816320 tid 3 2024-04-26T03:39:20.829 INFO:tasks.rados.rados.0.smithi092.stdout:291: writing smithi09235267-46 from 5816320 to 6078464 tid 4 2024-04-26T03:39:20.829 INFO:tasks.rados.rados.0.smithi092.stdout:292: read oid 5 snap -1 2024-04-26T03:39:20.829 INFO:tasks.rados.rados.0.smithi092.stdout:292: expect (ObjNum 125 snap 14 seq_num 125) 2024-04-26T03:39:20.830 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 25 current snap is 17 2024-04-26T03:39:20.830 INFO:tasks.rados.rados.0.smithi092.stdout:293: seq_num 139 ranges {3031040=786432,3817472=688128,4505600=475136,4980736=393216} 2024-04-26T03:39:20.850 INFO:tasks.rados.rados.0.smithi092.stdout:293: writing smithi09235267-25 from 3031040 to 3817472 tid 1 2024-04-26T03:39:20.855 INFO:tasks.rados.rados.0.smithi092.stdout:293: writing smithi09235267-25 from 3817472 to 4505600 tid 2 2024-04-26T03:39:20.858 INFO:tasks.rados.rados.0.smithi092.stdout:293: writing smithi09235267-25 from 4505600 to 4980736 tid 3 2024-04-26T03:39:20.861 INFO:tasks.rados.rados.0.smithi092.stdout:293: writing smithi09235267-25 from 4980736 to 5373952 tid 4 2024-04-26T03:39:20.861 INFO:tasks.rados.rados.0.smithi092.stdout:294: delete oid 6 current snap is 17 2024-04-26T03:39:20.861 INFO:tasks.rados.rados.0.smithi092.stdout:289: finishing write tid 1 to smithi09235267-22 2024-04-26T03:39:20.861 INFO:tasks.rados.rados.0.smithi092.stdout:289: finishing write tid 2 to smithi09235267-22 2024-04-26T03:39:20.862 INFO:tasks.rados.rados.0.smithi092.stdout:289: finishing write tid 3 to smithi09235267-22 2024-04-26T03:39:20.862 INFO:tasks.rados.rados.0.smithi092.stdout:289: finishing write tid 4 to smithi09235267-22 2024-04-26T03:39:20.862 INFO:tasks.rados.rados.0.smithi092.stdout:289: finishing write tid 5 to smithi09235267-22 2024-04-26T03:39:20.862 INFO:tasks.rados.rados.0.smithi092.stdout:289: finishing write tid 6 to smithi09235267-22 2024-04-26T03:39:20.862 INFO:tasks.rados.rados.0.smithi092.stdout:289: oid 22 updating version 0 to 55 2024-04-26T03:39:20.862 INFO:tasks.rados.rados.0.smithi092.stdout:289: oid 22 version 55 is already newer than 54 2024-04-26T03:39:20.862 INFO:tasks.rados.rados.0.smithi092.stdout:289: oid 22 version 55 is already newer than 51 2024-04-26T03:39:20.862 INFO:tasks.rados.rados.0.smithi092.stdout:289: oid 22 version 55 is already newer than 52 2024-04-26T03:39:20.862 INFO:tasks.rados.rados.0.smithi092.stdout:289: oid 22 version 55 is already newer than 53 2024-04-26T03:39:20.862 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 22 v 55 (ObjNum 136 snap 17 seq_num 136) dirty exists 2024-04-26T03:39:20.862 INFO:tasks.rados.rados.0.smithi092.stdout:289: left oid 22 (ObjNum 136 snap 17 seq_num 136) 2024-04-26T03:39:20.862 INFO:tasks.rados.rados.0.smithi092.stdout:291: finishing write tid 1 to smithi09235267-46 2024-04-26T03:39:20.862 INFO:tasks.rados.rados.0.smithi092.stdout:292: expect (ObjNum 125 snap 14 seq_num 125) 2024-04-26T03:39:20.883 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:39:20.883 INFO:teuthology.orchestra.run.smithi028.stdout: "ops": [], 2024-04-26T03:39:20.883 INFO:teuthology.orchestra.run.smithi028.stdout: "complaint_time": 30, 2024-04-26T03:39:20.883 INFO:teuthology.orchestra.run.smithi028.stdout: "num_blocked_ops": 0 2024-04-26T03:39:20.883 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:39:20.892 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-26T03:39:20.924 INFO:tasks.rados.rados.0.smithi092.stdout:291: finishing write tid 2 to smithi09235267-46 2024-04-26T03:39:20.924 INFO:tasks.rados.rados.0.smithi092.stdout:291: finishing write tid 3 to smithi09235267-46 2024-04-26T03:39:20.924 INFO:tasks.rados.rados.0.smithi092.stdout:291: finishing write tid 4 to smithi09235267-46 2024-04-26T03:39:20.924 INFO:tasks.rados.rados.0.smithi092.stdout:291: finishing write tid 5 to smithi09235267-46 2024-04-26T03:39:20.924 INFO:tasks.rados.rados.0.smithi092.stdout:291: finishing write tid 6 to smithi09235267-46 2024-04-26T03:39:20.925 INFO:tasks.rados.rados.0.smithi092.stdout:291: oid 46 updating version 0 to 61 2024-04-26T03:39:20.925 INFO:tasks.rados.rados.0.smithi092.stdout:291: oid 46 updating version 61 to 62 2024-04-26T03:39:20.925 INFO:tasks.rados.rados.0.smithi092.stdout:291: oid 46 version 62 is already newer than 60 2024-04-26T03:39:20.925 INFO:tasks.rados.rados.0.smithi092.stdout:291: oid 46 updating version 62 to 63 2024-04-26T03:39:20.925 INFO:tasks.rados.rados.0.smithi092.stdout:291: oid 46 version 63 is already newer than 59 2024-04-26T03:39:20.925 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 46 v 63 (ObjNum 138 snap 17 seq_num 138) dirty exists 2024-04-26T03:39:20.925 INFO:tasks.rados.rados.0.smithi092.stdout:291: left oid 46 (ObjNum 138 snap 17 seq_num 138) 2024-04-26T03:39:20.925 INFO:tasks.rados.rados.0.smithi092.stdout:289: done (4 left) 2024-04-26T03:39:20.925 INFO:tasks.rados.rados.0.smithi092.stdout:291: done (3 left) 2024-04-26T03:39:20.925 INFO:tasks.rados.rados.0.smithi092.stdout:292: done (2 left) 2024-04-26T03:39:20.925 INFO:tasks.rados.rados.0.smithi092.stdout:294: done (1 left) 2024-04-26T03:39:20.925 INFO:tasks.rados.rados.0.smithi092.stdout:295: delete oid 29 current snap is 17 2024-04-26T03:39:20.925 INFO:tasks.rados.rados.0.smithi092.stdout:293: finishing write tid 1 to smithi09235267-25 2024-04-26T03:39:20.926 INFO:tasks.rados.rados.0.smithi092.stdout:293: finishing write tid 2 to smithi09235267-25 2024-04-26T03:39:20.926 INFO:tasks.rados.rados.0.smithi092.stdout:293: finishing write tid 3 to smithi09235267-25 2024-04-26T03:39:20.926 INFO:tasks.rados.rados.0.smithi092.stdout:293: finishing write tid 4 to smithi09235267-25 2024-04-26T03:39:20.926 INFO:tasks.rados.rados.0.smithi092.stdout:293: finishing write tid 5 to smithi09235267-25 2024-04-26T03:39:20.926 INFO:tasks.rados.rados.0.smithi092.stdout:293: finishing write tid 6 to smithi09235267-25 2024-04-26T03:39:20.926 INFO:tasks.rados.rados.0.smithi092.stdout:293: oid 25 updating version 0 to 79 2024-04-26T03:39:20.926 INFO:tasks.rados.rados.0.smithi092.stdout:293: oid 25 updating version 79 to 80 2024-04-26T03:39:20.926 INFO:tasks.rados.rados.0.smithi092.stdout:293: oid 25 version 80 is already newer than 76 2024-04-26T03:39:20.926 INFO:tasks.rados.rados.0.smithi092.stdout:293: oid 25 version 80 is already newer than 78 2024-04-26T03:39:20.926 INFO:tasks.rados.rados.0.smithi092.stdout:293: oid 25 version 80 is already newer than 77 2024-04-26T03:39:20.926 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 25 v 80 (ObjNum 139 snap 17 seq_num 139) dirty exists 2024-04-26T03:39:20.927 INFO:tasks.rados.rados.0.smithi092.stdout:293: left oid 25 (ObjNum 139 snap 17 seq_num 139) 2024-04-26T03:39:20.928 INFO:tasks.rados.rados.0.smithi092.stdout:293: done (1 left) 2024-04-26T03:39:20.928 INFO:tasks.rados.rados.0.smithi092.stdout:295: done (0 left) 2024-04-26T03:39:20.928 INFO:tasks.rados.rados.0.smithi092.stdout:296: copy_from oid 28 from oid 26 current snap is 17 2024-04-26T03:39:20.928 INFO:tasks.rados.rados.0.smithi092.stdout:297: rollback oid 50 current snap is 17 2024-04-26T03:39:20.928 INFO:tasks.rados.rados.0.smithi092.stdout:rollback oid 50 to 12 2024-04-26T03:39:20.928 INFO:tasks.rados.rados.0.smithi092.stdout:298: snap_create 2024-04-26T03:39:20.928 INFO:tasks.rados.rados.0.smithi092.stdout:296: finishing copy_from to smithi09235267-28 2024-04-26T03:39:20.928 INFO:tasks.rados.rados.0.smithi092.stdout:296: got expected ENOENT (src dne) 2024-04-26T03:39:20.928 INFO:tasks.rados.rados.0.smithi092.stdout:296: finishing copy_from racing read to smithi09235267-28 2024-04-26T03:39:20.930 INFO:tasks.rados.rados.0.smithi092.stdout:297: finishing rollback tid 0 to smithi09235267-50 2024-04-26T03:39:20.930 INFO:tasks.rados.rados.0.smithi092.stdout:297: finishing rollback tid 1 to smithi09235267-50 2024-04-26T03:39:20.931 INFO:tasks.rados.rados.0.smithi092.stdout:297: finishing rollback tid 2 to smithi09235267-50 2024-04-26T03:39:20.931 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 50 v 84 (ObjNum 89 snap 7 seq_num 89) dirty exists 2024-04-26T03:39:20.936 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:39:20.934+0000 7fba0d2ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31961) UID: 0 2024-04-26T03:39:21.001 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:39:21.001 INFO:teuthology.orchestra.run.smithi028.stdout: "size": 20, 2024-04-26T03:39:21.001 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 600, 2024-04-26T03:39:21.001 INFO:teuthology.orchestra.run.smithi028.stdout: "ops": [ 2024-04-26T03:39:21.001 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.001 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:9 3.bs0 3:dbf634d0:::smithi09235267-3:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:21.001 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.481597+0000", 2024-04-26T03:39:21.001 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 29.516854357, 2024-04-26T03:39:21.002 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.024373156999999999, 2024-04-26T03:39:21.002 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:21.002 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:21.002 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "started", 2024-04-26T03:39:21.002 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:21.002 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:21.002 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:21.002 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 9 2024-04-26T03:39:21.002 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.002 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:21.002 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.002 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:21.002 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.481597+0000", 2024-04-26T03:39:21.002 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.003 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.003 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.003 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:21.003 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.481597+0000", 2024-04-26T03:39:21.003 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.003 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.003 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.003 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:21.003 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.481597+0000", 2024-04-26T03:39:21.003 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.5599999999999995e-07 2024-04-26T03:39:21.003 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.003 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.003 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:21.004 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.481610+0000", 2024-04-26T03:39:21.004 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.2330000000000001e-05 2024-04-26T03:39:21.004 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.004 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.004 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:21.004 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.481618+0000", 2024-04-26T03:39:21.004 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.7159999999999992e-06 2024-04-26T03:39:21.004 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.004 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.004 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:21.004 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.481624+0000", 2024-04-26T03:39:21.004 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.8989999999999998e-06 2024-04-26T03:39:21.004 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.005 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.005 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.005 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.484627+0000", 2024-04-26T03:39:21.005 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00300257 2024-04-26T03:39:21.005 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.005 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.005 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:21.005 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.484712+0000", 2024-04-26T03:39:21.005 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.4753000000000007e-05 2024-04-26T03:39:21.005 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.005 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.005 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.005 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.494632+0000", 2024-04-26T03:39:21.005 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0099199100000000005 2024-04-26T03:39:21.005 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.006 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.006 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:21.006 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.494899+0000", 2024-04-26T03:39:21.006 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00026711399999999999 2024-04-26T03:39:21.006 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.006 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.006 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:21.006 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.505970+0000", 2024-04-26T03:39:21.006 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.011071309 2024-04-26T03:39:21.006 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.006 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:21.006 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.006 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.007 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.007 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:24 3.7s0 3:e644bba7:::smithi09235267-8:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:21.007 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.485110+0000", 2024-04-26T03:39:21.007 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 29.513341310000001, 2024-04-26T03:39:21.007 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.015789734, 2024-04-26T03:39:21.007 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:21.007 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:21.007 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "started", 2024-04-26T03:39:21.007 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:21.007 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:21.007 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:21.007 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 24 2024-04-26T03:39:21.007 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.007 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:21.008 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.008 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:21.008 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.485110+0000", 2024-04-26T03:39:21.008 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.008 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.008 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.008 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:21.008 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.485110+0000", 2024-04-26T03:39:21.008 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.008 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.008 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.008 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:21.008 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.485110+0000", 2024-04-26T03:39:21.008 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.4099999999999999e-07 2024-04-26T03:39:21.008 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.009 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.009 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:21.009 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.485114+0000", 2024-04-26T03:39:21.009 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.2430000000000001e-06 2024-04-26T03:39:21.009 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.009 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.009 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:21.009 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.485120+0000", 2024-04-26T03:39:21.009 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.5810000000000002e-06 2024-04-26T03:39:21.009 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.009 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.009 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:21.009 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.485125+0000", 2024-04-26T03:39:21.009 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.0270000000000003e-06 2024-04-26T03:39:21.009 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.010 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.010 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.010 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.488721+0000", 2024-04-26T03:39:21.010 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.003595844 2024-04-26T03:39:21.010 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.010 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.010 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:21.010 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.488795+0000", 2024-04-26T03:39:21.010 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.3850000000000006e-05 2024-04-26T03:39:21.010 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.010 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.010 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.010 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.496954+0000", 2024-04-26T03:39:21.010 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0081585890000000008 2024-04-26T03:39:21.011 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.011 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.011 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:21.011 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.497038+0000", 2024-04-26T03:39:21.011 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.3934e-05 2024-04-26T03:39:21.011 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.011 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.011 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:21.011 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.500900+0000", 2024-04-26T03:39:21.011 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0038621250000000001 2024-04-26T03:39:21.011 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.011 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:21.011 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.011 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.012 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.012 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:37 3.bs0 3:d42e9299:::smithi09235267-13:head [append 0~212992,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:21.012 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.492880+0000", 2024-04-26T03:39:21.012 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 29.505571275000001, 2024-04-26T03:39:21.012 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.019386614999999999, 2024-04-26T03:39:21.012 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:21.012 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:21.012 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:21.012 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:21.012 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:21.012 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:21.012 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 37 2024-04-26T03:39:21.012 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.012 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:21.012 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.013 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:21.013 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.492880+0000", 2024-04-26T03:39:21.013 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.013 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.013 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.013 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:21.013 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.492880+0000", 2024-04-26T03:39:21.013 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.013 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.013 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.013 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:21.013 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.492898+0000", 2024-04-26T03:39:21.013 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.7759000000000001e-05 2024-04-26T03:39:21.013 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.013 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.014 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:21.014 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493142+0000", 2024-04-26T03:39:21.014 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00024467400000000001 2024-04-26T03:39:21.014 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.014 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.014 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:21.014 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493155+0000", 2024-04-26T03:39:21.014 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.2862e-05 2024-04-26T03:39:21.014 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.014 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.014 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:21.014 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493197+0000", 2024-04-26T03:39:21.014 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.2239e-05 2024-04-26T03:39:21.014 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.015 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.015 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.015 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495213+0000", 2024-04-26T03:39:21.015 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0020159090000000002 2024-04-26T03:39:21.015 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.015 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.015 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:21.015 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495342+0000", 2024-04-26T03:39:21.015 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000128214 2024-04-26T03:39:21.015 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.015 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.015 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:21.015 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.501605+0000", 2024-04-26T03:39:21.016 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0062634530000000004 2024-04-26T03:39:21.016 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.016 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.016 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:21.016 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.505099+0000", 2024-04-26T03:39:21.016 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0034938859999999999 2024-04-26T03:39:21.016 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.016 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.016 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:21.016 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.512085+0000", 2024-04-26T03:39:21.016 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0069863369999999996 2024-04-26T03:39:21.016 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.016 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.016 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:21.017 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.512267+0000", 2024-04-26T03:39:21.017 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000181282 2024-04-26T03:39:21.017 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.017 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:21.017 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.017 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.017 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.017 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:38 3.bs0 3:d42e9299:::smithi09235267-13:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:21.017 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.493215+0000", 2024-04-26T03:39:21.017 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 29.505236062000002, 2024-04-26T03:39:21.017 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.025985364, 2024-04-26T03:39:21.017 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:21.017 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:21.017 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:21.018 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:21.018 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:21.018 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:21.018 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 38 2024-04-26T03:39:21.018 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.018 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:21.018 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.018 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:21.018 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493215+0000", 2024-04-26T03:39:21.018 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.018 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.018 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.018 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:21.018 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493215+0000", 2024-04-26T03:39:21.019 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.019 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.019 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.019 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:21.019 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493216+0000", 2024-04-26T03:39:21.019 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.4700000000000001e-07 2024-04-26T03:39:21.019 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.019 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.019 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:21.019 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493261+0000", 2024-04-26T03:39:21.019 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.4604000000000002e-05 2024-04-26T03:39:21.019 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.019 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.019 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:21.020 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493271+0000", 2024-04-26T03:39:21.020 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0550999999999999e-05 2024-04-26T03:39:21.020 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.020 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.020 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:21.020 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493279+0000", 2024-04-26T03:39:21.020 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.3790000000000006e-06 2024-04-26T03:39:21.020 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.020 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.020 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.020 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.506027+0000", 2024-04-26T03:39:21.020 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.012747058 2024-04-26T03:39:21.020 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.020 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.020 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:21.021 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.506166+0000", 2024-04-26T03:39:21.021 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000139357 2024-04-26T03:39:21.021 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.021 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.021 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:21.021 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.506670+0000", 2024-04-26T03:39:21.021 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00050440100000000002 2024-04-26T03:39:21.021 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.021 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.021 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:21.021 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.509149+0000", 2024-04-26T03:39:21.021 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.002478839 2024-04-26T03:39:21.021 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.021 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.022 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:21.022 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.519099+0000", 2024-04-26T03:39:21.022 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0099501880000000004 2024-04-26T03:39:21.022 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.022 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.022 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:21.022 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.519201+0000", 2024-04-26T03:39:21.022 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00010124 2024-04-26T03:39:21.022 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.022 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:21.022 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.022 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.022 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.023 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:39 3.bs0 3:d42e9299:::smithi09235267-13:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:21.023 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.493292+0000", 2024-04-26T03:39:21.023 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 29.505159686999999, 2024-04-26T03:39:21.023 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.029045519999999998, 2024-04-26T03:39:21.023 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:21.023 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:21.023 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "started", 2024-04-26T03:39:21.023 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:21.023 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:21.023 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:21.023 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 39 2024-04-26T03:39:21.023 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.023 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:21.023 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.024 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:21.024 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493292+0000", 2024-04-26T03:39:21.024 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.024 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.024 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.024 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:21.024 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493292+0000", 2024-04-26T03:39:21.024 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.024 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.024 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.024 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:21.024 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493292+0000", 2024-04-26T03:39:21.024 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.3099999999999997e-07 2024-04-26T03:39:21.025 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.025 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.025 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:21.025 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493296+0000", 2024-04-26T03:39:21.025 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.0820000000000001e-06 2024-04-26T03:39:21.025 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.025 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.025 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:21.025 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493304+0000", 2024-04-26T03:39:21.025 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.0269999999999995e-06 2024-04-26T03:39:21.025 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.025 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.025 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:21.025 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493311+0000", 2024-04-26T03:39:21.026 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.1140000000000002e-06 2024-04-26T03:39:21.026 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.026 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.026 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.026 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.506997+0000", 2024-04-26T03:39:21.026 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.013686218999999999 2024-04-26T03:39:21.026 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.026 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.026 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:21.026 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.507073+0000", 2024-04-26T03:39:21.026 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.6766000000000003e-05 2024-04-26T03:39:21.026 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.026 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.026 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.026 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.519285+0000", 2024-04-26T03:39:21.027 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.012211324000000001 2024-04-26T03:39:21.027 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.027 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.027 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:21.027 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.519498+0000", 2024-04-26T03:39:21.027 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000213348 2024-04-26T03:39:21.027 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.027 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.029 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:21.029 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.522337+0000", 2024-04-26T03:39:21.030 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0028389090000000001 2024-04-26T03:39:21.030 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.030 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:21.030 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.030 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.030 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.030 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:128 3.bs0 3:d339032c:::smithi09235267-43:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:21.030 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.600125+0000", 2024-04-26T03:39:21.030 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 29.398326124, 2024-04-26T03:39:21.030 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.018721534000000001, 2024-04-26T03:39:21.030 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:21.030 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:21.030 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:21.031 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:21.031 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:21.031 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:21.031 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 128 2024-04-26T03:39:21.031 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.031 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:21.031 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.031 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:21.031 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.600125+0000", 2024-04-26T03:39:21.031 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.031 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.031 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.031 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:21.031 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.600125+0000", 2024-04-26T03:39:21.032 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.032 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.032 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.032 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:21.032 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.600126+0000", 2024-04-26T03:39:21.032 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.2100000000000002e-07 2024-04-26T03:39:21.032 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.032 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.032 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:21.032 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.600128+0000", 2024-04-26T03:39:21.032 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.5150000000000001e-06 2024-04-26T03:39:21.032 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.032 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.032 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:21.032 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.600133+0000", 2024-04-26T03:39:21.033 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.4240000000000001e-06 2024-04-26T03:39:21.033 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.033 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.033 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:21.033 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.600137+0000", 2024-04-26T03:39:21.033 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.4300000000000002e-06 2024-04-26T03:39:21.033 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.033 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.033 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.033 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.607830+0000", 2024-04-26T03:39:21.033 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0076931029999999997 2024-04-26T03:39:21.033 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.033 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.033 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:21.034 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.607888+0000", 2024-04-26T03:39:21.034 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.8260999999999997e-05 2024-04-26T03:39:21.034 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.034 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.034 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:21.034 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.608325+0000", 2024-04-26T03:39:21.034 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00043624700000000001 2024-04-26T03:39:21.034 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.034 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.034 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:21.034 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.608965+0000", 2024-04-26T03:39:21.034 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00064069400000000003 2024-04-26T03:39:21.035 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.035 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.035 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:21.035 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.618751+0000", 2024-04-26T03:39:21.035 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0097859209999999995 2024-04-26T03:39:21.035 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.035 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.035 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:21.035 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.618847+0000", 2024-04-26T03:39:21.035 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.5518000000000003e-05 2024-04-26T03:39:21.035 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.035 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:21.035 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.035 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.036 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.036 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:129 3.bs0 3:d339032c:::smithi09235267-43:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:21.036 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.600159+0000", 2024-04-26T03:39:21.036 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 29.398292124000001, 2024-04-26T03:39:21.036 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.020601199000000001, 2024-04-26T03:39:21.036 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:21.036 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:21.036 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "started", 2024-04-26T03:39:21.036 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:21.036 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:21.036 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:21.036 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 129 2024-04-26T03:39:21.036 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.036 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:21.036 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.037 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:21.037 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.600159+0000", 2024-04-26T03:39:21.037 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.037 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.037 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.037 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:21.037 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.600159+0000", 2024-04-26T03:39:21.037 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.037 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.037 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.037 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:21.037 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.600159+0000", 2024-04-26T03:39:21.037 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.2000000000000001e-07 2024-04-26T03:39:21.037 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.038 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.038 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:21.038 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.600162+0000", 2024-04-26T03:39:21.038 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.3070000000000001e-06 2024-04-26T03:39:21.038 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.038 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.038 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:21.038 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.600170+0000", 2024-04-26T03:39:21.038 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.568e-06 2024-04-26T03:39:21.038 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.038 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.038 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:21.038 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.600174+0000", 2024-04-26T03:39:21.039 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.241e-06 2024-04-26T03:39:21.039 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.039 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.039 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.039 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.608991+0000", 2024-04-26T03:39:21.039 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0088175800000000002 2024-04-26T03:39:21.039 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.039 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.039 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:21.039 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.609072+0000", 2024-04-26T03:39:21.039 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.1075000000000006e-05 2024-04-26T03:39:21.039 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.039 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.039 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.039 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.618897+0000", 2024-04-26T03:39:21.040 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0098250460000000005 2024-04-26T03:39:21.040 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.040 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.040 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:21.040 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.618974+0000", 2024-04-26T03:39:21.040 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.7075000000000003e-05 2024-04-26T03:39:21.040 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.040 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.040 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:21.040 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.620760+0000", 2024-04-26T03:39:21.040 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0017859869999999999 2024-04-26T03:39:21.040 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.040 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:21.040 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.041 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.041 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.041 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:361 3.bs0 3:d339032c:::smithi09235267-43:head [append 0~573440] snapc 7=[7,6,5] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-26T03:39:21.041 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:01.713821+0000", 2024-04-26T03:39:21.041 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 19.284630555, 2024-04-26T03:39:21.041 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.014165384, 2024-04-26T03:39:21.041 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:21.041 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:21.041 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:21.041 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:21.041 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:21.041 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:21.041 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 361 2024-04-26T03:39:21.041 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.041 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:21.042 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.042 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:21.042 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.713821+0000", 2024-04-26T03:39:21.042 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.042 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.042 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.042 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:21.042 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.713821+0000", 2024-04-26T03:39:21.042 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.042 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.042 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.042 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:21.042 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.713823+0000", 2024-04-26T03:39:21.042 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.841e-06 2024-04-26T03:39:21.043 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.043 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.043 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:21.043 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.714403+0000", 2024-04-26T03:39:21.043 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00058032400000000001 2024-04-26T03:39:21.043 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.043 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.043 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:21.043 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.714424+0000", 2024-04-26T03:39:21.043 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.1287000000000001e-05 2024-04-26T03:39:21.043 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.043 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.043 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:21.043 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.714437+0000", 2024-04-26T03:39:21.044 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.2792000000000001e-05 2024-04-26T03:39:21.044 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.044 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.044 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.044 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.714508+0000", 2024-04-26T03:39:21.044 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.1159000000000001e-05 2024-04-26T03:39:21.044 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.044 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.044 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:21.044 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.714588+0000", 2024-04-26T03:39:21.044 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.9994999999999996e-05 2024-04-26T03:39:21.044 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.044 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.045 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:21.045 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.715219+0000", 2024-04-26T03:39:21.045 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00063070400000000005 2024-04-26T03:39:21.045 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.045 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.045 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:21.045 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.727309+0000", 2024-04-26T03:39:21.045 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.012090638000000001 2024-04-26T03:39:21.045 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.045 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.045 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:21.045 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.727884+0000", 2024-04-26T03:39:21.045 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00057441199999999995 2024-04-26T03:39:21.046 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.046 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.046 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:21.046 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.727986+0000", 2024-04-26T03:39:21.046 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00010223200000000001 2024-04-26T03:39:21.046 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.046 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:21.046 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.046 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.046 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.046 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.13s5 56/55 ECSubWrite(tid=104, reqid=client.4556.0:604, at_version=56'47, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:39:21.046 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:11.774442+0000", 2024-04-26T03:39:21.046 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 9.2240090660000007, 2024-04-26T03:39:21.046 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.016284751, 2024-04-26T03:39:21.047 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:21.047 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:21.047 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:21.047 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:21.047 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.047 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:21.047 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.774442+0000", 2024-04-26T03:39:21.047 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.047 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.047 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.047 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:21.047 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.774442+0000", 2024-04-26T03:39:21.047 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.047 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.048 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.048 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:21.048 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.774444+0000", 2024-04-26T03:39:21.048 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.406e-06 2024-04-26T03:39:21.048 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.048 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.048 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:21.048 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.776475+0000", 2024-04-26T03:39:21.048 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.002030933 2024-04-26T03:39:21.048 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.048 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.048 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:21.048 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.776486+0000", 2024-04-26T03:39:21.048 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1221e-05 2024-04-26T03:39:21.049 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.049 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.049 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:21.049 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.776492+0000", 2024-04-26T03:39:21.049 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.9499999999999998e-06 2024-04-26T03:39:21.049 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.049 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.049 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.049 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.776534+0000", 2024-04-26T03:39:21.049 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.2404000000000003e-05 2024-04-26T03:39:21.049 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.049 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.049 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:21.049 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.776540+0000", 2024-04-26T03:39:21.050 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.1249999999999998e-06 2024-04-26T03:39:21.050 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.050 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.050 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:21.050 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.790691+0000", 2024-04-26T03:39:21.050 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.014150805000000001 2024-04-26T03:39:21.050 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.050 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.050 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:21.050 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.790727+0000", 2024-04-26T03:39:21.050 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.5907e-05 2024-04-26T03:39:21.050 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.050 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:21.051 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.051 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.051 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.051 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:684 3.bs0 3:d42e9299:::smithi09235267-13:head [append 0~573440] snapc f=[f,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-04-26T03:39:21.051 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:15.930550+0000", 2024-04-26T03:39:21.051 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 5.0679008579999998, 2024-04-26T03:39:21.051 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.020486684000000002, 2024-04-26T03:39:21.051 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:21.051 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:21.051 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:21.051 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:21.051 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:21.051 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:21.052 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 684 2024-04-26T03:39:21.052 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.052 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:21.052 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.052 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:21.052 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.930550+0000", 2024-04-26T03:39:21.052 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.052 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.052 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.052 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:21.052 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.930550+0000", 2024-04-26T03:39:21.052 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.052 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.052 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.052 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:21.053 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.930552+0000", 2024-04-26T03:39:21.053 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.4169999999999999e-06 2024-04-26T03:39:21.053 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.053 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.053 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:21.053 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.931191+0000", 2024-04-26T03:39:21.053 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00063884100000000002 2024-04-26T03:39:21.053 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.053 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.053 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:21.053 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.931205+0000", 2024-04-26T03:39:21.053 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.4521999999999999e-05 2024-04-26T03:39:21.053 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.053 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.054 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:21.054 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.931213+0000", 2024-04-26T03:39:21.054 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.8769999999999993e-06 2024-04-26T03:39:21.054 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.054 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.054 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.054 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.934870+0000", 2024-04-26T03:39:21.054 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0036572810000000001 2024-04-26T03:39:21.054 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.054 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.054 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:21.054 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.934956+0000", 2024-04-26T03:39:21.054 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.5513999999999995e-05 2024-04-26T03:39:21.054 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.055 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.055 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:21.055 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.935556+0000", 2024-04-26T03:39:21.055 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00060064099999999996 2024-04-26T03:39:21.055 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.055 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.055 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:21.055 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.950858+0000", 2024-04-26T03:39:21.055 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.015301726999999999 2024-04-26T03:39:21.055 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.055 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.055 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:21.055 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.950956+0000", 2024-04-26T03:39:21.056 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.7608000000000005e-05 2024-04-26T03:39:21.056 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.056 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.056 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:21.056 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.951037+0000", 2024-04-26T03:39:21.056 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.1255999999999996e-05 2024-04-26T03:39:21.056 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.056 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:21.056 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.056 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.056 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.056 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:685 3.bs0 3:d42e9299:::smithi09235267-13:head [append 0~688128] snapc f=[f,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-04-26T03:39:21.056 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:15.934342+0000", 2024-04-26T03:39:21.056 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 5.0641096020000003, 2024-04-26T03:39:21.056 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.017104579000000002, 2024-04-26T03:39:21.057 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:21.057 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:21.057 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:21.057 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:21.057 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:21.057 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:21.057 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 685 2024-04-26T03:39:21.057 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.057 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:21.057 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.057 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:21.057 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.934342+0000", 2024-04-26T03:39:21.057 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.057 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.058 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.058 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:21.058 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.934342+0000", 2024-04-26T03:39:21.058 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.058 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.058 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.058 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:21.058 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.934343+0000", 2024-04-26T03:39:21.058 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.0800000000000003e-07 2024-04-26T03:39:21.058 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.058 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.058 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:21.058 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.934992+0000", 2024-04-26T03:39:21.059 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00064962500000000003 2024-04-26T03:39:21.059 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.059 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.059 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:21.059 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.935001+0000", 2024-04-26T03:39:21.059 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.0059999999999998e-06 2024-04-26T03:39:21.059 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.059 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.059 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:21.059 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.935006+0000", 2024-04-26T03:39:21.059 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.9740000000000001e-06 2024-04-26T03:39:21.059 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.059 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.059 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.060 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.939492+0000", 2024-04-26T03:39:21.060 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0044860309999999997 2024-04-26T03:39:21.060 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.060 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.060 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:21.060 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.939600+0000", 2024-04-26T03:39:21.060 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000108044 2024-04-26T03:39:21.060 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.060 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.060 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:21.060 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.940601+0000", 2024-04-26T03:39:21.060 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0010011180000000001 2024-04-26T03:39:21.060 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.060 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.061 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:21.061 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.951051+0000", 2024-04-26T03:39:21.061 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.010449738 2024-04-26T03:39:21.061 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.061 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.061 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:21.061 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.951390+0000", 2024-04-26T03:39:21.061 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00033857100000000002 2024-04-26T03:39:21.061 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.061 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.061 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:21.061 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.951446+0000", 2024-04-26T03:39:21.061 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.6564000000000003e-05 2024-04-26T03:39:21.061 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.062 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:21.062 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.062 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.062 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.062 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:686 3.bs0 3:d42e9299:::smithi09235267-13:head [append 0~294912] snapc f=[f,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-04-26T03:39:21.062 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:15.935994+0000", 2024-04-26T03:39:21.062 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 5.0624572289999996, 2024-04-26T03:39:21.062 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.016832585000000001, 2024-04-26T03:39:21.062 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:21.062 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:21.062 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:21.062 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:21.062 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:21.063 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:21.063 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 686 2024-04-26T03:39:21.063 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.063 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:21.063 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.063 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:21.063 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.935994+0000", 2024-04-26T03:39:21.063 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.063 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.063 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.063 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:21.063 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.935994+0000", 2024-04-26T03:39:21.063 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.063 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.063 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.064 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:21.064 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.935995+0000", 2024-04-26T03:39:21.064 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.4130000000000001e-06 2024-04-26T03:39:21.064 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.064 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.064 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:21.064 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.936361+0000", 2024-04-26T03:39:21.064 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00036540299999999998 2024-04-26T03:39:21.064 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.064 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.064 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:21.064 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.936376+0000", 2024-04-26T03:39:21.064 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.4983e-05 2024-04-26T03:39:21.064 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.065 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.065 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:21.065 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.936384+0000", 2024-04-26T03:39:21.065 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.2530000000000004e-06 2024-04-26T03:39:21.065 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.065 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.065 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.065 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.947040+0000", 2024-04-26T03:39:21.065 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.010655825000000001 2024-04-26T03:39:21.065 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.065 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.065 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:21.065 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.947149+0000", 2024-04-26T03:39:21.065 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00010880299999999999 2024-04-26T03:39:21.066 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.066 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.066 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:21.066 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.947876+0000", 2024-04-26T03:39:21.066 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00072771100000000003 2024-04-26T03:39:21.066 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.066 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.066 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:21.066 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.952293+0000", 2024-04-26T03:39:21.066 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0044169999999999999 2024-04-26T03:39:21.066 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.066 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.066 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:21.067 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.952760+0000", 2024-04-26T03:39:21.067 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00046631800000000002 2024-04-26T03:39:21.067 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.067 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.067 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:21.067 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.952827+0000", 2024-04-26T03:39:21.067 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.6876000000000001e-05 2024-04-26T03:39:21.067 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.067 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:21.067 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.067 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.067 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.067 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:687 3.bs0 3:d42e9299:::smithi09235267-13:head [setxattr _header (57) in=7b] snapc f=[f,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-04-26T03:39:21.067 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:15.936397+0000", 2024-04-26T03:39:21.067 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 5.0620537700000003, 2024-04-26T03:39:21.068 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.018657155000000002, 2024-04-26T03:39:21.068 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:21.068 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:21.068 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:21.068 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:21.068 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:21.068 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:21.068 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 687 2024-04-26T03:39:21.068 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.068 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:21.068 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.068 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:21.068 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.936397+0000", 2024-04-26T03:39:21.068 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.068 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.069 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.069 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:21.069 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.936397+0000", 2024-04-26T03:39:21.069 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.069 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.069 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.069 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:21.069 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.936398+0000", 2024-04-26T03:39:21.069 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.5300000000000004e-07 2024-04-26T03:39:21.069 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.069 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.069 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:21.069 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.936402+0000", 2024-04-26T03:39:21.069 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.8580000000000002e-06 2024-04-26T03:39:21.070 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.070 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.070 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:21.070 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.936412+0000", 2024-04-26T03:39:21.070 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.5340000000000005e-06 2024-04-26T03:39:21.070 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.070 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.070 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:21.070 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.936418+0000", 2024-04-26T03:39:21.070 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.5350000000000001e-06 2024-04-26T03:39:21.070 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.070 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.070 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.070 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.951554+0000", 2024-04-26T03:39:21.071 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.015135964 2024-04-26T03:39:21.071 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.071 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.071 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:21.071 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.951659+0000", 2024-04-26T03:39:21.071 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000105376 2024-04-26T03:39:21.071 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.071 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.071 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:21.071 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.952057+0000", 2024-04-26T03:39:21.071 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000397258 2024-04-26T03:39:21.071 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.071 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.071 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:21.072 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.952942+0000", 2024-04-26T03:39:21.072 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00088508900000000004 2024-04-26T03:39:21.072 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.072 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.072 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:21.072 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.954966+0000", 2024-04-26T03:39:21.072 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0020244669999999998 2024-04-26T03:39:21.072 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.072 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.072 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:21.072 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.955055+0000", 2024-04-26T03:39:21.072 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.8420999999999999e-05 2024-04-26T03:39:21.072 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.072 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:21.073 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.073 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.073 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.073 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:688 3.bs0 3:d42e9299:::smithi09235267-13:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e60)", 2024-04-26T03:39:21.073 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:15.936428+0000", 2024-04-26T03:39:21.073 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 5.0620232400000003, 2024-04-26T03:39:21.073 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.020098925, 2024-04-26T03:39:21.073 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:21.073 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:21.073 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "started", 2024-04-26T03:39:21.073 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:21.073 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:21.073 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:21.073 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 688 2024-04-26T03:39:21.074 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.074 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:21.074 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.074 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:21.074 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.936428+0000", 2024-04-26T03:39:21.074 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.074 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.074 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.074 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:21.074 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.936428+0000", 2024-04-26T03:39:21.074 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.074 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.074 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.074 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:21.075 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.936429+0000", 2024-04-26T03:39:21.075 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.44e-07 2024-04-26T03:39:21.075 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.075 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.075 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:21.075 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.936431+0000", 2024-04-26T03:39:21.075 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.621e-06 2024-04-26T03:39:21.075 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.075 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.075 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:21.075 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.936440+0000", 2024-04-26T03:39:21.075 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.1379999999999996e-06 2024-04-26T03:39:21.075 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.075 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.076 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:21.076 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.936446+0000", 2024-04-26T03:39:21.076 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.0990000000000004e-06 2024-04-26T03:39:21.076 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.076 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.076 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.076 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.952325+0000", 2024-04-26T03:39:21.076 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.015878541999999999 2024-04-26T03:39:21.076 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.076 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.076 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:21.076 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.952405+0000", 2024-04-26T03:39:21.076 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.0042000000000004e-05 2024-04-26T03:39:21.076 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.077 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.077 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.077 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.955424+0000", 2024-04-26T03:39:21.077 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0030187619999999999 2024-04-26T03:39:21.077 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.077 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.077 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:21.077 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.955503+0000", 2024-04-26T03:39:21.077 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.8850000000000006e-05 2024-04-26T03:39:21.077 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.077 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.077 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:21.077 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.956527+0000", 2024-04-26T03:39:21.077 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.001024327 2024-04-26T03:39:21.078 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.078 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:21.078 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.078 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.078 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.078 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.10s3 60/48 ECSubWrite(tid=272, reqid=client.4556.0:703, at_version=60'20, trim_to=0'0, roll_forward_to=60'18))", 2024-04-26T03:39:21.078 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:15.958315+0000", 2024-04-26T03:39:21.078 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 5.0401364700000002, 2024-04-26T03:39:21.078 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.014611541, 2024-04-26T03:39:21.078 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:21.078 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:21.078 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:21.078 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:21.078 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.079 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:21.079 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.958315+0000", 2024-04-26T03:39:21.079 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.079 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.079 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.079 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:21.079 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.958315+0000", 2024-04-26T03:39:21.079 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.079 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.079 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.079 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:21.079 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.958315+0000", 2024-04-26T03:39:21.079 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.3099999999999997e-07 2024-04-26T03:39:21.079 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.080 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.080 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:21.080 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.959300+0000", 2024-04-26T03:39:21.080 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00098483399999999997 2024-04-26T03:39:21.080 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.080 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.080 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:21.080 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.959313+0000", 2024-04-26T03:39:21.080 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.2662e-05 2024-04-26T03:39:21.080 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.080 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.080 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:21.080 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.959321+0000", 2024-04-26T03:39:21.080 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.1799999999999996e-06 2024-04-26T03:39:21.081 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.081 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.081 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.081 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.959385+0000", 2024-04-26T03:39:21.081 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.4199e-05 2024-04-26T03:39:21.081 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.081 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.081 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:21.081 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.959396+0000", 2024-04-26T03:39:21.081 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0277e-05 2024-04-26T03:39:21.081 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.081 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.081 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:21.081 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.972883+0000", 2024-04-26T03:39:21.081 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.013487025 2024-04-26T03:39:21.082 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.082 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.082 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:21.082 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.972926+0000", 2024-04-26T03:39:21.082 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.3733000000000003e-05 2024-04-26T03:39:21.082 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.082 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:21.082 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.082 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.082 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.082 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:746 3.bs0 3:d42e9299:::smithi09235267-13:head [append 0~573440] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:21.082 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:20.717432+0000", 2024-04-26T03:39:21.082 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 0.28101910200000002, 2024-04-26T03:39:21.082 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.020989714, 2024-04-26T03:39:21.083 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:21.083 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:21.083 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:21.083 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:21.083 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:21.083 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:21.083 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 746 2024-04-26T03:39:21.083 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.083 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:21.083 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.083 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:21.083 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.717432+0000", 2024-04-26T03:39:21.083 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.083 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.084 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.084 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:21.084 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.717432+0000", 2024-04-26T03:39:21.084 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.084 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.084 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.084 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:21.084 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.717434+0000", 2024-04-26T03:39:21.084 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.6050000000000001e-06 2024-04-26T03:39:21.084 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.084 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.084 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:21.084 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.717989+0000", 2024-04-26T03:39:21.084 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00055509299999999999 2024-04-26T03:39:21.085 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.085 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.085 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:21.085 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.718007+0000", 2024-04-26T03:39:21.085 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.8349999999999999e-05 2024-04-26T03:39:21.085 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.085 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.085 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:21.085 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.718018+0000", 2024-04-26T03:39:21.085 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.082e-05 2024-04-26T03:39:21.085 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.085 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.085 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.085 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.722909+0000", 2024-04-26T03:39:21.086 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0048909770000000003 2024-04-26T03:39:21.086 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.086 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.086 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:21.086 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.723028+0000", 2024-04-26T03:39:21.086 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00011944599999999999 2024-04-26T03:39:21.086 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.086 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.086 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:21.086 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.723782+0000", 2024-04-26T03:39:21.086 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00075392899999999997 2024-04-26T03:39:21.086 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.086 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.086 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:21.087 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.737780+0000", 2024-04-26T03:39:21.087 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.013997921999999999 2024-04-26T03:39:21.087 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.087 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.087 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:21.087 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.738324+0000", 2024-04-26T03:39:21.087 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00054407299999999995 2024-04-26T03:39:21.087 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.087 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.087 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:21.087 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.738422+0000", 2024-04-26T03:39:21.087 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.7498999999999996e-05 2024-04-26T03:39:21.087 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.087 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:21.087 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.088 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.088 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.088 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:747 3.bs0 3:d42e9299:::smithi09235267-13:head [append 0~753664] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:21.088 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:20.721726+0000", 2024-04-26T03:39:21.088 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 0.27672562699999997, 2024-04-26T03:39:21.088 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.018181914, 2024-04-26T03:39:21.088 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:21.088 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:21.088 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:21.088 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:21.088 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:21.088 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:21.088 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 747 2024-04-26T03:39:21.088 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.089 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:21.089 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.089 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:21.089 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.721726+0000", 2024-04-26T03:39:21.089 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.089 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.089 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.089 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:21.089 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.721726+0000", 2024-04-26T03:39:21.089 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.089 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.089 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.089 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:21.089 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.721727+0000", 2024-04-26T03:39:21.090 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.762e-06 2024-04-26T03:39:21.090 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.090 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.090 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:21.090 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.722450+0000", 2024-04-26T03:39:21.090 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00072252799999999995 2024-04-26T03:39:21.090 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.090 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.090 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:21.090 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.722459+0000", 2024-04-26T03:39:21.090 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.1759999999999999e-06 2024-04-26T03:39:21.090 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.090 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.090 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:21.091 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.722464+0000", 2024-04-26T03:39:21.091 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.9239999999999998e-06 2024-04-26T03:39:21.091 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.091 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.091 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.091 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.727880+0000", 2024-04-26T03:39:21.091 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0054160809999999997 2024-04-26T03:39:21.091 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.091 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.091 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:21.091 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.727949+0000", 2024-04-26T03:39:21.091 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.9286e-05 2024-04-26T03:39:21.091 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.091 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.092 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:21.092 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.728724+0000", 2024-04-26T03:39:21.092 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00077420600000000005 2024-04-26T03:39:21.092 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.092 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.092 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:21.092 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.737782+0000", 2024-04-26T03:39:21.092 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0090582360000000008 2024-04-26T03:39:21.092 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.092 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.092 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:21.092 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.739827+0000", 2024-04-26T03:39:21.092 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0020447070000000002 2024-04-26T03:39:21.092 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.093 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.093 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:21.093 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.739908+0000", 2024-04-26T03:39:21.093 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.1007999999999994e-05 2024-04-26T03:39:21.093 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.093 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:21.093 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.093 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.093 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.093 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:748 3.bs0 3:d42e9299:::smithi09235267-13:head [append 0~229376] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:21.093 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:20.723036+0000", 2024-04-26T03:39:21.093 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 0.27541482099999998, 2024-04-26T03:39:21.093 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.018068820999999999, 2024-04-26T03:39:21.093 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:21.094 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:21.094 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:21.094 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:21.094 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:21.094 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:21.094 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 748 2024-04-26T03:39:21.094 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.094 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:21.094 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.094 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:21.094 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.723036+0000", 2024-04-26T03:39:21.094 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.095 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.095 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.095 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:21.095 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.723036+0000", 2024-04-26T03:39:21.095 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.095 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.095 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.095 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:21.095 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.723037+0000", 2024-04-26T03:39:21.095 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.6099999999999999e-07 2024-04-26T03:39:21.095 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.095 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.095 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:21.095 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.723287+0000", 2024-04-26T03:39:21.096 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00025004699999999999 2024-04-26T03:39:21.096 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.096 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.096 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:21.096 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.723297+0000", 2024-04-26T03:39:21.096 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.3540000000000006e-06 2024-04-26T03:39:21.096 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.096 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.096 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:21.096 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.723306+0000", 2024-04-26T03:39:21.096 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.8729999999999996e-06 2024-04-26T03:39:21.096 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.096 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.096 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.097 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.735103+0000", 2024-04-26T03:39:21.097 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.011797813000000001 2024-04-26T03:39:21.097 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.097 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.097 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:21.097 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.735219+0000", 2024-04-26T03:39:21.097 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000115074 2024-04-26T03:39:21.097 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.097 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.097 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:21.097 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.735876+0000", 2024-04-26T03:39:21.097 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00065787099999999998 2024-04-26T03:39:21.097 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.097 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.098 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:21.098 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.740018+0000", 2024-04-26T03:39:21.098 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0041418519999999997 2024-04-26T03:39:21.098 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.098 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.098 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:21.098 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.741049+0000", 2024-04-26T03:39:21.098 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0010308509999999999 2024-04-26T03:39:21.098 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.098 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.098 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:21.098 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.741105+0000", 2024-04-26T03:39:21.098 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.6125000000000002e-05 2024-04-26T03:39:21.098 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.099 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:21.099 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.099 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.099 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.099 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:749 3.bs0 3:d42e9299:::smithi09235267-13:head [setxattr _header (57) in=7b] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:21.099 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:20.723321+0000", 2024-04-26T03:39:21.099 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 0.27513033799999997, 2024-04-26T03:39:21.099 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.018015394000000001, 2024-04-26T03:39:21.099 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:21.099 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:21.099 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:21.099 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:21.099 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:21.099 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:21.100 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 749 2024-04-26T03:39:21.100 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.100 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:21.100 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.100 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:21.100 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.723321+0000", 2024-04-26T03:39:21.100 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.100 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.100 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.100 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:21.100 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.723321+0000", 2024-04-26T03:39:21.100 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.100 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.100 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.101 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:21.101 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.723322+0000", 2024-04-26T03:39:21.101 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.3500000000000005e-07 2024-04-26T03:39:21.101 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.101 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.101 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:21.101 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.723326+0000", 2024-04-26T03:39:21.101 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.8700000000000002e-06 2024-04-26T03:39:21.101 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.101 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.101 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:21.101 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.723333+0000", 2024-04-26T03:39:21.101 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.3529999999999997e-06 2024-04-26T03:39:21.101 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.102 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.102 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:21.102 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.723339+0000", 2024-04-26T03:39:21.102 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.7490000000000004e-06 2024-04-26T03:39:21.102 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.102 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.102 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.102 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.738816+0000", 2024-04-26T03:39:21.102 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.015476867 2024-04-26T03:39:21.102 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.102 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.102 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:21.102 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.738908+0000", 2024-04-26T03:39:21.102 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.2083999999999995e-05 2024-04-26T03:39:21.103 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.103 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.103 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:21.103 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.739356+0000", 2024-04-26T03:39:21.103 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00044818599999999999 2024-04-26T03:39:21.103 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.103 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.103 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:21.103 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.740366+0000", 2024-04-26T03:39:21.103 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.001010101 2024-04-26T03:39:21.103 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.103 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.103 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:21.103 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.741225+0000", 2024-04-26T03:39:21.104 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00085920799999999998 2024-04-26T03:39:21.104 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.104 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.104 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:21.104 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.741336+0000", 2024-04-26T03:39:21.104 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000111141 2024-04-26T03:39:21.104 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.104 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:21.104 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.104 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.104 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.104 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:750 3.bs0 3:d42e9299:::smithi09235267-13:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:21.104 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:20.723347+0000", 2024-04-26T03:39:21.104 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 0.27510414399999999, 2024-04-26T03:39:21.105 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.019487478999999999, 2024-04-26T03:39:21.105 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:21.105 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:21.105 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "started", 2024-04-26T03:39:21.105 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:21.105 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:21.105 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:21.105 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 750 2024-04-26T03:39:21.105 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.105 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:21.105 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.105 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:21.105 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.723347+0000", 2024-04-26T03:39:21.105 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.106 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.106 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.106 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:21.106 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.723347+0000", 2024-04-26T03:39:21.106 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.106 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.106 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.106 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:21.106 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.723348+0000", 2024-04-26T03:39:21.106 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.9900000000000001e-07 2024-04-26T03:39:21.106 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.106 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.106 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:21.106 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.723350+0000", 2024-04-26T03:39:21.107 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.6800000000000002e-06 2024-04-26T03:39:21.107 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.107 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.107 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:21.107 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.723357+0000", 2024-04-26T03:39:21.107 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.4609999999999996e-06 2024-04-26T03:39:21.107 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.107 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.107 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:21.107 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.723360+0000", 2024-04-26T03:39:21.107 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.455e-06 2024-04-26T03:39:21.107 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.107 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.107 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.108 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.739606+0000", 2024-04-26T03:39:21.108 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.016245622000000001 2024-04-26T03:39:21.108 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.108 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.108 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:21.108 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.739704+0000", 2024-04-26T03:39:21.108 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.8397e-05 2024-04-26T03:39:21.108 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.108 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.108 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.108 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.741743+0000", 2024-04-26T03:39:21.108 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.002038593 2024-04-26T03:39:21.108 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.108 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.108 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:21.109 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.741818+0000", 2024-04-26T03:39:21.109 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.5142999999999995e-05 2024-04-26T03:39:21.109 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.109 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.109 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:21.109 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.742835+0000", 2024-04-26T03:39:21.109 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.001016629 2024-04-26T03:39:21.109 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.109 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:21.109 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.109 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.109 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:21.109 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:39:21.111 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-04-26T03:39:21.110+0000 7fdc2d21c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31819) UID: 0 2024-04-26T03:39:21.111 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-04-26T03:39:21.212 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:39:21.211+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:39:21.259 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:39:21.259 INFO:teuthology.orchestra.run.smithi028.stdout: "ops": [], 2024-04-26T03:39:21.259 INFO:teuthology.orchestra.run.smithi028.stdout: "num_ops": 0 2024-04-26T03:39:21.260 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:39:21.268 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-04-26T03:39:21.313 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-04-26T03:39:21.312+0000 7fdc2d21c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31819) UID: 0 2024-04-26T03:39:21.375 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:39:21.375 INFO:teuthology.orchestra.run.smithi028.stdout: "ops": [], 2024-04-26T03:39:21.375 INFO:teuthology.orchestra.run.smithi028.stdout: "complaint_time": 30, 2024-04-26T03:39:21.375 INFO:teuthology.orchestra.run.smithi028.stdout: "num_blocked_ops": 0 2024-04-26T03:39:21.375 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:39:21.383 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-04-26T03:39:21.413 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:39:21.412+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:39:21.492 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:39:21.493 INFO:teuthology.orchestra.run.smithi028.stdout: "size": 20, 2024-04-26T03:39:21.493 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 600, 2024-04-26T03:39:21.493 INFO:teuthology.orchestra.run.smithi028.stdout: "ops": [ 2024-04-26T03:39:21.493 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.493 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:34 3.4s0 3:2ee2abb1:::smithi09235267-12:head [append 0~196608,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:21.493 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.494587+0000", 2024-04-26T03:39:21.493 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 29.995531397000001, 2024-04-26T03:39:21.493 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.014012205, 2024-04-26T03:39:21.493 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:21.493 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:21.493 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:21.493 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:21.493 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:21.494 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:21.494 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 34 2024-04-26T03:39:21.494 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.494 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:21.494 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.494 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:21.494 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.494587+0000", 2024-04-26T03:39:21.494 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.494 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.494 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.494 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:21.494 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.494587+0000", 2024-04-26T03:39:21.494 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.495 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.495 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.495 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:21.495 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.494589+0000", 2024-04-26T03:39:21.495 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.3530000000000002e-06 2024-04-26T03:39:21.495 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.495 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.495 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:21.495 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.494944+0000", 2024-04-26T03:39:21.495 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00035535200000000002 2024-04-26T03:39:21.495 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.495 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.495 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:21.495 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.494963+0000", 2024-04-26T03:39:21.495 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.8745000000000001e-05 2024-04-26T03:39:21.495 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.496 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.496 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:21.496 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.494972+0000", 2024-04-26T03:39:21.496 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.0629999999999995e-06 2024-04-26T03:39:21.496 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.496 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.496 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.496 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495054+0000", 2024-04-26T03:39:21.496 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.1810000000000004e-05 2024-04-26T03:39:21.496 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.496 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.496 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:21.496 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495184+0000", 2024-04-26T03:39:21.496 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00012965399999999999 2024-04-26T03:39:21.497 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.497 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.497 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:21.497 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495943+0000", 2024-04-26T03:39:21.497 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00075938199999999996 2024-04-26T03:39:21.497 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.497 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.497 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:21.497 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.500314+0000", 2024-04-26T03:39:21.497 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0043711669999999996 2024-04-26T03:39:21.497 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.497 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.497 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:21.497 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.508524+0000", 2024-04-26T03:39:21.497 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0082094520000000008 2024-04-26T03:39:21.498 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.498 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.498 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:21.498 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.508599+0000", 2024-04-26T03:39:21.498 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.5227000000000003e-05 2024-04-26T03:39:21.498 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.498 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:21.498 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.498 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.498 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.498 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:36 3.4s0 3:2ee2abb1:::smithi09235267-12:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:21.498 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.495035+0000", 2024-04-26T03:39:21.498 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 29.995082573000001, 2024-04-26T03:39:21.499 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.018906236, 2024-04-26T03:39:21.499 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:21.499 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:21.499 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "started", 2024-04-26T03:39:21.499 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:21.499 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:21.499 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:21.499 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 36 2024-04-26T03:39:21.499 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.499 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:21.499 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.499 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:21.499 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495035+0000", 2024-04-26T03:39:21.499 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.499 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.500 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.500 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:21.500 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495035+0000", 2024-04-26T03:39:21.500 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.500 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.500 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.500 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:21.500 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495036+0000", 2024-04-26T03:39:21.500 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.5199999999999997e-07 2024-04-26T03:39:21.500 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.500 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.500 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:21.500 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495039+0000", 2024-04-26T03:39:21.500 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.5770000000000001e-06 2024-04-26T03:39:21.501 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.501 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.501 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:21.501 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495045+0000", 2024-04-26T03:39:21.501 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.8199999999999999e-06 2024-04-26T03:39:21.501 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.501 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.501 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:21.501 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495050+0000", 2024-04-26T03:39:21.501 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.8520000000000003e-06 2024-04-26T03:39:21.501 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.501 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.501 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.501 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.500349+0000", 2024-04-26T03:39:21.501 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0052990069999999997 2024-04-26T03:39:21.502 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.502 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.502 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:21.502 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.500436+0000", 2024-04-26T03:39:21.502 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.6327000000000002e-05 2024-04-26T03:39:21.502 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.502 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.502 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.502 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.508818+0000", 2024-04-26T03:39:21.502 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0083819900000000006 2024-04-26T03:39:21.502 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.502 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.502 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:21.502 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.508897+0000", 2024-04-26T03:39:21.503 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.9580999999999996e-05 2024-04-26T03:39:21.503 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.503 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.503 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:21.503 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.513942+0000", 2024-04-26T03:39:21.503 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0050445300000000002 2024-04-26T03:39:21.503 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.503 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:21.503 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.503 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.503 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.503 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:101 3.4s0 3:2869fefd:::smithi09235267-34:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:21.503 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.559280+0000", 2024-04-26T03:39:21.503 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 29.930837918000002, 2024-04-26T03:39:21.504 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.015441082, 2024-04-26T03:39:21.504 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:21.504 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:21.504 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:21.504 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:21.504 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:21.504 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:21.504 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 101 2024-04-26T03:39:21.504 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.504 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:21.504 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.504 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:21.504 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.559280+0000", 2024-04-26T03:39:21.504 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.504 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.505 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.505 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:21.505 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.559280+0000", 2024-04-26T03:39:21.505 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.505 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.505 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.505 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:21.505 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.559281+0000", 2024-04-26T03:39:21.505 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.4600000000000004e-07 2024-04-26T03:39:21.505 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.505 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.505 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:21.505 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.559285+0000", 2024-04-26T03:39:21.505 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.5129999999999998e-06 2024-04-26T03:39:21.505 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.506 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.506 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:21.506 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.559294+0000", 2024-04-26T03:39:21.506 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.0059999999999998e-06 2024-04-26T03:39:21.506 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.506 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.506 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:21.506 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.559303+0000", 2024-04-26T03:39:21.506 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.1999999999999994e-06 2024-04-26T03:39:21.506 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.506 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.506 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.506 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.564299+0000", 2024-04-26T03:39:21.506 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0049963150000000003 2024-04-26T03:39:21.507 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.507 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.507 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:21.507 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.564369+0000", 2024-04-26T03:39:21.507 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.0238000000000001e-05 2024-04-26T03:39:21.507 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.507 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.507 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:21.507 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.564677+0000", 2024-04-26T03:39:21.507 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00030746099999999998 2024-04-26T03:39:21.507 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.507 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.507 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:21.507 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.565235+0000", 2024-04-26T03:39:21.508 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00055854500000000003 2024-04-26T03:39:21.508 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.508 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.508 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:21.508 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.574673+0000", 2024-04-26T03:39:21.508 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0094382830000000004 2024-04-26T03:39:21.508 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.508 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.508 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:21.508 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.574721+0000", 2024-04-26T03:39:21.508 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.7775000000000003e-05 2024-04-26T03:39:21.508 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.508 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:21.508 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.508 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.509 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.509 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:102 3.4s0 3:2869fefd:::smithi09235267-34:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:21.509 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.559325+0000", 2024-04-26T03:39:21.509 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 29.930793239, 2024-04-26T03:39:21.509 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.017964909000000001, 2024-04-26T03:39:21.509 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:21.509 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:21.509 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "started", 2024-04-26T03:39:21.509 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:21.509 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:21.509 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:21.509 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 102 2024-04-26T03:39:21.509 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.509 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:21.509 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.510 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:21.510 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.559325+0000", 2024-04-26T03:39:21.510 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.510 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.510 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.510 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:21.510 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.559325+0000", 2024-04-26T03:39:21.510 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.510 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.510 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.510 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:21.510 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.559325+0000", 2024-04-26T03:39:21.510 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.2900000000000003e-07 2024-04-26T03:39:21.511 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.511 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.511 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:21.511 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.559330+0000", 2024-04-26T03:39:21.511 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.7029999999999997e-06 2024-04-26T03:39:21.511 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.511 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.511 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:21.511 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.559339+0000", 2024-04-26T03:39:21.511 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.8969999999999997e-06 2024-04-26T03:39:21.511 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.511 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.511 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:21.511 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.559346+0000", 2024-04-26T03:39:21.511 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.8560000000000001e-06 2024-04-26T03:39:21.512 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.512 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.512 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.512 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.564881+0000", 2024-04-26T03:39:21.512 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0055355830000000002 2024-04-26T03:39:21.512 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.512 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.512 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:21.512 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.564965+0000", 2024-04-26T03:39:21.512 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.3248e-05 2024-04-26T03:39:21.512 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.512 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.832 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.832 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.574764+0000", 2024-04-26T03:39:21.833 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0097988769999999992 2024-04-26T03:39:21.833 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.833 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.833 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:21.833 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.574811+0000", 2024-04-26T03:39:21.833 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.7466999999999998e-05 2024-04-26T03:39:21.833 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.833 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.833 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:21.833 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.577290+0000", 2024-04-26T03:39:21.833 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0024786489999999999 2024-04-26T03:39:21.833 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.833 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:21.833 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.834 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.834 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.834 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:262 3.4s0 3:2d740be2:::smithi09235267-28:head [append 0~720896] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T03:39:21.834 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:56.518030+0000", 2024-04-26T03:39:21.834 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 24.972087813999998, 2024-04-26T03:39:21.834 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.018690736999999999, 2024-04-26T03:39:21.834 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:21.834 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:21.834 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:21.834 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:21.834 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:21.834 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:21.834 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 262 2024-04-26T03:39:21.834 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.835 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:21.835 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.835 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:21.835 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.518030+0000", 2024-04-26T03:39:21.835 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.835 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.835 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.835 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:21.835 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.518030+0000", 2024-04-26T03:39:21.835 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.835 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.835 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.835 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:21.835 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.518039+0000", 2024-04-26T03:39:21.836 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.3999999999999992e-06 2024-04-26T03:39:21.836 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.836 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.836 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:21.836 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.519077+0000", 2024-04-26T03:39:21.836 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.001038593 2024-04-26T03:39:21.836 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.836 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.836 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:21.836 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.519089+0000", 2024-04-26T03:39:21.836 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.2092e-05 2024-04-26T03:39:21.836 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.836 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.836 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:21.837 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.519097+0000", 2024-04-26T03:39:21.837 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.7680000000000008e-06 2024-04-26T03:39:21.837 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.837 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.837 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.837 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.519162+0000", 2024-04-26T03:39:21.837 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.4517999999999995e-05 2024-04-26T03:39:21.837 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.837 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.837 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:21.837 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.519303+0000", 2024-04-26T03:39:21.837 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000141602 2024-04-26T03:39:21.837 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.837 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.837 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:21.838 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.520994+0000", 2024-04-26T03:39:21.838 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0016909780000000001 2024-04-26T03:39:21.838 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.838 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.838 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:21.838 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.531345+0000", 2024-04-26T03:39:21.838 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.010350471999999999 2024-04-26T03:39:21.838 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.838 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.838 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:21.838 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.536659+0000", 2024-04-26T03:39:21.838 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0053146649999999997 2024-04-26T03:39:21.838 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.839 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.839 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:21.839 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.536721+0000", 2024-04-26T03:39:21.839 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.1649000000000006e-05 2024-04-26T03:39:21.839 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.839 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:21.839 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.839 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.839 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.839 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:263 3.4s0 3:2d740be2:::smithi09235267-28:head [append 0~376832] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T03:39:21.839 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:56.520937+0000", 2024-04-26T03:39:21.839 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 24.969181296999999, 2024-04-26T03:39:21.839 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.016100099999999999, 2024-04-26T03:39:21.839 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:21.840 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:21.840 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:21.840 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:21.840 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:21.840 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:21.840 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 263 2024-04-26T03:39:21.840 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.840 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:21.840 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.840 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:21.840 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.520937+0000", 2024-04-26T03:39:21.840 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.840 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.840 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.840 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:21.841 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.520937+0000", 2024-04-26T03:39:21.841 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.841 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.841 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.841 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:21.841 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.520939+0000", 2024-04-26T03:39:21.841 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.0669999999999999e-06 2024-04-26T03:39:21.841 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.841 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.841 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:21.841 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.521258+0000", 2024-04-26T03:39:21.841 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00031960300000000001 2024-04-26T03:39:21.841 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.841 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.842 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:21.842 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.521281+0000", 2024-04-26T03:39:21.842 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.2600999999999999e-05 2024-04-26T03:39:21.842 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.842 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.842 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:21.842 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.521290+0000", 2024-04-26T03:39:21.842 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.1169999999999993e-06 2024-04-26T03:39:21.842 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.842 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.842 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.842 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.528401+0000", 2024-04-26T03:39:21.842 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0071106499999999996 2024-04-26T03:39:21.842 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.843 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.843 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:21.843 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.528472+0000", 2024-04-26T03:39:21.843 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.1081999999999995e-05 2024-04-26T03:39:21.843 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.843 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.843 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:21.843 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.529015+0000", 2024-04-26T03:39:21.843 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00054347300000000005 2024-04-26T03:39:21.843 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.843 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.843 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:21.843 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.531961+0000", 2024-04-26T03:39:21.843 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.002945526 2024-04-26T03:39:21.843 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.844 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.844 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:21.844 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.536998+0000", 2024-04-26T03:39:21.844 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0050376960000000004 2024-04-26T03:39:21.844 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.844 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.844 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:21.844 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.537037+0000", 2024-04-26T03:39:21.844 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.8284999999999997e-05 2024-04-26T03:39:21.844 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.844 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:21.845 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.845 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.845 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.845 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:264 3.4s0 3:2d740be2:::smithi09235267-28:head [setxattr _header (56) in=7b] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T03:39:21.845 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:56.521315+0000", 2024-04-26T03:39:21.845 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 24.968802697000001, 2024-04-26T03:39:21.845 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.015842726000000001, 2024-04-26T03:39:21.845 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:21.845 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:21.845 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:21.845 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:21.845 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:21.845 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:21.845 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 264 2024-04-26T03:39:21.846 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.846 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:21.846 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.846 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:21.846 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.521315+0000", 2024-04-26T03:39:21.846 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.846 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.846 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.846 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:21.846 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.521315+0000", 2024-04-26T03:39:21.846 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.846 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.846 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.846 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:21.847 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.521316+0000", 2024-04-26T03:39:21.847 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.2799999999999995e-07 2024-04-26T03:39:21.847 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.847 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.847 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:21.847 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.521321+0000", 2024-04-26T03:39:21.847 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.9230000000000001e-06 2024-04-26T03:39:21.847 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.847 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.847 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:21.847 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.521330+0000", 2024-04-26T03:39:21.847 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.1859999999999998e-06 2024-04-26T03:39:21.847 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.847 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.847 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:21.848 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.521336+0000", 2024-04-26T03:39:21.848 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.8499999999999999e-06 2024-04-26T03:39:21.848 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.848 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.848 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.848 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.531370+0000", 2024-04-26T03:39:21.848 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.010033767000000001 2024-04-26T03:39:21.848 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.848 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.848 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:21.848 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.531431+0000", 2024-04-26T03:39:21.848 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.1359000000000007e-05 2024-04-26T03:39:21.848 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.848 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.849 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:21.849 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.531718+0000", 2024-04-26T03:39:21.849 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00028652000000000001 2024-04-26T03:39:21.849 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.849 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.849 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:21.849 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.532128+0000", 2024-04-26T03:39:21.849 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00041016700000000002 2024-04-26T03:39:21.849 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.849 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.849 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:21.849 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.537102+0000", 2024-04-26T03:39:21.849 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0049742520000000002 2024-04-26T03:39:21.849 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.850 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.850 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:21.850 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.537158+0000", 2024-04-26T03:39:21.850 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.5874000000000001e-05 2024-04-26T03:39:21.850 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.850 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:21.850 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.850 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.850 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.850 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:265 3.4s0 3:2d740be2:::smithi09235267-28:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e40)", 2024-04-26T03:39:21.850 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:56.521345+0000", 2024-04-26T03:39:21.850 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 24.968773387999999, 2024-04-26T03:39:21.850 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.017065469999999999, 2024-04-26T03:39:21.850 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:21.851 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:21.851 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "started", 2024-04-26T03:39:21.851 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:21.851 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:21.851 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:21.851 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 265 2024-04-26T03:39:21.851 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.851 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:21.851 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.851 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:21.851 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.521345+0000", 2024-04-26T03:39:21.851 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.851 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.851 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.851 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:21.852 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.521345+0000", 2024-04-26T03:39:21.852 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.852 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.852 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.852 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:21.852 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.521345+0000", 2024-04-26T03:39:21.852 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.1799999999999995e-07 2024-04-26T03:39:21.852 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.852 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.852 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:21.852 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.521347+0000", 2024-04-26T03:39:21.852 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.2199999999999999e-06 2024-04-26T03:39:21.852 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.852 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.853 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:21.853 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.521355+0000", 2024-04-26T03:39:21.853 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.9140000000000008e-06 2024-04-26T03:39:21.853 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.853 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.853 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:21.853 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.521360+0000", 2024-04-26T03:39:21.853 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.9989999999999999e-06 2024-04-26T03:39:21.853 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.853 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.853 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.853 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.531908+0000", 2024-04-26T03:39:21.853 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.010547707 2024-04-26T03:39:21.853 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.854 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.854 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:21.854 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.531953+0000", 2024-04-26T03:39:21.854 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.5543000000000001e-05 2024-04-26T03:39:21.854 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.854 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.854 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.854 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.537458+0000", 2024-04-26T03:39:21.854 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0055043699999999997 2024-04-26T03:39:21.854 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.854 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.854 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:21.854 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.537538+0000", 2024-04-26T03:39:21.854 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.9798999999999999e-05 2024-04-26T03:39:21.855 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.855 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.855 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:21.855 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.538410+0000", 2024-04-26T03:39:21.855 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00087239999999999996 2024-04-26T03:39:21.855 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.855 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:21.855 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.855 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.855 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.855 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:438 3.14s0 3:2d740be2:::smithi09235267-28:head [append 0~704512] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-04-26T03:39:21.855 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:04.723796+0000", 2024-04-26T03:39:21.855 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 16.766321574999999, 2024-04-26T03:39:21.855 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.016243738000000001, 2024-04-26T03:39:21.856 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:21.856 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:21.856 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:21.856 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:21.856 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:21.856 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:21.856 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 438 2024-04-26T03:39:21.856 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.856 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:21.856 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.856 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:21.856 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.723796+0000", 2024-04-26T03:39:21.856 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.856 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.857 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.857 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:21.857 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.723796+0000", 2024-04-26T03:39:21.857 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.857 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.857 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.857 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:21.857 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.723798+0000", 2024-04-26T03:39:21.857 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.4580000000000001e-06 2024-04-26T03:39:21.857 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.857 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.857 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:21.857 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.724544+0000", 2024-04-26T03:39:21.857 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00074663500000000005 2024-04-26T03:39:21.858 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.858 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.858 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:21.858 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.724557+0000", 2024-04-26T03:39:21.859 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.2758999999999999e-05 2024-04-26T03:39:21.859 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.859 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.859 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:21.859 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.724563+0000", 2024-04-26T03:39:21.859 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.2299999999999996e-06 2024-04-26T03:39:21.859 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.859 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.859 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.859 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.724629+0000", 2024-04-26T03:39:21.859 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.5907000000000004e-05 2024-04-26T03:39:21.859 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.859 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.859 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:21.860 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.724806+0000", 2024-04-26T03:39:21.860 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00017629200000000001 2024-04-26T03:39:21.860 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.861 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.861 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:21.861 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.726160+0000", 2024-04-26T03:39:21.861 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0013545289999999999 2024-04-26T03:39:21.861 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.861 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.861 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:21.861 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.739782+0000", 2024-04-26T03:39:21.861 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.013621416000000001 2024-04-26T03:39:21.861 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.861 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.861 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:21.861 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.739954+0000", 2024-04-26T03:39:21.861 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000172483 2024-04-26T03:39:21.862 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.862 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.862 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:21.862 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.740040+0000", 2024-04-26T03:39:21.862 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.6028999999999999e-05 2024-04-26T03:39:21.862 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.862 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:21.862 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.862 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.862 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.862 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:439 3.14s0 3:2d740be2:::smithi09235267-28:head [append 0~688128] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-04-26T03:39:21.862 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:04.727745+0000", 2024-04-26T03:39:21.862 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 16.762373091000001, 2024-04-26T03:39:21.862 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.015468773999999999, 2024-04-26T03:39:21.863 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:21.863 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:21.863 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:21.863 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:21.863 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:21.863 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:21.863 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 439 2024-04-26T03:39:21.863 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.863 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:21.863 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.863 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:21.863 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.727745+0000", 2024-04-26T03:39:21.863 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.863 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.864 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.864 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:21.864 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.727745+0000", 2024-04-26T03:39:21.864 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.864 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.864 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.864 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:21.864 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.727746+0000", 2024-04-26T03:39:21.864 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.5239999999999999e-06 2024-04-26T03:39:21.864 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.864 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.864 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:21.864 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.728329+0000", 2024-04-26T03:39:21.864 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00058246000000000003 2024-04-26T03:39:21.864 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.865 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.865 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:21.865 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.728340+0000", 2024-04-26T03:39:21.865 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1599e-05 2024-04-26T03:39:21.865 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.866 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.866 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:21.866 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.728346+0000", 2024-04-26T03:39:21.866 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.4310000000000002e-06 2024-04-26T03:39:21.866 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.866 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.866 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.866 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.733661+0000", 2024-04-26T03:39:21.866 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0053146790000000001 2024-04-26T03:39:21.866 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.866 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.866 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:21.867 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.733770+0000", 2024-04-26T03:39:21.867 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000109372 2024-04-26T03:39:21.867 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.867 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.867 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:21.867 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.734582+0000", 2024-04-26T03:39:21.867 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00081156700000000002 2024-04-26T03:39:21.867 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.867 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.867 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:21.867 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.742177+0000", 2024-04-26T03:39:21.867 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0075957860000000002 2024-04-26T03:39:21.867 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.867 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.867 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:21.868 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.743148+0000", 2024-04-26T03:39:21.868 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00097102400000000002 2024-04-26T03:39:21.868 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.868 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.868 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:21.868 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.743214+0000", 2024-04-26T03:39:21.868 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.5332000000000004e-05 2024-04-26T03:39:21.868 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.868 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:21.868 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.868 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.868 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.868 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:440 3.14s0 3:2d740be2:::smithi09235267-28:head [append 0~81920] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-04-26T03:39:21.868 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:04.728360+0000", 2024-04-26T03:39:21.869 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 16.761758134000001, 2024-04-26T03:39:21.869 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.015584582, 2024-04-26T03:39:21.869 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:21.869 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:21.869 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:21.869 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:21.869 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:21.869 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:21.869 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 440 2024-04-26T03:39:21.869 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.869 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:21.869 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.869 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:21.869 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.728360+0000", 2024-04-26T03:39:21.870 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.870 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.870 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.870 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:21.870 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.728360+0000", 2024-04-26T03:39:21.870 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.870 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.870 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.870 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:21.870 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.728360+0000", 2024-04-26T03:39:21.870 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.1399999999999997e-07 2024-04-26T03:39:21.870 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.870 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.870 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:21.871 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.728409+0000", 2024-04-26T03:39:21.871 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.8939e-05 2024-04-26T03:39:21.871 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.871 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.871 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:21.871 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.728418+0000", 2024-04-26T03:39:21.871 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.8100000000000004e-06 2024-04-26T03:39:21.871 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.871 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.871 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:21.871 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.728422+0000", 2024-04-26T03:39:21.871 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.9430000000000002e-06 2024-04-26T03:39:21.871 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.871 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.871 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.872 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.740077+0000", 2024-04-26T03:39:21.872 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.011654912 2024-04-26T03:39:21.872 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.872 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.872 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:21.872 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.740144+0000", 2024-04-26T03:39:21.872 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.6656999999999996e-05 2024-04-26T03:39:21.872 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.873 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.873 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:21.873 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.740515+0000", 2024-04-26T03:39:21.873 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00037100799999999998 2024-04-26T03:39:21.873 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.873 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.873 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:21.873 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.742297+0000", 2024-04-26T03:39:21.873 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.001782217 2024-04-26T03:39:21.873 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.873 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.873 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:21.873 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.743902+0000", 2024-04-26T03:39:21.873 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.001605569 2024-04-26T03:39:21.874 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.874 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.874 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:21.874 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.743944+0000", 2024-04-26T03:39:21.874 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.2012999999999999e-05 2024-04-26T03:39:21.874 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.874 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:21.874 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.874 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.874 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.874 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:441 3.14s0 3:2d740be2:::smithi09235267-28:head [setxattr _header (56) in=7b] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-04-26T03:39:21.874 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:04.728430+0000", 2024-04-26T03:39:21.874 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 16.761688051, 2024-04-26T03:39:21.874 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.015636728999999999, 2024-04-26T03:39:21.875 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:21.875 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:21.875 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:21.875 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:21.875 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:21.875 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:21.875 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 441 2024-04-26T03:39:21.875 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.875 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:21.875 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.875 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:21.875 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.728430+0000", 2024-04-26T03:39:21.875 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.875 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.876 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.876 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:21.876 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.728430+0000", 2024-04-26T03:39:21.876 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.876 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.876 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.876 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:21.876 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.728430+0000", 2024-04-26T03:39:21.876 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.7300000000000002e-07 2024-04-26T03:39:21.876 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.876 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.876 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:21.876 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.728433+0000", 2024-04-26T03:39:21.876 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.202e-06 2024-04-26T03:39:21.876 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.877 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.877 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:21.877 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.728438+0000", 2024-04-26T03:39:21.877 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.6280000000000004e-06 2024-04-26T03:39:21.877 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.877 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.877 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:21.877 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.728441+0000", 2024-04-26T03:39:21.877 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.9790000000000002e-06 2024-04-26T03:39:21.877 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.877 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.877 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.877 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.741303+0000", 2024-04-26T03:39:21.877 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.01286177 2024-04-26T03:39:21.878 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.878 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.878 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:21.878 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.741438+0000", 2024-04-26T03:39:21.878 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00013489599999999999 2024-04-26T03:39:21.878 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.878 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.878 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:21.878 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.741973+0000", 2024-04-26T03:39:21.878 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00053482699999999998 2024-04-26T03:39:21.878 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.878 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.878 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:21.878 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.742703+0000", 2024-04-26T03:39:21.879 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00073040699999999997 2024-04-26T03:39:21.879 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.879 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.879 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:21.879 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.744014+0000", 2024-04-26T03:39:21.879 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.001310819 2024-04-26T03:39:21.879 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.879 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.879 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:21.879 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.744067+0000", 2024-04-26T03:39:21.879 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.2828000000000003e-05 2024-04-26T03:39:21.879 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.879 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:21.879 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.879 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.880 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.880 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:442 3.14s0 3:2d740be2:::smithi09235267-28:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e49)", 2024-04-26T03:39:21.880 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:04.728445+0000", 2024-04-26T03:39:21.880 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 16.761672699000002, 2024-04-26T03:39:21.880 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.016969079000000001, 2024-04-26T03:39:21.880 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:21.880 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:21.880 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "started", 2024-04-26T03:39:21.880 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:21.880 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:21.880 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:21.880 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 442 2024-04-26T03:39:21.880 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.880 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:21.880 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.881 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:21.881 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.728445+0000", 2024-04-26T03:39:21.881 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.881 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.881 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.881 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:21.881 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.728445+0000", 2024-04-26T03:39:21.881 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.881 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.881 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.881 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:21.881 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.728446+0000", 2024-04-26T03:39:21.881 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.9299999999999999e-07 2024-04-26T03:39:21.881 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.882 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.882 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:21.882 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.728447+0000", 2024-04-26T03:39:21.882 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.3710000000000001e-06 2024-04-26T03:39:21.882 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.882 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.882 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:21.882 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.728452+0000", 2024-04-26T03:39:21.882 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.7990000000000001e-06 2024-04-26T03:39:21.882 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.882 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.882 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:21.882 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.728454+0000", 2024-04-26T03:39:21.882 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.717e-06 2024-04-26T03:39:21.883 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.883 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.883 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.883 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.742339+0000", 2024-04-26T03:39:21.883 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.013884931 2024-04-26T03:39:21.883 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.883 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.883 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:21.883 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.742421+0000", 2024-04-26T03:39:21.883 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.1386999999999999e-05 2024-04-26T03:39:21.883 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.883 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.883 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.883 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.744351+0000", 2024-04-26T03:39:21.884 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.001930153 2024-04-26T03:39:21.884 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.884 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.884 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:21.884 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.744430+0000", 2024-04-26T03:39:21.884 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.9515e-05 2024-04-26T03:39:21.884 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.884 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.884 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:21.884 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.745414+0000", 2024-04-26T03:39:21.884 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00098391300000000011 2024-04-26T03:39:21.884 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.884 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:21.884 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.884 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.885 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.885 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.10s4 60/48 ECSubWrite(tid=274, reqid=client.4556.0:705, at_version=60'22, trim_to=0'0, roll_forward_to=60'18))", 2024-04-26T03:39:21.885 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:15.966885+0000", 2024-04-26T03:39:21.885 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 5.5232326149999995, 2024-04-26T03:39:21.885 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.014214999000000001, 2024-04-26T03:39:21.885 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:21.885 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:21.885 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:21.885 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:21.885 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.885 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:21.885 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.966885+0000", 2024-04-26T03:39:21.885 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.885 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.886 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.886 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:21.886 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.966885+0000", 2024-04-26T03:39:21.886 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.886 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.886 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.886 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:21.886 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.966886+0000", 2024-04-26T03:39:21.886 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.9500000000000001e-07 2024-04-26T03:39:21.886 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.886 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.886 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:21.886 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.967191+0000", 2024-04-26T03:39:21.886 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00030450300000000002 2024-04-26T03:39:21.887 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.887 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.887 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:21.887 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.967204+0000", 2024-04-26T03:39:21.887 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.3642000000000001e-05 2024-04-26T03:39:21.887 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.887 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.887 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:21.887 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.967212+0000", 2024-04-26T03:39:21.887 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.9850000000000006e-06 2024-04-26T03:39:21.887 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.887 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.887 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.887 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.968412+0000", 2024-04-26T03:39:21.888 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0011998499999999999 2024-04-26T03:39:21.888 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.888 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.888 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:21.888 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.968421+0000", 2024-04-26T03:39:21.888 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.4989999999999994e-06 2024-04-26T03:39:21.888 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.888 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.888 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:21.888 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.981073+0000", 2024-04-26T03:39:21.888 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.012652527 2024-04-26T03:39:21.888 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.888 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.888 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:21.888 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.981100+0000", 2024-04-26T03:39:21.889 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.7197999999999999e-05 2024-04-26T03:39:21.889 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.889 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:21.889 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.889 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.889 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.889 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:754 3.14s0 3:2ee2abb1:::smithi09235267-12:head [append 0~688128] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:21.889 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:20.729310+0000", 2024-04-26T03:39:21.889 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 0.76080757600000004, 2024-04-26T03:39:21.889 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0196145, 2024-04-26T03:39:21.889 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:21.889 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:21.889 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:21.889 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:21.889 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:21.890 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:21.890 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 754 2024-04-26T03:39:21.890 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.890 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:21.890 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.890 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:21.890 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.729310+0000", 2024-04-26T03:39:21.890 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.890 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.890 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.890 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:21.890 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.729310+0000", 2024-04-26T03:39:21.890 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.891 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.891 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.891 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:21.891 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.729312+0000", 2024-04-26T03:39:21.891 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.0679999999999999e-06 2024-04-26T03:39:21.891 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.891 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.891 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:21.891 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.730088+0000", 2024-04-26T03:39:21.891 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00077547900000000003 2024-04-26T03:39:21.891 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.891 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.891 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:21.891 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.730109+0000", 2024-04-26T03:39:21.891 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.0707999999999999e-05 2024-04-26T03:39:21.892 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.892 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.892 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:21.892 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.730118+0000", 2024-04-26T03:39:21.892 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.5880000000000004e-06 2024-04-26T03:39:21.892 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.892 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.892 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.892 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.730179+0000", 2024-04-26T03:39:21.892 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.0545999999999999e-05 2024-04-26T03:39:21.892 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.892 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.892 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:21.892 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.730260+0000", 2024-04-26T03:39:21.893 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.1324999999999998e-05 2024-04-26T03:39:21.893 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.893 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.893 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:21.893 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.731473+0000", 2024-04-26T03:39:21.893 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.001212917 2024-04-26T03:39:21.893 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.893 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.893 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:21.893 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.748526+0000", 2024-04-26T03:39:21.893 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.017053467999999999 2024-04-26T03:39:21.893 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.893 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.893 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:21.894 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.748849+0000", 2024-04-26T03:39:21.894 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00032208099999999999 2024-04-26T03:39:21.894 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.894 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.894 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:21.894 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.748925+0000", 2024-04-26T03:39:21.894 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.6320000000000001e-05 2024-04-26T03:39:21.894 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.894 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:21.894 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.894 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.894 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.894 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:755 3.14s0 3:2ee2abb1:::smithi09235267-12:head [append 0~720896] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:21.894 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:20.733275+0000", 2024-04-26T03:39:21.895 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 0.75684273499999999, 2024-04-26T03:39:21.895 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.019559626, 2024-04-26T03:39:21.895 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:21.895 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:21.895 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:21.895 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:21.895 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:21.895 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:21.895 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 755 2024-04-26T03:39:21.895 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.895 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:21.895 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.895 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:21.895 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.733275+0000", 2024-04-26T03:39:21.895 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.896 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.896 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.896 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:21.896 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.733275+0000", 2024-04-26T03:39:21.896 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.896 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.896 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.896 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:21.896 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.733277+0000", 2024-04-26T03:39:21.896 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.283e-06 2024-04-26T03:39:21.896 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.896 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.896 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:21.896 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.733948+0000", 2024-04-26T03:39:21.897 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000671348 2024-04-26T03:39:21.897 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.897 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.897 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:21.897 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.733962+0000", 2024-04-26T03:39:21.897 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.4144999999999999e-05 2024-04-26T03:39:21.897 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.897 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.897 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:21.897 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.733971+0000", 2024-04-26T03:39:21.897 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.7849999999999997e-06 2024-04-26T03:39:21.897 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.897 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.897 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.898 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.737490+0000", 2024-04-26T03:39:21.898 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0035195790000000001 2024-04-26T03:39:21.898 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.898 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.898 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:21.898 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.737615+0000", 2024-04-26T03:39:21.898 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00012457899999999999 2024-04-26T03:39:21.898 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.898 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.898 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:21.898 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.738785+0000", 2024-04-26T03:39:21.898 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.001170405 2024-04-26T03:39:21.898 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.898 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.899 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:21.899 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.748528+0000", 2024-04-26T03:39:21.899 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0097424030000000002 2024-04-26T03:39:21.899 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.899 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.899 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:21.899 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.752780+0000", 2024-04-26T03:39:21.899 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0042521879999999996 2024-04-26T03:39:21.899 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.899 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.899 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:21.899 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.752835+0000", 2024-04-26T03:39:21.899 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.4911000000000003e-05 2024-04-26T03:39:21.899 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.899 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:21.900 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.900 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.900 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.900 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:756 3.14s0 3:2ee2abb1:::smithi09235267-12:head [append 0~475136] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:21.900 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:20.735986+0000", 2024-04-26T03:39:21.900 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 0.75413168600000002, 2024-04-26T03:39:21.900 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.020284473000000001, 2024-04-26T03:39:21.900 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:21.900 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:21.900 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:21.900 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:21.900 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:21.900 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:21.900 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 756 2024-04-26T03:39:21.901 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.901 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:21.901 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.901 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:21.901 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.735986+0000", 2024-04-26T03:39:21.901 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.901 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.901 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.901 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:21.901 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.735986+0000", 2024-04-26T03:39:21.901 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.901 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.901 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.901 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:21.902 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.735988+0000", 2024-04-26T03:39:21.902 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.773e-06 2024-04-26T03:39:21.902 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.902 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.902 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:21.902 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.736400+0000", 2024-04-26T03:39:21.902 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000412311 2024-04-26T03:39:21.902 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.902 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.902 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:21.902 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.736413+0000", 2024-04-26T03:39:21.902 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.2554e-05 2024-04-26T03:39:21.902 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.902 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.903 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:21.903 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.736419+0000", 2024-04-26T03:39:21.903 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.5429999999999999e-06 2024-04-26T03:39:21.903 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.903 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.903 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.903 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.744075+0000", 2024-04-26T03:39:21.903 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0076556050000000002 2024-04-26T03:39:21.903 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.903 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.903 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:21.903 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.744181+0000", 2024-04-26T03:39:21.903 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000105784 2024-04-26T03:39:21.903 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.903 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.904 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:21.904 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.745066+0000", 2024-04-26T03:39:21.904 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00088487699999999995 2024-04-26T03:39:21.904 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.904 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.904 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:21.904 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.749042+0000", 2024-04-26T03:39:21.904 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.003976757 2024-04-26T03:39:21.904 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.904 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.904 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:21.904 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.756213+0000", 2024-04-26T03:39:21.904 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0071703009999999996 2024-04-26T03:39:21.904 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.904 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.904 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:21.905 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.756271+0000", 2024-04-26T03:39:21.905 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.7967999999999998e-05 2024-04-26T03:39:21.905 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.905 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:21.905 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.905 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.905 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.905 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:757 3.14s0 3:2ee2abb1:::smithi09235267-12:head [append 0~360448] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:21.905 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:20.737966+0000", 2024-04-26T03:39:21.905 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 0.75215226800000001, 2024-04-26T03:39:21.905 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.018472444000000001, 2024-04-26T03:39:21.905 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:21.905 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:21.906 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:21.906 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:21.906 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:21.906 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:21.906 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 757 2024-04-26T03:39:21.906 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.906 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:21.906 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.906 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:21.906 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.737966+0000", 2024-04-26T03:39:21.906 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.906 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.906 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.906 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:21.906 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.737966+0000", 2024-04-26T03:39:21.907 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.907 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.907 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.907 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:21.907 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.737967+0000", 2024-04-26T03:39:21.907 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.031e-06 2024-04-26T03:39:21.907 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.907 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.907 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:21.907 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.738361+0000", 2024-04-26T03:39:21.907 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000394163 2024-04-26T03:39:21.907 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.907 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.907 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:21.908 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.738372+0000", 2024-04-26T03:39:21.908 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1545999999999999e-05 2024-04-26T03:39:21.908 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.908 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.908 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:21.908 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.738380+0000", 2024-04-26T03:39:21.908 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.6029999999999999e-06 2024-04-26T03:39:21.908 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.908 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.908 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.908 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.749098+0000", 2024-04-26T03:39:21.908 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.010717585 2024-04-26T03:39:21.908 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.908 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.908 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:21.909 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.749165+0000", 2024-04-26T03:39:21.909 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.7364000000000001e-05 2024-04-26T03:39:21.909 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.909 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.909 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:21.909 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.749659+0000", 2024-04-26T03:39:21.909 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00049434999999999995 2024-04-26T03:39:21.909 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.909 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.909 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:21.909 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.753323+0000", 2024-04-26T03:39:21.909 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0036639889999999999 2024-04-26T03:39:21.909 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.909 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.909 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:21.910 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.756392+0000", 2024-04-26T03:39:21.910 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0030688410000000001 2024-04-26T03:39:21.910 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.910 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.910 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:21.910 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.756438+0000", 2024-04-26T03:39:21.910 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.5972e-05 2024-04-26T03:39:21.910 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.910 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:21.910 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.910 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.910 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.910 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:758 3.14s0 3:2ee2abb1:::smithi09235267-12:head [setxattr _header (57) in=7b] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:21.910 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:20.738397+0000", 2024-04-26T03:39:21.911 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 0.75172146500000003, 2024-04-26T03:39:21.911 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.018504256, 2024-04-26T03:39:21.911 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:21.911 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:21.911 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:21.911 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:21.911 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:21.911 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:21.911 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 758 2024-04-26T03:39:21.911 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.911 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:21.911 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.911 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:21.911 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.738397+0000", 2024-04-26T03:39:21.912 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.912 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.912 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.912 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:21.912 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.738397+0000", 2024-04-26T03:39:21.912 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.912 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.912 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.912 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:21.912 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.738397+0000", 2024-04-26T03:39:21.912 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.2900000000000002e-07 2024-04-26T03:39:21.912 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.912 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.912 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:21.912 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.738402+0000", 2024-04-26T03:39:21.913 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.9150000000000004e-06 2024-04-26T03:39:21.913 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.913 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.913 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:21.913 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.738409+0000", 2024-04-26T03:39:21.913 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.8419999999999999e-06 2024-04-26T03:39:21.913 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.913 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.913 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:21.913 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.738414+0000", 2024-04-26T03:39:21.913 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.1800000000000004e-06 2024-04-26T03:39:21.913 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.913 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.913 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.913 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.751944+0000", 2024-04-26T03:39:21.914 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.013529617000000001 2024-04-26T03:39:21.914 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.914 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.914 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:21.914 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.752019+0000", 2024-04-26T03:39:21.914 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.4784999999999996e-05 2024-04-26T03:39:21.914 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.914 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.914 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:21.914 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.752320+0000", 2024-04-26T03:39:21.914 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000301052 2024-04-26T03:39:21.914 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.914 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.914 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:21.914 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.753331+0000", 2024-04-26T03:39:21.915 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0010112610000000001 2024-04-26T03:39:21.915 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.915 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.915 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:21.915 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.756849+0000", 2024-04-26T03:39:21.915 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.003518017 2024-04-26T03:39:21.915 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.915 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.915 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:21.915 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.756901+0000", 2024-04-26T03:39:21.915 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.1758000000000001e-05 2024-04-26T03:39:21.915 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.915 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:21.915 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.915 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.916 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.916 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:759 3.14s0 3:2ee2abb1:::smithi09235267-12:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:21.916 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:20.738422+0000", 2024-04-26T03:39:21.916 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 0.751696009, 2024-04-26T03:39:21.916 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.020008317000000001, 2024-04-26T03:39:21.916 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:21.916 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:21.916 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "started", 2024-04-26T03:39:21.916 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:21.916 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:21.916 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:21.916 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 759 2024-04-26T03:39:21.916 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.916 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:21.917 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.917 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:21.917 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.738422+0000", 2024-04-26T03:39:21.917 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.917 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.917 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.917 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:21.917 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.738422+0000", 2024-04-26T03:39:21.917 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:21.917 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.917 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.917 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:21.917 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.738423+0000", 2024-04-26T03:39:21.917 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.4199999999999996e-07 2024-04-26T03:39:21.917 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.918 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.918 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:21.918 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.738425+0000", 2024-04-26T03:39:21.918 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.6910000000000002e-06 2024-04-26T03:39:21.918 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.918 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.918 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:21.918 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.738432+0000", 2024-04-26T03:39:21.918 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.2629999999999997e-06 2024-04-26T03:39:21.918 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.918 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.918 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:21.918 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.738438+0000", 2024-04-26T03:39:21.918 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.2360000000000004e-06 2024-04-26T03:39:21.918 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.919 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.919 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.919 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.752936+0000", 2024-04-26T03:39:21.919 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.014498363 2024-04-26T03:39:21.919 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.919 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.919 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:21.919 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.753003+0000", 2024-04-26T03:39:21.919 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.7168000000000005e-05 2024-04-26T03:39:21.919 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.919 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.919 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:21.919 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.757220+0000", 2024-04-26T03:39:21.919 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0042164799999999999 2024-04-26T03:39:21.920 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.920 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.920 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:21.920 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.757311+0000", 2024-04-26T03:39:21.920 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.0828000000000005e-05 2024-04-26T03:39:21.920 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:21.920 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:21.920 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:21.920 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.758430+0000", 2024-04-26T03:39:21.920 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.001119746 2024-04-26T03:39:21.920 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.920 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:21.920 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.920 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:21.920 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:21.921 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:39:21.923 INFO:tasks.rados.rados.0.smithi092.stdout:296: done (2 left) 2024-04-26T03:39:21.923 INFO:tasks.rados.rados.0.smithi092.stdout:297: done (1 left) 2024-04-26T03:39:21.923 INFO:tasks.rados.rados.0.smithi092.stdout:298: done (0 left) 2024-04-26T03:39:21.923 INFO:tasks.rados.rados.0.smithi092.stdout:299: read oid 4 snap 12 2024-04-26T03:39:21.923 INFO:tasks.rados.rados.0.smithi092.stdout:299: expect (ObjNum 56 snap 0 seq_num 56) 2024-04-26T03:39:21.923 INFO:tasks.rados.rados.0.smithi092.stdout:300: snap_create 2024-04-26T03:39:21.923 INFO:tasks.rados.rados.0.smithi092.stdout:299: expect (ObjNum 56 snap 0 seq_num 56) 2024-04-26T03:39:21.923 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-04-26T03:39:21.924 INFO:tasks.ceph.osd.10.smithi066.stderr:2024-04-26T03:39:21.921+0000 7f130fcc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 31763) UID: 0 2024-04-26T03:39:22.023 INFO:tasks.ceph.osd.8.smithi028.stderr:2024-04-26T03:39:22.021+0000 7f5bb75d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 31968) UID: 0 2024-04-26T03:39:22.033 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:39:22.033 INFO:teuthology.orchestra.run.smithi028.stdout: "ops": [], 2024-04-26T03:39:22.033 INFO:teuthology.orchestra.run.smithi028.stdout: "num_ops": 0 2024-04-26T03:39:22.033 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:39:22.042 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-04-26T03:39:22.124 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-26T03:39:22.123+0000 7f610b3a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31816) UID: 0 2024-04-26T03:39:22.189 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:39:22.189 INFO:teuthology.orchestra.run.smithi028.stdout: "ops": [], 2024-04-26T03:39:22.189 INFO:teuthology.orchestra.run.smithi028.stdout: "complaint_time": 30, 2024-04-26T03:39:22.189 INFO:teuthology.orchestra.run.smithi028.stdout: "num_blocked_ops": 0 2024-04-26T03:39:22.189 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:39:22.198 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-04-26T03:39:22.225 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:39:22.223+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:39:22.307 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:39:22.307 INFO:teuthology.orchestra.run.smithi028.stdout: "size": 20, 2024-04-26T03:39:22.307 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 600, 2024-04-26T03:39:22.307 INFO:teuthology.orchestra.run.smithi028.stdout: "ops": [ 2024-04-26T03:39:22.308 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.308 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.es6 35/32 ECSubWrite(tid=2, reqid=client.4556.0:58, at_version=35'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:39:22.308 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.512937+0000", 2024-04-26T03:39:22.308 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 30.790945408999999, 2024-04-26T03:39:22.308 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0046369269999999999, 2024-04-26T03:39:22.308 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:22.308 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:22.309 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:22.309 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:22.309 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.309 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:22.309 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.512937+0000", 2024-04-26T03:39:22.309 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:22.310 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.310 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.310 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:22.310 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.512937+0000", 2024-04-26T03:39:22.310 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:22.310 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.310 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.311 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:22.311 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.512937+0000", 2024-04-26T03:39:22.311 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.8099999999999999e-07 2024-04-26T03:39:22.311 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.311 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.311 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:22.311 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.513175+0000", 2024-04-26T03:39:22.312 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000237851 2024-04-26T03:39:22.312 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.312 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.312 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:22.312 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.513196+0000", 2024-04-26T03:39:22.312 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.0344999999999999e-05 2024-04-26T03:39:22.313 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.313 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.313 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:22.313 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.513207+0000", 2024-04-26T03:39:22.313 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1083999999999999e-05 2024-04-26T03:39:22.313 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.313 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.314 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:22.314 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.513309+0000", 2024-04-26T03:39:22.314 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000102321 2024-04-26T03:39:22.314 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.314 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.314 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:22.315 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.513318+0000", 2024-04-26T03:39:22.315 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.5219999999999996e-06 2024-04-26T03:39:22.315 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.315 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.315 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:22.315 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.517543+0000", 2024-04-26T03:39:22.315 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0042245160000000002 2024-04-26T03:39:22.315 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.315 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.315 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:22.315 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.517574+0000", 2024-04-26T03:39:22.316 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.1007000000000003e-05 2024-04-26T03:39:22.316 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:22.316 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:22.316 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:22.316 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.316 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.317 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.es6 35/32 ECSubWrite(tid=16, reqid=client.4556.0:85, at_version=35'5, trim_to=0'0, roll_forward_to=35'4))", 2024-04-26T03:39:22.317 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.542343+0000", 2024-04-26T03:39:22.317 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 30.761539845000001, 2024-04-26T03:39:22.317 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0051706219999999997, 2024-04-26T03:39:22.317 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:22.317 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:22.317 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:22.317 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:22.317 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.317 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:22.317 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.542343+0000", 2024-04-26T03:39:22.318 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:22.318 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.318 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.318 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:22.318 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.542343+0000", 2024-04-26T03:39:22.318 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:22.318 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.318 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.319 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:22.319 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.542344+0000", 2024-04-26T03:39:22.319 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.488e-06 2024-04-26T03:39:22.319 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.319 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.319 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:22.320 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.542511+0000", 2024-04-26T03:39:22.320 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000166844 2024-04-26T03:39:22.320 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.320 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.320 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:22.320 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.542528+0000", 2024-04-26T03:39:22.321 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.6548000000000001e-05 2024-04-26T03:39:22.321 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.321 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.321 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:22.321 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.542535+0000", 2024-04-26T03:39:22.321 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.7610000000000007e-06 2024-04-26T03:39:22.321 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.321 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.322 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:22.322 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.542610+0000", 2024-04-26T03:39:22.322 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.4632e-05 2024-04-26T03:39:22.322 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.322 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.322 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:22.322 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.542620+0000", 2024-04-26T03:39:22.323 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0108e-05 2024-04-26T03:39:22.323 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.323 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.323 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:22.323 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.547469+0000", 2024-04-26T03:39:22.323 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0048486129999999999 2024-04-26T03:39:22.323 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.323 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.323 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:22.324 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.547513+0000", 2024-04-26T03:39:22.324 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.4628e-05 2024-04-26T03:39:22.324 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:22.324 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:22.324 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:22.324 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.324 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.324 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.es6 35/32 ECSubWrite(tid=24, reqid=client.4556.0:124, at_version=35'7, trim_to=0'0, roll_forward_to=35'6))", 2024-04-26T03:39:22.324 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.596283+0000", 2024-04-26T03:39:22.325 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 30.707599546000001, 2024-04-26T03:39:22.325 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0047651710000000003, 2024-04-26T03:39:22.325 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:22.325 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:22.325 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:22.325 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:22.325 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.325 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:22.326 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.596283+0000", 2024-04-26T03:39:22.326 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:22.326 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.326 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.326 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:22.326 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.596283+0000", 2024-04-26T03:39:22.327 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:22.327 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.327 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.327 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:22.327 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.596284+0000", 2024-04-26T03:39:22.327 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0950000000000001e-06 2024-04-26T03:39:22.327 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.327 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.327 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:22.328 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.596595+0000", 2024-04-26T03:39:22.328 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00031093400000000002 2024-04-26T03:39:22.328 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.328 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.328 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:22.328 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.596610+0000", 2024-04-26T03:39:22.328 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.5264000000000002e-05 2024-04-26T03:39:22.328 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.328 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.329 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:22.329 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.596617+0000", 2024-04-26T03:39:22.329 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.0859999999999996e-06 2024-04-26T03:39:22.329 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.329 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.329 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:22.329 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.596669+0000", 2024-04-26T03:39:22.329 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.1436e-05 2024-04-26T03:39:22.329 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.330 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.330 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:22.330 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.596675+0000", 2024-04-26T03:39:22.330 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.4030000000000004e-06 2024-04-26T03:39:22.330 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.330 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.330 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:22.331 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601010+0000", 2024-04-26T03:39:22.331 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0043352030000000001 2024-04-26T03:39:22.331 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.331 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.331 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:22.331 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601048+0000", 2024-04-26T03:39:22.331 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.7750000000000003e-05 2024-04-26T03:39:22.331 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:22.331 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:22.332 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:22.332 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.332 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.332 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.bs3 35/32 ECSubWrite(tid=20, reqid=client.4556.0:127, at_version=35'7, trim_to=0'0, roll_forward_to=35'6))", 2024-04-26T03:39:22.332 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.601363+0000", 2024-04-26T03:39:22.332 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 30.702519589000001, 2024-04-26T03:39:22.332 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.007091913, 2024-04-26T03:39:22.332 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:22.333 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:22.333 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:22.333 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:22.333 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.333 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:22.333 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601363+0000", 2024-04-26T03:39:22.333 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:22.333 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.333 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.334 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:22.334 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601363+0000", 2024-04-26T03:39:22.334 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:22.334 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.334 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.334 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:22.334 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601364+0000", 2024-04-26T03:39:22.334 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.54e-07 2024-04-26T03:39:22.335 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.335 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.335 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:22.335 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601498+0000", 2024-04-26T03:39:22.335 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00013417600000000001 2024-04-26T03:39:22.335 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.335 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.335 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:22.335 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601514+0000", 2024-04-26T03:39:22.335 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.6371999999999998e-05 2024-04-26T03:39:22.335 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.335 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.335 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:22.335 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601523+0000", 2024-04-26T03:39:22.336 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.0189999999999995e-06 2024-04-26T03:39:22.336 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.336 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.336 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:22.336 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601589+0000", 2024-04-26T03:39:22.336 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.5924000000000001e-05 2024-04-26T03:39:22.336 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.336 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.336 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:22.336 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601598+0000", 2024-04-26T03:39:22.336 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.6940000000000001e-06 2024-04-26T03:39:22.336 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.336 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.336 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:22.337 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.608417+0000", 2024-04-26T03:39:22.337 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.006819121 2024-04-26T03:39:22.337 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.337 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.337 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:22.337 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.608455+0000", 2024-04-26T03:39:22.337 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.7753000000000003e-05 2024-04-26T03:39:22.337 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:22.337 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:22.337 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:22.337 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.337 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.337 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.bs3 35/32 ECSubWrite(tid=22, reqid=client.4556.0:133, at_version=35'9, trim_to=0'0, roll_forward_to=35'7))", 2024-04-26T03:39:22.337 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.610898+0000", 2024-04-26T03:39:22.338 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 30.692984380999999, 2024-04-26T03:39:22.338 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0048565170000000003, 2024-04-26T03:39:22.338 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:22.338 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:22.338 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:22.338 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:22.338 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.338 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:22.338 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.610898+0000", 2024-04-26T03:39:22.338 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:22.338 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.338 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.338 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:22.339 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.610898+0000", 2024-04-26T03:39:22.339 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:22.339 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.339 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.339 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:22.339 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.610899+0000", 2024-04-26T03:39:22.339 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.9500000000000001e-07 2024-04-26T03:39:22.339 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.339 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.339 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:22.339 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.611030+0000", 2024-04-26T03:39:22.339 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00013066099999999999 2024-04-26T03:39:22.339 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.339 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.340 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:22.340 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.611046+0000", 2024-04-26T03:39:22.340 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.6056e-05 2024-04-26T03:39:22.340 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.340 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.340 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:22.340 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.611053+0000", 2024-04-26T03:39:22.340 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.1999999999999997e-06 2024-04-26T03:39:22.340 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.340 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.340 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:22.340 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.611119+0000", 2024-04-26T03:39:22.340 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.5839000000000004e-05 2024-04-26T03:39:22.340 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.340 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.341 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:22.341 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.611136+0000", 2024-04-26T03:39:22.341 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.7070000000000001e-05 2024-04-26T03:39:22.341 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.341 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.341 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:22.341 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.615722+0000", 2024-04-26T03:39:22.341 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.004586499 2024-04-26T03:39:22.341 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.341 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.341 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:22.341 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.615755+0000", 2024-04-26T03:39:22.341 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.2296999999999999e-05 2024-04-26T03:39:22.341 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:22.342 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:22.342 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:22.342 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.342 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.342 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.es6 35/32 ECSubWrite(tid=34, reqid=client.4556.0:177, at_version=35'9, trim_to=0'0, roll_forward_to=35'8))", 2024-04-26T03:39:22.342 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.731787+0000", 2024-04-26T03:39:22.342 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 30.572095100999999, 2024-04-26T03:39:22.342 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0075575629999999998, 2024-04-26T03:39:22.342 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:22.342 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:22.342 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:22.342 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:22.342 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.342 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:22.343 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.731787+0000", 2024-04-26T03:39:22.343 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:22.343 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.343 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.343 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:22.343 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.731787+0000", 2024-04-26T03:39:22.343 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:22.343 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.343 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.343 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:22.343 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.731788+0000", 2024-04-26T03:39:22.343 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1200000000000001e-06 2024-04-26T03:39:22.343 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.343 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.343 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:22.344 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.732056+0000", 2024-04-26T03:39:22.344 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00026767099999999998 2024-04-26T03:39:22.344 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.344 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.344 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:22.344 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.732072+0000", 2024-04-26T03:39:22.344 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.6242000000000001e-05 2024-04-26T03:39:22.344 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.344 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.344 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:22.344 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.732083+0000", 2024-04-26T03:39:22.344 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0135e-05 2024-04-26T03:39:22.344 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.344 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.345 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:22.345 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.732152+0000", 2024-04-26T03:39:22.345 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.9325000000000005e-05 2024-04-26T03:39:22.345 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.345 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.345 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:22.345 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.732163+0000", 2024-04-26T03:39:22.345 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0728e-05 2024-04-26T03:39:22.345 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.345 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.345 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:22.345 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.739313+0000", 2024-04-26T03:39:22.345 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0071506599999999997 2024-04-26T03:39:22.345 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.346 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.346 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:22.346 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.739345+0000", 2024-04-26T03:39:22.346 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.1681999999999999e-05 2024-04-26T03:39:22.346 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:22.346 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:22.346 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:22.346 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.346 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.346 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.es6 35/32 ECSubWrite(tid=39, reqid=client.4556.0:207, at_version=35'12, trim_to=0'0, roll_forward_to=35'11))", 2024-04-26T03:39:22.346 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.807524+0000", 2024-04-26T03:39:22.346 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 30.496358675, 2024-04-26T03:39:22.346 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0081817109999999995, 2024-04-26T03:39:22.347 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:22.347 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:22.347 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:22.347 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:22.347 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.347 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:22.347 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.807524+0000", 2024-04-26T03:39:22.347 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:22.347 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.347 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.347 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:22.347 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.807524+0000", 2024-04-26T03:39:22.347 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:22.347 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.348 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.348 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:22.348 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.807525+0000", 2024-04-26T03:39:22.348 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.6700000000000002e-07 2024-04-26T03:39:22.348 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.348 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.348 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:22.348 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.808020+0000", 2024-04-26T03:39:22.348 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00049521799999999996 2024-04-26T03:39:22.348 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.348 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.348 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:22.348 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.808031+0000", 2024-04-26T03:39:22.348 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0862999999999999e-05 2024-04-26T03:39:22.349 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.349 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.349 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:22.349 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.808037+0000", 2024-04-26T03:39:22.349 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.438e-06 2024-04-26T03:39:22.349 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.349 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.349 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:22.349 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.808102+0000", 2024-04-26T03:39:22.349 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.4388999999999996e-05 2024-04-26T03:39:22.349 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.349 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.349 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:22.350 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.808114+0000", 2024-04-26T03:39:22.350 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.2211e-05 2024-04-26T03:39:22.350 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.350 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.350 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:22.350 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.815665+0000", 2024-04-26T03:39:22.350 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0075510259999999997 2024-04-26T03:39:22.350 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.350 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.350 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:22.350 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.815706+0000", 2024-04-26T03:39:22.350 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.0698999999999998e-05 2024-04-26T03:39:22.350 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:22.351 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:22.351 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:22.351 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.351 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.351 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.es6 35/32 ECSubWrite(tid=40, reqid=client.4556.0:208, at_version=35'13, trim_to=0'0, roll_forward_to=35'11))", 2024-04-26T03:39:22.351 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.811551+0000", 2024-04-26T03:39:22.351 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 30.492331682, 2024-04-26T03:39:22.351 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0045656179999999996, 2024-04-26T03:39:22.351 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:22.351 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:22.351 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:22.351 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:22.351 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.351 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:22.352 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.811551+0000", 2024-04-26T03:39:22.352 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:22.352 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.352 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.352 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:22.352 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.811551+0000", 2024-04-26T03:39:22.352 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:22.352 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.352 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.352 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:22.352 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.811551+0000", 2024-04-26T03:39:22.352 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.3900000000000005e-07 2024-04-26T03:39:22.352 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.352 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.353 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:22.353 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.811779+0000", 2024-04-26T03:39:22.353 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000227541 2024-04-26T03:39:22.353 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.353 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.353 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:22.353 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.811790+0000", 2024-04-26T03:39:22.353 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1076e-05 2024-04-26T03:39:22.353 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.353 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.353 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:22.353 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.811796+0000", 2024-04-26T03:39:22.353 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.643e-06 2024-04-26T03:39:22.353 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.354 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.354 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:22.354 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.813542+0000", 2024-04-26T03:39:22.354 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0017465969999999999 2024-04-26T03:39:22.354 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.354 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.354 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:22.354 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.813549+0000", 2024-04-26T03:39:22.354 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.5359999999999998e-06 2024-04-26T03:39:22.354 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.354 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.354 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:22.354 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.816088+0000", 2024-04-26T03:39:22.354 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0025395750000000001 2024-04-26T03:39:22.355 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.355 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.355 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:22.355 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.816116+0000", 2024-04-26T03:39:22.355 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.8110999999999999e-05 2024-04-26T03:39:22.355 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:22.355 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:22.355 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:22.355 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.355 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.355 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.es6 43/32 ECSubWrite(tid=48, reqid=client.4556.0:301, at_version=43'16, trim_to=0'0, roll_forward_to=35'14))", 2024-04-26T03:39:22.355 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:59.531146+0000", 2024-04-26T03:39:22.355 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 22.772736391999999, 2024-04-26T03:39:22.355 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.004955175, 2024-04-26T03:39:22.355 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:22.356 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:22.356 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:22.356 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:22.356 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.356 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:22.356 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.531146+0000", 2024-04-26T03:39:22.356 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:22.356 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.356 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.356 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:22.356 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.531146+0000", 2024-04-26T03:39:22.356 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:22.356 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.356 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.357 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:22.357 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.531146+0000", 2024-04-26T03:39:22.357 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.49e-07 2024-04-26T03:39:22.357 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.357 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.357 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:22.357 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.531483+0000", 2024-04-26T03:39:22.357 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00033656099999999998 2024-04-26T03:39:22.357 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.357 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.357 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:22.357 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.531497+0000", 2024-04-26T03:39:22.357 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.378e-05 2024-04-26T03:39:22.357 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.358 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.358 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:22.358 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.531506+0000", 2024-04-26T03:39:22.358 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.2159999999999995e-06 2024-04-26T03:39:22.358 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.358 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.358 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:22.358 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.531587+0000", 2024-04-26T03:39:22.358 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.1180999999999994e-05 2024-04-26T03:39:22.358 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.358 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.358 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:22.358 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.531599+0000", 2024-04-26T03:39:22.358 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.2169e-05 2024-04-26T03:39:22.359 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.359 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.359 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:22.359 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.536067+0000", 2024-04-26T03:39:22.359 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0044680099999999997 2024-04-26T03:39:22.359 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.359 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.359 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:22.359 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.536101+0000", 2024-04-26T03:39:22.359 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.4109000000000002e-05 2024-04-26T03:39:22.359 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:22.359 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:22.359 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:22.359 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.359 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.360 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.es6 43/32 ECSubWrite(tid=49, reqid=client.4556.0:302, at_version=43'17, trim_to=0'0, roll_forward_to=35'14))", 2024-04-26T03:39:22.360 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:59.537150+0000", 2024-04-26T03:39:22.360 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 22.766732693000002, 2024-04-26T03:39:22.360 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0058389130000000003, 2024-04-26T03:39:22.360 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:22.360 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:22.360 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:22.360 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:22.360 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.360 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:22.360 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.537150+0000", 2024-04-26T03:39:22.360 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:22.360 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.360 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.360 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:22.361 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.537150+0000", 2024-04-26T03:39:22.361 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:22.361 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.361 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.361 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:22.361 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.537151+0000", 2024-04-26T03:39:22.361 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.92e-07 2024-04-26T03:39:22.361 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.361 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.361 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:22.361 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.537707+0000", 2024-04-26T03:39:22.361 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00055628199999999996 2024-04-26T03:39:22.361 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.361 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.362 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:22.362 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.537720+0000", 2024-04-26T03:39:22.362 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.3434999999999999e-05 2024-04-26T03:39:22.362 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.362 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.362 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:22.362 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.537728+0000", 2024-04-26T03:39:22.362 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.8290000000000008e-06 2024-04-26T03:39:22.362 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.362 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.362 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:22.362 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.537787+0000", 2024-04-26T03:39:22.362 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.9249000000000001e-05 2024-04-26T03:39:22.362 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.363 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.363 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:22.363 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.537798+0000", 2024-04-26T03:39:22.363 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0997e-05 2024-04-26T03:39:22.363 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.363 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.363 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:22.363 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.542961+0000", 2024-04-26T03:39:22.363 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0051625320000000001 2024-04-26T03:39:22.363 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.363 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.363 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:22.363 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.542989+0000", 2024-04-26T03:39:22.363 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.7797000000000002e-05 2024-04-26T03:39:22.364 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:22.364 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:22.364 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:22.364 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.364 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.364 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.cs5 45/32 ECSubWrite(tid=53, reqid=client.4556.0:338, at_version=45'21, trim_to=0'0, roll_forward_to=42'19))", 2024-04-26T03:39:22.364 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:01.572023+0000", 2024-04-26T03:39:22.364 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 20.731859601, 2024-04-26T03:39:22.364 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0057687980000000003, 2024-04-26T03:39:22.364 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:22.364 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:22.364 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:22.364 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:22.364 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.365 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:22.365 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.572023+0000", 2024-04-26T03:39:22.365 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:22.365 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.365 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.365 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:22.365 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.572023+0000", 2024-04-26T03:39:22.365 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:22.365 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.365 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.365 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:22.365 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.572023+0000", 2024-04-26T03:39:22.365 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.8599999999999999e-07 2024-04-26T03:39:22.365 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.365 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.366 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:22.366 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.572451+0000", 2024-04-26T03:39:22.366 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00042800899999999997 2024-04-26T03:39:22.366 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.366 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.366 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:22.366 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.572468+0000", 2024-04-26T03:39:22.366 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.6991999999999999e-05 2024-04-26T03:39:22.366 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.366 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.366 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:22.366 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.572480+0000", 2024-04-26T03:39:22.366 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1338000000000001e-05 2024-04-26T03:39:22.366 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.367 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.367 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:22.367 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.572569+0000", 2024-04-26T03:39:22.367 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.9571e-05 2024-04-26T03:39:22.367 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.367 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.367 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:22.367 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.572582+0000", 2024-04-26T03:39:22.367 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.295e-05 2024-04-26T03:39:22.367 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.367 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.367 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:22.367 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.577759+0000", 2024-04-26T03:39:22.367 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0051768300000000003 2024-04-26T03:39:22.367 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.368 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.368 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:22.368 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.577792+0000", 2024-04-26T03:39:22.368 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.2722000000000003e-05 2024-04-26T03:39:22.368 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:22.368 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:22.368 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:22.368 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.368 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.368 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.cs5 45/32 ECSubWrite(tid=56, reqid=client.4556.0:339, at_version=45'22, trim_to=0'0, roll_forward_to=42'19))", 2024-04-26T03:39:22.368 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:01.578382+0000", 2024-04-26T03:39:22.368 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 20.725500631999999, 2024-04-26T03:39:22.368 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0044307010000000004, 2024-04-26T03:39:22.368 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:22.369 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:22.369 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:22.369 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:22.369 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.369 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:22.369 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.578382+0000", 2024-04-26T03:39:22.369 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:22.369 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.369 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.369 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:22.369 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.578382+0000", 2024-04-26T03:39:22.369 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:22.369 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.369 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.369 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:22.370 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.578383+0000", 2024-04-26T03:39:22.370 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.1299999999999999e-07 2024-04-26T03:39:22.370 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.370 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.370 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:22.370 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.578813+0000", 2024-04-26T03:39:22.370 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00043056000000000002 2024-04-26T03:39:22.370 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.370 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.370 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:22.370 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.578825+0000", 2024-04-26T03:39:22.370 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.167e-05 2024-04-26T03:39:22.370 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.370 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.371 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:22.371 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.578831+0000", 2024-04-26T03:39:22.371 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.088e-06 2024-04-26T03:39:22.371 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.371 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.371 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:22.371 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.578892+0000", 2024-04-26T03:39:22.371 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.1495000000000007e-05 2024-04-26T03:39:22.371 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.371 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.371 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:22.371 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.578899+0000", 2024-04-26T03:39:22.371 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.4019999999999999e-06 2024-04-26T03:39:22.371 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.371 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.372 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:22.372 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.582776+0000", 2024-04-26T03:39:22.372 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0038771360000000002 2024-04-26T03:39:22.372 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.372 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.372 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:22.372 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.582813+0000", 2024-04-26T03:39:22.372 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.6536999999999999e-05 2024-04-26T03:39:22.372 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:22.372 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:22.372 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:22.372 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.372 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.372 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.es6 45/32 ECSubWrite(tid=62, reqid=client.4556.0:349, at_version=45'21, trim_to=0'0, roll_forward_to=43'19))", 2024-04-26T03:39:22.373 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:01.632894+0000", 2024-04-26T03:39:22.373 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 20.670988947000001, 2024-04-26T03:39:22.373 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0051349860000000002, 2024-04-26T03:39:22.373 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:22.373 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:22.373 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:22.373 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:22.373 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.373 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:22.373 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.632894+0000", 2024-04-26T03:39:22.373 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:22.373 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.373 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.373 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:22.373 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.632894+0000", 2024-04-26T03:39:22.373 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:22.374 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.374 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.374 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:22.374 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.632894+0000", 2024-04-26T03:39:22.374 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.6100000000000002e-07 2024-04-26T03:39:22.374 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.374 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.374 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:22.374 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.633352+0000", 2024-04-26T03:39:22.374 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00045790799999999998 2024-04-26T03:39:22.374 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.374 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.374 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:22.374 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.633364+0000", 2024-04-26T03:39:22.375 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.2157999999999999e-05 2024-04-26T03:39:22.375 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.375 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.375 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:22.375 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.633372+0000", 2024-04-26T03:39:22.375 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.9589999999999995e-06 2024-04-26T03:39:22.375 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.375 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.375 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:22.375 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.633431+0000", 2024-04-26T03:39:22.375 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.8690999999999998e-05 2024-04-26T03:39:22.375 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.375 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.375 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:22.376 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.633437+0000", 2024-04-26T03:39:22.376 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.798e-06 2024-04-26T03:39:22.376 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.376 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.376 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:22.376 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.637987+0000", 2024-04-26T03:39:22.376 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0045496130000000001 2024-04-26T03:39:22.376 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.376 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.376 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:22.376 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.638029+0000", 2024-04-26T03:39:22.376 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.1498000000000001e-05 2024-04-26T03:39:22.376 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:22.376 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:22.377 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:22.377 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.377 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.377 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.es6 45/32 ECSubWrite(tid=63, reqid=client.4556.0:350, at_version=45'22, trim_to=0'0, roll_forward_to=43'19))", 2024-04-26T03:39:22.377 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:01.637512+0000", 2024-04-26T03:39:22.377 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 20.666370639, 2024-04-26T03:39:22.377 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0059848239999999997, 2024-04-26T03:39:22.377 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:22.377 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:22.377 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:22.377 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:22.377 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.377 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:22.378 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.637512+0000", 2024-04-26T03:39:22.378 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:22.378 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.378 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.378 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:22.378 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.637512+0000", 2024-04-26T03:39:22.378 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:22.378 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.378 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.378 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:22.378 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.637513+0000", 2024-04-26T03:39:22.378 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.001e-06 2024-04-26T03:39:22.378 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.378 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.379 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:22.379 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.638154+0000", 2024-04-26T03:39:22.379 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00064122099999999998 2024-04-26T03:39:22.379 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.379 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.379 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:22.379 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.638164+0000", 2024-04-26T03:39:22.379 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0325e-05 2024-04-26T03:39:22.379 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.379 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.379 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:22.379 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.638170+0000", 2024-04-26T03:39:22.379 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.6980000000000003e-06 2024-04-26T03:39:22.379 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.380 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.380 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:22.380 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.638223+0000", 2024-04-26T03:39:22.380 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.2667000000000002e-05 2024-04-26T03:39:22.380 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.380 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.380 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:22.380 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.638232+0000", 2024-04-26T03:39:22.380 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.9439999999999994e-06 2024-04-26T03:39:22.380 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.380 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.380 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:22.380 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.643458+0000", 2024-04-26T03:39:22.380 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0052263850000000001 2024-04-26T03:39:22.380 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.381 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.381 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:22.381 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.643497+0000", 2024-04-26T03:39:22.381 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.8583e-05 2024-04-26T03:39:22.381 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:22.381 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:22.381 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:22.381 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.381 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.381 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.bs3 45/32 ECSubWrite(tid=68, reqid=client.4556.0:361, at_version=45'31, trim_to=0'0, roll_forward_to=43'30))", 2024-04-26T03:39:22.381 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:01.715320+0000", 2024-04-26T03:39:22.381 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 20.588562097000001, 2024-04-26T03:39:22.381 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0047019460000000003, 2024-04-26T03:39:22.381 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:22.382 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:22.382 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:22.382 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:22.382 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.382 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:22.382 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.715320+0000", 2024-04-26T03:39:22.382 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:22.382 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.382 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.382 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:22.382 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.715320+0000", 2024-04-26T03:39:22.382 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:22.382 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.382 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.383 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:22.383 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.715322+0000", 2024-04-26T03:39:22.383 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.248e-06 2024-04-26T03:39:22.383 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.383 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.383 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:22.383 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.715385+0000", 2024-04-26T03:39:22.383 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.3443999999999998e-05 2024-04-26T03:39:22.383 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.383 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.383 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:22.383 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.715406+0000", 2024-04-26T03:39:22.383 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.0573000000000001e-05 2024-04-26T03:39:22.383 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.383 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.384 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:22.384 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.715418+0000", 2024-04-26T03:39:22.384 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1960000000000001e-05 2024-04-26T03:39:22.384 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.384 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.384 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:22.384 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.715525+0000", 2024-04-26T03:39:22.384 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000107694 2024-04-26T03:39:22.384 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.384 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.384 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:22.384 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.715537+0000", 2024-04-26T03:39:22.384 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1749999999999999e-05 2024-04-26T03:39:22.384 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.385 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.385 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:22.385 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.719962+0000", 2024-04-26T03:39:22.385 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0044252349999999996 2024-04-26T03:39:22.385 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.385 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.385 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:22.385 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.720022+0000", 2024-04-26T03:39:22.385 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.0041999999999999e-05 2024-04-26T03:39:22.385 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:22.385 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:22.385 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:22.385 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.385 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.385 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.bs3 60/32 ECSubWrite(tid=128, reqid=client.4556.0:685, at_version=60'75, trim_to=0'0, roll_forward_to=60'73))", 2024-04-26T03:39:22.386 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:15.940662+0000", 2024-04-26T03:39:22.386 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 6.3632206929999997, 2024-04-26T03:39:22.386 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0069063650000000002, 2024-04-26T03:39:22.386 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:22.386 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:22.386 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:22.386 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:22.386 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.386 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:22.386 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.940662+0000", 2024-04-26T03:39:22.386 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:22.386 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.386 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.387 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:22.387 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.940662+0000", 2024-04-26T03:39:22.387 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:22.387 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.387 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.387 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:22.387 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.940663+0000", 2024-04-26T03:39:22.387 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.037e-06 2024-04-26T03:39:22.387 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.387 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.387 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:22.387 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.940747+0000", 2024-04-26T03:39:22.387 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.3844000000000006e-05 2024-04-26T03:39:22.387 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.387 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.388 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:22.388 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.940763+0000", 2024-04-26T03:39:22.388 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.6506000000000001e-05 2024-04-26T03:39:22.388 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.388 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.388 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:22.388 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.940772+0000", 2024-04-26T03:39:22.388 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.4100000000000008e-06 2024-04-26T03:39:22.388 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.388 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.388 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:22.388 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.940861+0000", 2024-04-26T03:39:22.388 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.9896999999999997e-05 2024-04-26T03:39:22.388 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.388 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.389 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:22.389 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.940873+0000", 2024-04-26T03:39:22.389 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1344e-05 2024-04-26T03:39:22.389 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.389 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.389 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:22.389 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.947531+0000", 2024-04-26T03:39:22.389 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0066581390000000004 2024-04-26T03:39:22.389 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.389 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.389 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:22.389 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.947568+0000", 2024-04-26T03:39:22.389 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.7187999999999997e-05 2024-04-26T03:39:22.389 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:22.390 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:22.390 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:22.390 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.390 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.390 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.bs3 65/32 ECSubWrite(tid=135, reqid=client.4556.0:745, at_version=65'81, trim_to=0'0, roll_forward_to=64'79))", 2024-04-26T03:39:22.390 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:20.715978+0000", 2024-04-26T03:39:22.390 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 1.587904268, 2024-04-26T03:39:22.390 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.007376657, 2024-04-26T03:39:22.390 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:22.390 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:22.390 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:22.390 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:22.390 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.391 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:22.391 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.715978+0000", 2024-04-26T03:39:22.391 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:22.391 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.391 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.391 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:22.391 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.715978+0000", 2024-04-26T03:39:22.391 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:22.391 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.391 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.391 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:22.391 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.715978+0000", 2024-04-26T03:39:22.391 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.4499999999999998e-07 2024-04-26T03:39:22.391 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.392 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.392 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:22.392 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.716165+0000", 2024-04-26T03:39:22.392 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0001863 2024-04-26T03:39:22.392 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.392 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.392 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:22.392 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.716178+0000", 2024-04-26T03:39:22.392 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.3005e-05 2024-04-26T03:39:22.392 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.392 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.392 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:22.392 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.716187+0000", 2024-04-26T03:39:22.392 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.5279999999999992e-06 2024-04-26T03:39:22.392 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.393 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.393 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:22.393 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.716320+0000", 2024-04-26T03:39:22.393 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00013288899999999999 2024-04-26T03:39:22.393 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.393 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.393 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:22.393 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.716332+0000", 2024-04-26T03:39:22.393 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1997000000000001e-05 2024-04-26T03:39:22.393 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.393 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.393 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:22.393 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.723322+0000", 2024-04-26T03:39:22.393 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0069894759999999997 2024-04-26T03:39:22.393 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.394 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.394 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:22.394 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.723355+0000", 2024-04-26T03:39:22.394 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.3216999999999997e-05 2024-04-26T03:39:22.394 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:22.394 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:22.394 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:22.394 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.394 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.394 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.bs3 65/32 ECSubWrite(tid=137, reqid=client.4556.0:747, at_version=65'83, trim_to=0'0, roll_forward_to=65'81))", 2024-04-26T03:39:22.394 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:20.728761+0000", 2024-04-26T03:39:22.394 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 1.575121532, 2024-04-26T03:39:22.394 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0055263029999999998, 2024-04-26T03:39:22.394 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:22.395 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:22.395 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:22.395 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:22.395 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.395 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:22.395 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.728761+0000", 2024-04-26T03:39:22.395 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:22.395 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.395 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.395 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:22.395 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.728761+0000", 2024-04-26T03:39:22.395 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:22.395 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.395 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.395 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:22.396 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.728764+0000", 2024-04-26T03:39:22.396 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.2490000000000001e-06 2024-04-26T03:39:22.396 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.396 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.396 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:22.396 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.728897+0000", 2024-04-26T03:39:22.396 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00013275900000000001 2024-04-26T03:39:22.396 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.396 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.396 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:22.396 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.728914+0000", 2024-04-26T03:39:22.396 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.7187e-05 2024-04-26T03:39:22.396 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.396 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.396 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:22.397 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.728923+0000", 2024-04-26T03:39:22.397 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.3920000000000003e-06 2024-04-26T03:39:22.397 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.397 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.397 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:22.397 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.728996+0000", 2024-04-26T03:39:22.397 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.3875000000000007e-05 2024-04-26T03:39:22.397 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.397 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.397 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:22.397 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.729007+0000", 2024-04-26T03:39:22.397 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0916e-05 2024-04-26T03:39:22.397 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.397 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.397 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:22.398 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.734257+0000", 2024-04-26T03:39:22.398 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0052494409999999997 2024-04-26T03:39:22.398 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.398 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.398 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:22.398 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.734287+0000", 2024-04-26T03:39:22.398 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.0484000000000002e-05 2024-04-26T03:39:22.398 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:22.398 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:22.398 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:22.398 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.398 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.398 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.es6 65/32 ECSubWrite(tid=127, reqid=client.4556.0:768, at_version=65'50, trim_to=0'0, roll_forward_to=63'48))", 2024-04-26T03:39:22.398 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:20.765003+0000", 2024-04-26T03:39:22.399 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 1.538879449, 2024-04-26T03:39:22.399 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0053955619999999996, 2024-04-26T03:39:22.399 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:22.399 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:22.399 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:22.399 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:22.399 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.399 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:22.399 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.765003+0000", 2024-04-26T03:39:22.399 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:22.399 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.399 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.399 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:22.399 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.765003+0000", 2024-04-26T03:39:22.399 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:22.400 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.400 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.400 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:22.400 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.765003+0000", 2024-04-26T03:39:22.400 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.4999999999999998e-07 2024-04-26T03:39:22.400 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.400 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.400 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:22.400 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.765522+0000", 2024-04-26T03:39:22.400 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00051831700000000002 2024-04-26T03:39:22.400 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.400 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.400 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:22.401 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.765533+0000", 2024-04-26T03:39:22.401 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1294000000000001e-05 2024-04-26T03:39:22.401 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.401 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.401 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:22.401 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.765540+0000", 2024-04-26T03:39:22.401 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.6159999999999998e-06 2024-04-26T03:39:22.401 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.401 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.401 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:22.401 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.765605+0000", 2024-04-26T03:39:22.401 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.5735999999999995e-05 2024-04-26T03:39:22.401 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.401 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.401 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:22.402 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.765618+0000", 2024-04-26T03:39:22.402 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.2792000000000001e-05 2024-04-26T03:39:22.402 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.402 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.402 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:22.402 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.770374+0000", 2024-04-26T03:39:22.402 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0047560329999999998 2024-04-26T03:39:22.402 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.402 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.402 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:22.402 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.770399+0000", 2024-04-26T03:39:22.402 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.4423999999999999e-05 2024-04-26T03:39:22.402 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:22.403 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:22.403 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:22.403 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.403 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.403 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.es6 65/32 ECSubWrite(tid=129, reqid=client.4556.0:770, at_version=65'52, trim_to=0'0, roll_forward_to=65'50))", 2024-04-26T03:39:22.403 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:20.779240+0000", 2024-04-26T03:39:22.403 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 1.5246424410000001, 2024-04-26T03:39:22.403 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0055201820000000002, 2024-04-26T03:39:22.403 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:22.403 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:22.403 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:22.403 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:22.403 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.403 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:22.403 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.779240+0000", 2024-04-26T03:39:22.404 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:22.404 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.404 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.404 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:22.404 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.779240+0000", 2024-04-26T03:39:22.404 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:22.404 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.404 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.404 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:22.404 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.779241+0000", 2024-04-26T03:39:22.404 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.4600000000000004e-07 2024-04-26T03:39:22.404 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.404 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.404 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:22.405 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.779790+0000", 2024-04-26T03:39:22.405 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00054915699999999999 2024-04-26T03:39:22.405 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.405 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.405 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:22.405 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.779801+0000", 2024-04-26T03:39:22.405 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0847e-05 2024-04-26T03:39:22.405 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.405 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.405 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:22.405 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.779806+0000", 2024-04-26T03:39:22.405 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.2449999999999998e-06 2024-04-26T03:39:22.405 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.405 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.406 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:22.406 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.779853+0000", 2024-04-26T03:39:22.406 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.6592999999999999e-05 2024-04-26T03:39:22.406 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.406 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.406 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:22.406 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.779858+0000", 2024-04-26T03:39:22.406 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.7350000000000001e-06 2024-04-26T03:39:22.406 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.406 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.406 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:22.406 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.784731+0000", 2024-04-26T03:39:22.406 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0048723680000000002 2024-04-26T03:39:22.406 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:22.407 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:22.407 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:22.407 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.784760+0000", 2024-04-26T03:39:22.407 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.9590999999999998e-05 2024-04-26T03:39:22.407 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:22.407 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:22.407 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:22.407 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:22.407 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:22.407 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:39:22.407 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-26T03:39:22.409 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-26T03:39:22.410 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:39:22.408+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:39:22.510 INFO:tasks.ceph.osd.14.smithi066.stderr:2024-04-26T03:39:22.509+0000 7f5395f08640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 31765) UID: 0 2024-04-26T03:39:22.529 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-26T03:39:22.529 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-04-26T03:39:22.529 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-04-26T03:39:22.529 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-26T03:39:22.538 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-26T03:39:22.573 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0: {} 2024-04-26T03:39:22.574 INFO:teuthology.orchestra.run.smithi028.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T03:39:22.592 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1: {} 2024-04-26T03:39:22.592 INFO:teuthology.orchestra.run.smithi028.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T03:39:22.611 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2: {} 2024-04-26T03:39:22.611 INFO:teuthology.orchestra.run.smithi028.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T03:39:22.611 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-04-26T03:39:22.610+0000 7fee562fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31759) UID: 0 2024-04-26T03:39:22.628 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3: {} 2024-04-26T03:39:22.629 INFO:teuthology.orchestra.run.smithi028.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T03:39:22.646 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4: {} 2024-04-26T03:39:22.646 INFO:teuthology.orchestra.run.smithi028.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T03:39:22.646 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:22.646 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:22.655 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-26T03:39:22.663 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5: {} 2024-04-26T03:39:22.663 INFO:teuthology.orchestra.run.smithi028.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T03:39:22.681 INFO:teuthology.orchestra.run.smithi028.stdout:osd.6: {} 2024-04-26T03:39:22.682 INFO:teuthology.orchestra.run.smithi028.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T03:39:22.693 INFO:tasks.rados.rados.0.smithi092.stdout:299: done (1 left) 2024-04-26T03:39:22.693 INFO:tasks.rados.rados.0.smithi092.stdout:300: done (0 left) 2024-04-26T03:39:22.693 INFO:tasks.rados.rados.0.smithi092.stdout:301: read oid 1 snap 16 2024-04-26T03:39:22.693 INFO:tasks.rados.rados.0.smithi092.stdout:301: expect deleted 2024-04-26T03:39:22.693 INFO:tasks.rados.rados.0.smithi092.stdout:302: read oid 3 snap 10 2024-04-26T03:39:22.693 INFO:tasks.rados.rados.0.smithi092.stdout:302: expect (ObjNum 108 snap 8 seq_num 108) 2024-04-26T03:39:22.693 INFO:tasks.rados.rados.0.smithi092.stdout:303: read oid 18 snap 18 2024-04-26T03:39:22.693 INFO:tasks.rados.rados.0.smithi092.stdout:303: expect (ObjNum 124 snap 14 seq_num 124) 2024-04-26T03:39:22.693 INFO:tasks.rados.rados.0.smithi092.stdout:304: read oid 10 snap -1 2024-04-26T03:39:22.694 INFO:tasks.rados.rados.0.smithi092.stdout:304: expect (ObjNum 90 snap 7 seq_num 90) 2024-04-26T03:39:22.694 INFO:tasks.rados.rados.0.smithi092.stdout:305: delete oid 31 current snap is 19 2024-04-26T03:39:22.700 INFO:tasks.rados.rados.0.smithi092.stdout:301: done (4 left) 2024-04-26T03:39:22.700 INFO:tasks.rados.rados.0.smithi092.stdout:305: done (3 left) 2024-04-26T03:39:22.700 INFO:tasks.rados.rados.0.smithi092.stdout:306: rmattr oid 42 current snap is 19 2024-04-26T03:39:22.700 INFO:tasks.rados.rados.0.smithi092.stdout:306: done (3 left) 2024-04-26T03:39:22.700 INFO:tasks.rados.rados.0.smithi092.stdout:307: delete oid 20 current snap is 19 2024-04-26T03:39:22.702 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-26T03:39:22.706 INFO:tasks.rados.rados.0.smithi092.stdout:307: done (3 left) 2024-04-26T03:39:22.706 INFO:tasks.rados.rados.0.smithi092.stdout:308: rollback oid 6 current snap is 19 2024-04-26T03:39:22.706 INFO:tasks.rados.rados.0.smithi092.stdout:rollback oid 6 to 16 2024-04-26T03:39:22.706 INFO:tasks.rados.rados.0.smithi092.stdout:309: delete oid 19 current snap is 19 2024-04-26T03:39:22.707 INFO:teuthology.orchestra.run.smithi028.stdout:osd.7: {} 2024-04-26T03:39:22.708 INFO:teuthology.orchestra.run.smithi028.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T03:39:22.710 INFO:tasks.rados.rados.0.smithi092.stdout:302: expect (ObjNum 108 snap 8 seq_num 108) 2024-04-26T03:39:22.712 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:39:22.710+0000 7fba0d2ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31961) UID: 0 2024-04-26T03:39:22.726 INFO:teuthology.orchestra.run.smithi028.stdout:osd.8: {} 2024-04-26T03:39:22.726 INFO:teuthology.orchestra.run.smithi028.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T03:39:22.744 INFO:teuthology.orchestra.run.smithi028.stdout:osd.9: {} 2024-04-26T03:39:22.744 INFO:teuthology.orchestra.run.smithi028.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T03:39:22.750 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-04-26T03:39:22.751 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-26T03:39:22.751 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-04-26T03:39:22.751 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-04-26T03:39:22.761 INFO:tasks.rados.rados.0.smithi092.stdout:303: expect (ObjNum 124 snap 14 seq_num 124) 2024-04-26T03:39:22.764 INFO:teuthology.orchestra.run.smithi028.stdout:osd.10: {} 2024-04-26T03:39:22.764 INFO:teuthology.orchestra.run.smithi028.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T03:39:22.767 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:22.768 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:22.777 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-26T03:39:22.782 INFO:teuthology.orchestra.run.smithi028.stdout:osd.11: {} 2024-04-26T03:39:22.782 INFO:teuthology.orchestra.run.smithi028.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T03:39:22.800 INFO:teuthology.orchestra.run.smithi028.stdout:osd.12: {} 2024-04-26T03:39:22.800 INFO:teuthology.orchestra.run.smithi028.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T03:39:22.812 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-04-26T03:39:22.811+0000 7fdc2d21c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31819) UID: 0 2024-04-26T03:39:22.815 INFO:tasks.rados.rados.0.smithi092.stdout:308: finishing rollback tid 1 to smithi09235267-6 2024-04-26T03:39:22.815 INFO:tasks.rados.rados.0.smithi092.stdout:308: finishing rollback tid 2 to smithi09235267-6 2024-04-26T03:39:22.815 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 6 v 61 (ObjNum 102 snap 8 seq_num 102) dirty exists 2024-04-26T03:39:22.815 INFO:tasks.rados.rados.0.smithi092.stdout:304: expect (ObjNum 90 snap 7 seq_num 90) 2024-04-26T03:39:22.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:39:22.818+0000 7f3ab5efc640 0 -- 172.21.15.28:0/3961176556 >> [v2:172.21.15.42:6817/301360893,v1:172.21.15.42:6819/301360893] conn(0x7f3ab021cbd0 msgr2=0x7f3ab021efc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-26T03:39:22.820 INFO:teuthology.orchestra.run.smithi028.stdout:osd.13: {} 2024-04-26T03:39:22.820 INFO:teuthology.orchestra.run.smithi028.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T03:39:22.838 INFO:teuthology.orchestra.run.smithi028.stdout:osd.14: {} 2024-04-26T03:39:22.839 INFO:teuthology.orchestra.run.smithi028.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T03:39:22.857 INFO:teuthology.orchestra.run.smithi028.stdout:osd.15: {} 2024-04-26T03:39:22.857 INFO:teuthology.orchestra.run.smithi028.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T03:39:22.884 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:22.884 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:22.893 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-26T03:39:22.901 INFO:tasks.rados.rados.0.smithi092.stdout:302: done (4 left) 2024-04-26T03:39:22.901 INFO:tasks.rados.rados.0.smithi092.stdout:303: done (3 left) 2024-04-26T03:39:22.902 INFO:tasks.rados.rados.0.smithi092.stdout:304: done (2 left) 2024-04-26T03:39:22.902 INFO:tasks.rados.rados.0.smithi092.stdout:308: done (1 left) 2024-04-26T03:39:22.902 INFO:tasks.rados.rados.0.smithi092.stdout:309: done (0 left) 2024-04-26T03:39:22.902 INFO:tasks.rados.rados.0.smithi092.stdout:310: snap_create 2024-04-26T03:39:22.913 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-04-26T03:39:22.912+0000 7fdc2d21c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31819) UID: 0 2024-04-26T03:39:22.999 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:22.999 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:23.008 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-26T03:39:23.014 INFO:tasks.ceph.osd.8.smithi028.stderr:2024-04-26T03:39:23.012+0000 7f5bb75d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 31968) UID: 0 2024-04-26T03:39:23.094 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:39:23.095 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:39:23.095 INFO:teuthology.orchestra.run.smithi028.stdout: "epoch": 67, 2024-04-26T03:39:23.095 INFO:teuthology.orchestra.run.smithi028.stdout: "fsid": "c5a13c99-5c9a-4b8b-a893-4f6f4211677a", 2024-04-26T03:39:23.095 INFO:teuthology.orchestra.run.smithi028.stdout: "created": "2024-04-26T03:38:19.896648+0000", 2024-04-26T03:39:23.095 INFO:teuthology.orchestra.run.smithi028.stdout: "modified": "2024-04-26T03:39:22.686418+0000", 2024-04-26T03:39:23.095 INFO:teuthology.orchestra.run.smithi028.stdout: "last_up_change": "2024-04-26T03:38:33.308561+0000", 2024-04-26T03:39:23.095 INFO:teuthology.orchestra.run.smithi028.stdout: "last_in_change": "2024-04-26T03:38:24.150336+0000", 2024-04-26T03:39:23.095 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-26T03:39:23.095 INFO:teuthology.orchestra.run.smithi028.stdout: "flags_num": 5799936, 2024-04-26T03:39:23.095 INFO:teuthology.orchestra.run.smithi028.stdout: "flags_set": [ 2024-04-26T03:39:23.095 INFO:teuthology.orchestra.run.smithi028.stdout: "pglog_hardlimit", 2024-04-26T03:39:23.095 INFO:teuthology.orchestra.run.smithi028.stdout: "purged_snapdirs", 2024-04-26T03:39:23.096 INFO:teuthology.orchestra.run.smithi028.stdout: "recovery_deletes", 2024-04-26T03:39:23.096 INFO:teuthology.orchestra.run.smithi028.stdout: "sortbitwise" 2024-04-26T03:39:23.096 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-04-26T03:39:23.096 INFO:teuthology.orchestra.run.smithi028.stdout: "crush_version": 6, 2024-04-26T03:39:23.096 INFO:teuthology.orchestra.run.smithi028.stdout: "full_ratio": 0.94999998807907104, 2024-04-26T03:39:23.096 INFO:teuthology.orchestra.run.smithi028.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-04-26T03:39:23.096 INFO:teuthology.orchestra.run.smithi028.stdout: "nearfull_ratio": 0.85000002384185791, 2024-04-26T03:39:23.096 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_snapshot": "", 2024-04-26T03:39:23.096 INFO:teuthology.orchestra.run.smithi028.stdout: "pool_max": 3, 2024-04-26T03:39:23.096 INFO:teuthology.orchestra.run.smithi028.stdout: "max_osd": 16, 2024-04-26T03:39:23.096 INFO:teuthology.orchestra.run.smithi028.stdout: "require_min_compat_client": "luminous", 2024-04-26T03:39:23.096 INFO:teuthology.orchestra.run.smithi028.stdout: "min_compat_client": "luminous", 2024-04-26T03:39:23.097 INFO:teuthology.orchestra.run.smithi028.stdout: "require_osd_release": "squid", 2024-04-26T03:39:23.097 INFO:teuthology.orchestra.run.smithi028.stdout: "allow_crimson": false, 2024-04-26T03:39:23.097 INFO:teuthology.orchestra.run.smithi028.stdout: "pools": [ 2024-04-26T03:39:23.097 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.097 INFO:teuthology.orchestra.run.smithi028.stdout: "pool": 1, 2024-04-26T03:39:23.097 INFO:teuthology.orchestra.run.smithi028.stdout: "pool_name": ".mgr", 2024-04-26T03:39:23.097 INFO:teuthology.orchestra.run.smithi028.stdout: "create_time": "2024-04-26T03:38:32.306696+0000", 2024-04-26T03:39:23.097 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-04-26T03:39:23.097 INFO:teuthology.orchestra.run.smithi028.stdout: "flags_names": "hashpspool", 2024-04-26T03:39:23.097 INFO:teuthology.orchestra.run.smithi028.stdout: "type": 1, 2024-04-26T03:39:23.097 INFO:teuthology.orchestra.run.smithi028.stdout: "size": 2, 2024-04-26T03:39:23.097 INFO:teuthology.orchestra.run.smithi028.stdout: "min_size": 1, 2024-04-26T03:39:23.097 INFO:teuthology.orchestra.run.smithi028.stdout: "crush_rule": 0, 2024-04-26T03:39:23.098 INFO:teuthology.orchestra.run.smithi028.stdout: "peering_crush_bucket_count": 0, 2024-04-26T03:39:23.098 INFO:teuthology.orchestra.run.smithi028.stdout: "peering_crush_bucket_target": 0, 2024-04-26T03:39:23.098 INFO:teuthology.orchestra.run.smithi028.stdout: "peering_crush_bucket_barrier": 0, 2024-04-26T03:39:23.098 INFO:teuthology.orchestra.run.smithi028.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-26T03:39:23.098 INFO:teuthology.orchestra.run.smithi028.stdout: "object_hash": 2, 2024-04-26T03:39:23.098 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_autoscale_mode": "off", 2024-04-26T03:39:23.098 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_num": 1, 2024-04-26T03:39:23.098 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_placement_num": 1, 2024-04-26T03:39:23.098 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_placement_num_target": 1, 2024-04-26T03:39:23.098 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_num_target": 1, 2024-04-26T03:39:23.098 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_num_pending": 1, 2024-04-26T03:39:23.098 INFO:teuthology.orchestra.run.smithi028.stdout: "last_pg_merge_meta": { 2024-04-26T03:39:23.098 INFO:teuthology.orchestra.run.smithi028.stdout: "source_pgid": "0.0", 2024-04-26T03:39:23.099 INFO:teuthology.orchestra.run.smithi028.stdout: "ready_epoch": 0, 2024-04-26T03:39:23.099 INFO:teuthology.orchestra.run.smithi028.stdout: "last_epoch_started": 0, 2024-04-26T03:39:23.099 INFO:teuthology.orchestra.run.smithi028.stdout: "last_epoch_clean": 0, 2024-04-26T03:39:23.099 INFO:teuthology.orchestra.run.smithi028.stdout: "source_version": "0'0", 2024-04-26T03:39:23.099 INFO:teuthology.orchestra.run.smithi028.stdout: "target_version": "0'0" 2024-04-26T03:39:23.099 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.099 INFO:teuthology.orchestra.run.smithi028.stdout: "last_change": "26", 2024-04-26T03:39:23.099 INFO:teuthology.orchestra.run.smithi028.stdout: "last_force_op_resend": "0", 2024-04-26T03:39:23.099 INFO:teuthology.orchestra.run.smithi028.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-26T03:39:23.099 INFO:teuthology.orchestra.run.smithi028.stdout: "last_force_op_resend_preluminous": "0", 2024-04-26T03:39:23.099 INFO:teuthology.orchestra.run.smithi028.stdout: "auid": 0, 2024-04-26T03:39:23.099 INFO:teuthology.orchestra.run.smithi028.stdout: "snap_mode": "selfmanaged", 2024-04-26T03:39:23.099 INFO:teuthology.orchestra.run.smithi028.stdout: "snap_seq": 0, 2024-04-26T03:39:23.100 INFO:teuthology.orchestra.run.smithi028.stdout: "snap_epoch": 0, 2024-04-26T03:39:23.100 INFO:teuthology.orchestra.run.smithi028.stdout: "pool_snaps": [], 2024-04-26T03:39:23.100 INFO:teuthology.orchestra.run.smithi028.stdout: "removed_snaps": "[]", 2024-04-26T03:39:23.100 INFO:teuthology.orchestra.run.smithi028.stdout: "quota_max_bytes": 0, 2024-04-26T03:39:23.100 INFO:teuthology.orchestra.run.smithi028.stdout: "quota_max_objects": 0, 2024-04-26T03:39:23.100 INFO:teuthology.orchestra.run.smithi028.stdout: "tiers": [], 2024-04-26T03:39:23.100 INFO:teuthology.orchestra.run.smithi028.stdout: "tier_of": -1, 2024-04-26T03:39:23.100 INFO:teuthology.orchestra.run.smithi028.stdout: "read_tier": -1, 2024-04-26T03:39:23.100 INFO:teuthology.orchestra.run.smithi028.stdout: "write_tier": -1, 2024-04-26T03:39:23.100 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_mode": "none", 2024-04-26T03:39:23.100 INFO:teuthology.orchestra.run.smithi028.stdout: "target_max_bytes": 0, 2024-04-26T03:39:23.100 INFO:teuthology.orchestra.run.smithi028.stdout: "target_max_objects": 0, 2024-04-26T03:39:23.100 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-26T03:39:23.100 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-26T03:39:23.100 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-26T03:39:23.101 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_min_flush_age": 0, 2024-04-26T03:39:23.101 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_min_evict_age": 0, 2024-04-26T03:39:23.101 INFO:teuthology.orchestra.run.smithi028.stdout: "erasure_code_profile": "", 2024-04-26T03:39:23.101 INFO:teuthology.orchestra.run.smithi028.stdout: "hit_set_params": { 2024-04-26T03:39:23.101 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "none" 2024-04-26T03:39:23.101 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.101 INFO:teuthology.orchestra.run.smithi028.stdout: "hit_set_period": 0, 2024-04-26T03:39:23.101 INFO:teuthology.orchestra.run.smithi028.stdout: "hit_set_count": 0, 2024-04-26T03:39:23.101 INFO:teuthology.orchestra.run.smithi028.stdout: "use_gmt_hitset": true, 2024-04-26T03:39:23.101 INFO:teuthology.orchestra.run.smithi028.stdout: "min_read_recency_for_promote": 0, 2024-04-26T03:39:23.101 INFO:teuthology.orchestra.run.smithi028.stdout: "min_write_recency_for_promote": 0, 2024-04-26T03:39:23.101 INFO:teuthology.orchestra.run.smithi028.stdout: "hit_set_grade_decay_rate": 0, 2024-04-26T03:39:23.101 INFO:teuthology.orchestra.run.smithi028.stdout: "hit_set_search_last_n": 0, 2024-04-26T03:39:23.102 INFO:teuthology.orchestra.run.smithi028.stdout: "grade_table": [], 2024-04-26T03:39:23.102 INFO:teuthology.orchestra.run.smithi028.stdout: "stripe_width": 0, 2024-04-26T03:39:23.102 INFO:teuthology.orchestra.run.smithi028.stdout: "expected_num_objects": 0, 2024-04-26T03:39:23.102 INFO:teuthology.orchestra.run.smithi028.stdout: "fast_read": false, 2024-04-26T03:39:23.102 INFO:teuthology.orchestra.run.smithi028.stdout: "options": { 2024-04-26T03:39:23.102 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_num_max": 32, 2024-04-26T03:39:23.102 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_num_min": 1 2024-04-26T03:39:23.102 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.102 INFO:teuthology.orchestra.run.smithi028.stdout: "application_metadata": { 2024-04-26T03:39:23.102 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": {} 2024-04-26T03:39:23.102 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.102 INFO:teuthology.orchestra.run.smithi028.stdout: "read_balance": { 2024-04-26T03:39:23.102 INFO:teuthology.orchestra.run.smithi028.stdout: "score_type": "Fair distribution", 2024-04-26T03:39:23.103 INFO:teuthology.orchestra.run.smithi028.stdout: "score_acting": 15.380000114440918, 2024-04-26T03:39:23.103 INFO:teuthology.orchestra.run.smithi028.stdout: "score_stable": 15.380000114440918, 2024-04-26T03:39:23.103 INFO:teuthology.orchestra.run.smithi028.stdout: "optimal_score": 0.12999999523162842, 2024-04-26T03:39:23.103 INFO:teuthology.orchestra.run.smithi028.stdout: "raw_score_acting": 2, 2024-04-26T03:39:23.103 INFO:teuthology.orchestra.run.smithi028.stdout: "raw_score_stable": 2, 2024-04-26T03:39:23.103 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity_weighted": 1, 2024-04-26T03:39:23.103 INFO:teuthology.orchestra.run.smithi028.stdout: "average_primary_affinity": 1, 2024-04-26T03:39:23.103 INFO:teuthology.orchestra.run.smithi028.stdout: "average_primary_affinity_weighted": 1 2024-04-26T03:39:23.103 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.104 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.104 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.104 INFO:teuthology.orchestra.run.smithi028.stdout: "pool": 2, 2024-04-26T03:39:23.104 INFO:teuthology.orchestra.run.smithi028.stdout: "pool_name": "rbd", 2024-04-26T03:39:23.104 INFO:teuthology.orchestra.run.smithi028.stdout: "create_time": "2024-04-26T03:38:34.592633+0000", 2024-04-26T03:39:23.104 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 1, 2024-04-26T03:39:23.104 INFO:teuthology.orchestra.run.smithi028.stdout: "flags_names": "hashpspool", 2024-04-26T03:39:23.104 INFO:teuthology.orchestra.run.smithi028.stdout: "type": 1, 2024-04-26T03:39:23.104 INFO:teuthology.orchestra.run.smithi028.stdout: "size": 2, 2024-04-26T03:39:23.104 INFO:teuthology.orchestra.run.smithi028.stdout: "min_size": 1, 2024-04-26T03:39:23.104 INFO:teuthology.orchestra.run.smithi028.stdout: "crush_rule": 0, 2024-04-26T03:39:23.104 INFO:teuthology.orchestra.run.smithi028.stdout: "peering_crush_bucket_count": 0, 2024-04-26T03:39:23.104 INFO:teuthology.orchestra.run.smithi028.stdout: "peering_crush_bucket_target": 0, 2024-04-26T03:39:23.105 INFO:teuthology.orchestra.run.smithi028.stdout: "peering_crush_bucket_barrier": 0, 2024-04-26T03:39:23.105 INFO:teuthology.orchestra.run.smithi028.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-26T03:39:23.105 INFO:teuthology.orchestra.run.smithi028.stdout: "object_hash": 2, 2024-04-26T03:39:23.105 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_autoscale_mode": "off", 2024-04-26T03:39:23.105 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_num": 8, 2024-04-26T03:39:23.105 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_placement_num": 8, 2024-04-26T03:39:23.105 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_placement_num_target": 8, 2024-04-26T03:39:23.105 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_num_target": 8, 2024-04-26T03:39:23.106 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_num_pending": 8, 2024-04-26T03:39:23.106 INFO:teuthology.orchestra.run.smithi028.stdout: "last_pg_merge_meta": { 2024-04-26T03:39:23.106 INFO:teuthology.orchestra.run.smithi028.stdout: "source_pgid": "0.0", 2024-04-26T03:39:23.106 INFO:teuthology.orchestra.run.smithi028.stdout: "ready_epoch": 0, 2024-04-26T03:39:23.106 INFO:teuthology.orchestra.run.smithi028.stdout: "last_epoch_started": 0, 2024-04-26T03:39:23.106 INFO:teuthology.orchestra.run.smithi028.stdout: "last_epoch_clean": 0, 2024-04-26T03:39:23.106 INFO:teuthology.orchestra.run.smithi028.stdout: "source_version": "0'0", 2024-04-26T03:39:23.106 INFO:teuthology.orchestra.run.smithi028.stdout: "target_version": "0'0" 2024-04-26T03:39:23.107 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.107 INFO:teuthology.orchestra.run.smithi028.stdout: "last_change": "29", 2024-04-26T03:39:23.107 INFO:teuthology.orchestra.run.smithi028.stdout: "last_force_op_resend": "0", 2024-04-26T03:39:23.107 INFO:teuthology.orchestra.run.smithi028.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-26T03:39:23.107 INFO:teuthology.orchestra.run.smithi028.stdout: "last_force_op_resend_preluminous": "0", 2024-04-26T03:39:23.107 INFO:teuthology.orchestra.run.smithi028.stdout: "auid": 0, 2024-04-26T03:39:23.107 INFO:teuthology.orchestra.run.smithi028.stdout: "snap_mode": "selfmanaged", 2024-04-26T03:39:23.107 INFO:teuthology.orchestra.run.smithi028.stdout: "snap_seq": 0, 2024-04-26T03:39:23.108 INFO:teuthology.orchestra.run.smithi028.stdout: "snap_epoch": 0, 2024-04-26T03:39:23.108 INFO:teuthology.orchestra.run.smithi028.stdout: "pool_snaps": [], 2024-04-26T03:39:23.108 INFO:teuthology.orchestra.run.smithi028.stdout: "removed_snaps": "[]", 2024-04-26T03:39:23.108 INFO:teuthology.orchestra.run.smithi028.stdout: "quota_max_bytes": 0, 2024-04-26T03:39:23.108 INFO:teuthology.orchestra.run.smithi028.stdout: "quota_max_objects": 0, 2024-04-26T03:39:23.108 INFO:teuthology.orchestra.run.smithi028.stdout: "tiers": [], 2024-04-26T03:39:23.108 INFO:teuthology.orchestra.run.smithi028.stdout: "tier_of": -1, 2024-04-26T03:39:23.108 INFO:teuthology.orchestra.run.smithi028.stdout: "read_tier": -1, 2024-04-26T03:39:23.109 INFO:teuthology.orchestra.run.smithi028.stdout: "write_tier": -1, 2024-04-26T03:39:23.109 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_mode": "none", 2024-04-26T03:39:23.109 INFO:teuthology.orchestra.run.smithi028.stdout: "target_max_bytes": 0, 2024-04-26T03:39:23.109 INFO:teuthology.orchestra.run.smithi028.stdout: "target_max_objects": 0, 2024-04-26T03:39:23.109 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-26T03:39:23.109 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-26T03:39:23.109 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-26T03:39:23.109 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_min_flush_age": 0, 2024-04-26T03:39:23.109 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_min_evict_age": 0, 2024-04-26T03:39:23.110 INFO:teuthology.orchestra.run.smithi028.stdout: "erasure_code_profile": "", 2024-04-26T03:39:23.110 INFO:teuthology.orchestra.run.smithi028.stdout: "hit_set_params": { 2024-04-26T03:39:23.110 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "none" 2024-04-26T03:39:23.110 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.110 INFO:teuthology.orchestra.run.smithi028.stdout: "hit_set_period": 0, 2024-04-26T03:39:23.110 INFO:teuthology.orchestra.run.smithi028.stdout: "hit_set_count": 0, 2024-04-26T03:39:23.110 INFO:teuthology.orchestra.run.smithi028.stdout: "use_gmt_hitset": true, 2024-04-26T03:39:23.111 INFO:teuthology.orchestra.run.smithi028.stdout: "min_read_recency_for_promote": 0, 2024-04-26T03:39:23.111 INFO:teuthology.orchestra.run.smithi028.stdout: "min_write_recency_for_promote": 0, 2024-04-26T03:39:23.111 INFO:teuthology.orchestra.run.smithi028.stdout: "hit_set_grade_decay_rate": 0, 2024-04-26T03:39:23.111 INFO:teuthology.orchestra.run.smithi028.stdout: "hit_set_search_last_n": 0, 2024-04-26T03:39:23.111 INFO:teuthology.orchestra.run.smithi028.stdout: "grade_table": [], 2024-04-26T03:39:23.111 INFO:teuthology.orchestra.run.smithi028.stdout: "stripe_width": 0, 2024-04-26T03:39:23.111 INFO:teuthology.orchestra.run.smithi028.stdout: "expected_num_objects": 0, 2024-04-26T03:39:23.111 INFO:teuthology.orchestra.run.smithi028.stdout: "fast_read": false, 2024-04-26T03:39:23.112 INFO:teuthology.orchestra.run.smithi028.stdout: "options": {}, 2024-04-26T03:39:23.112 INFO:teuthology.orchestra.run.smithi028.stdout: "application_metadata": { 2024-04-26T03:39:23.112 INFO:teuthology.orchestra.run.smithi028.stdout: "rbd": {} 2024-04-26T03:39:23.112 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.112 INFO:teuthology.orchestra.run.smithi028.stdout: "read_balance": { 2024-04-26T03:39:23.112 INFO:teuthology.orchestra.run.smithi028.stdout: "score_type": "Fair distribution", 2024-04-26T03:39:23.112 INFO:teuthology.orchestra.run.smithi028.stdout: "score_acting": 4, 2024-04-26T03:39:23.112 INFO:teuthology.orchestra.run.smithi028.stdout: "score_stable": 4, 2024-04-26T03:39:23.113 INFO:teuthology.orchestra.run.smithi028.stdout: "optimal_score": 0.75, 2024-04-26T03:39:23.113 INFO:teuthology.orchestra.run.smithi028.stdout: "raw_score_acting": 3, 2024-04-26T03:39:23.113 INFO:teuthology.orchestra.run.smithi028.stdout: "raw_score_stable": 3, 2024-04-26T03:39:23.113 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity_weighted": 1, 2024-04-26T03:39:23.113 INFO:teuthology.orchestra.run.smithi028.stdout: "average_primary_affinity": 1, 2024-04-26T03:39:23.113 INFO:teuthology.orchestra.run.smithi028.stdout: "average_primary_affinity_weighted": 1 2024-04-26T03:39:23.113 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.113 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.113 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.114 INFO:teuthology.orchestra.run.smithi028.stdout: "pool": 3, 2024-04-26T03:39:23.114 INFO:teuthology.orchestra.run.smithi028.stdout: "pool_name": "unique_pool_0", 2024-04-26T03:39:23.114 INFO:teuthology.orchestra.run.smithi028.stdout: "create_time": "2024-04-26T03:38:47.497176+0000", 2024-04-26T03:39:23.114 INFO:teuthology.orchestra.run.smithi028.stdout: "flags": 8193, 2024-04-26T03:39:23.114 INFO:teuthology.orchestra.run.smithi028.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-04-26T03:39:23.114 INFO:teuthology.orchestra.run.smithi028.stdout: "type": 3, 2024-04-26T03:39:23.114 INFO:teuthology.orchestra.run.smithi028.stdout: "size": 7, 2024-04-26T03:39:23.114 INFO:teuthology.orchestra.run.smithi028.stdout: "min_size": 5, 2024-04-26T03:39:23.115 INFO:teuthology.orchestra.run.smithi028.stdout: "crush_rule": 1, 2024-04-26T03:39:23.115 INFO:teuthology.orchestra.run.smithi028.stdout: "peering_crush_bucket_count": 0, 2024-04-26T03:39:23.115 INFO:teuthology.orchestra.run.smithi028.stdout: "peering_crush_bucket_target": 0, 2024-04-26T03:39:23.115 INFO:teuthology.orchestra.run.smithi028.stdout: "peering_crush_bucket_barrier": 0, 2024-04-26T03:39:23.115 INFO:teuthology.orchestra.run.smithi028.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-26T03:39:23.115 INFO:teuthology.orchestra.run.smithi028.stdout: "object_hash": 2, 2024-04-26T03:39:23.115 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_autoscale_mode": "off", 2024-04-26T03:39:23.115 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_num": 23, 2024-04-26T03:39:23.116 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_placement_num": 20, 2024-04-26T03:39:23.116 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_placement_num_target": 16, 2024-04-26T03:39:23.116 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_num_target": 16, 2024-04-26T03:39:23.116 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_num_pending": 23, 2024-04-26T03:39:23.116 INFO:teuthology.orchestra.run.smithi028.stdout: "last_pg_merge_meta": { 2024-04-26T03:39:23.116 INFO:teuthology.orchestra.run.smithi028.stdout: "source_pgid": "3.17", 2024-04-26T03:39:23.116 INFO:teuthology.orchestra.run.smithi028.stdout: "ready_epoch": 67, 2024-04-26T03:39:23.116 INFO:teuthology.orchestra.run.smithi028.stdout: "last_epoch_started": 66, 2024-04-26T03:39:23.117 INFO:teuthology.orchestra.run.smithi028.stdout: "last_epoch_clean": 66, 2024-04-26T03:39:23.117 INFO:teuthology.orchestra.run.smithi028.stdout: "source_version": "43'6", 2024-04-26T03:39:23.117 INFO:teuthology.orchestra.run.smithi028.stdout: "target_version": "49'10" 2024-04-26T03:39:23.117 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.117 INFO:teuthology.orchestra.run.smithi028.stdout: "last_change": "67", 2024-04-26T03:39:23.117 INFO:teuthology.orchestra.run.smithi028.stdout: "last_force_op_resend": "0", 2024-04-26T03:39:23.117 INFO:teuthology.orchestra.run.smithi028.stdout: "last_force_op_resend_prenautilus": "67", 2024-04-26T03:39:23.118 INFO:teuthology.orchestra.run.smithi028.stdout: "last_force_op_resend_preluminous": "65", 2024-04-26T03:39:23.118 INFO:teuthology.orchestra.run.smithi028.stdout: "auid": 0, 2024-04-26T03:39:23.118 INFO:teuthology.orchestra.run.smithi028.stdout: "snap_mode": "selfmanaged", 2024-04-26T03:39:23.118 INFO:teuthology.orchestra.run.smithi028.stdout: "snap_seq": 20, 2024-04-26T03:39:23.118 INFO:teuthology.orchestra.run.smithi028.stdout: "snap_epoch": 67, 2024-04-26T03:39:23.118 INFO:teuthology.orchestra.run.smithi028.stdout: "pool_snaps": [], 2024-04-26T03:39:23.118 INFO:teuthology.orchestra.run.smithi028.stdout: "removed_snaps": "[]", 2024-04-26T03:39:23.119 INFO:teuthology.orchestra.run.smithi028.stdout: "quota_max_bytes": 0, 2024-04-26T03:39:23.119 INFO:teuthology.orchestra.run.smithi028.stdout: "quota_max_objects": 0, 2024-04-26T03:39:23.119 INFO:teuthology.orchestra.run.smithi028.stdout: "tiers": [], 2024-04-26T03:39:23.119 INFO:teuthology.orchestra.run.smithi028.stdout: "tier_of": -1, 2024-04-26T03:39:23.119 INFO:teuthology.orchestra.run.smithi028.stdout: "read_tier": -1, 2024-04-26T03:39:23.119 INFO:teuthology.orchestra.run.smithi028.stdout: "write_tier": -1, 2024-04-26T03:39:23.119 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_mode": "none", 2024-04-26T03:39:23.119 INFO:teuthology.orchestra.run.smithi028.stdout: "target_max_bytes": 0, 2024-04-26T03:39:23.119 INFO:teuthology.orchestra.run.smithi028.stdout: "target_max_objects": 0, 2024-04-26T03:39:23.119 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-26T03:39:23.119 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-26T03:39:23.119 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-26T03:39:23.120 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_min_flush_age": 0, 2024-04-26T03:39:23.120 INFO:teuthology.orchestra.run.smithi028.stdout: "cache_min_evict_age": 0, 2024-04-26T03:39:23.120 INFO:teuthology.orchestra.run.smithi028.stdout: "erasure_code_profile": "shecprofile", 2024-04-26T03:39:23.120 INFO:teuthology.orchestra.run.smithi028.stdout: "hit_set_params": { 2024-04-26T03:39:23.120 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "none" 2024-04-26T03:39:23.120 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.120 INFO:teuthology.orchestra.run.smithi028.stdout: "hit_set_period": 0, 2024-04-26T03:39:23.120 INFO:teuthology.orchestra.run.smithi028.stdout: "hit_set_count": 0, 2024-04-26T03:39:23.120 INFO:teuthology.orchestra.run.smithi028.stdout: "use_gmt_hitset": true, 2024-04-26T03:39:23.120 INFO:teuthology.orchestra.run.smithi028.stdout: "min_read_recency_for_promote": 0, 2024-04-26T03:39:23.120 INFO:teuthology.orchestra.run.smithi028.stdout: "min_write_recency_for_promote": 0, 2024-04-26T03:39:23.120 INFO:teuthology.orchestra.run.smithi028.stdout: "hit_set_grade_decay_rate": 0, 2024-04-26T03:39:23.120 INFO:teuthology.orchestra.run.smithi028.stdout: "hit_set_search_last_n": 0, 2024-04-26T03:39:23.121 INFO:teuthology.orchestra.run.smithi028.stdout: "grade_table": [], 2024-04-26T03:39:23.121 INFO:teuthology.orchestra.run.smithi028.stdout: "stripe_width": 16384, 2024-04-26T03:39:23.121 INFO:teuthology.orchestra.run.smithi028.stdout: "expected_num_objects": 0, 2024-04-26T03:39:23.121 INFO:teuthology.orchestra.run.smithi028.stdout: "fast_read": false, 2024-04-26T03:39:23.121 INFO:teuthology.orchestra.run.smithi028.stdout: "options": {}, 2024-04-26T03:39:23.121 INFO:teuthology.orchestra.run.smithi028.stdout: "application_metadata": { 2024-04-26T03:39:23.121 INFO:teuthology.orchestra.run.smithi028.stdout: "rados": {} 2024-04-26T03:39:23.121 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.121 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.121 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-04-26T03:39:23.121 INFO:teuthology.orchestra.run.smithi028.stdout: "osds": [ 2024-04-26T03:39:23.121 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.121 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 0, 2024-04-26T03:39:23.121 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "31b0e7a9-455d-4874-9c60-26efac8d0b1e", 2024-04-26T03:39:23.122 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:39:23.122 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:39:23.122 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:39:23.122 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:39:23.122 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:39:23.122 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:39:23.122 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 23, 2024-04-26T03:39:23.122 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 44, 2024-04-26T03:39:23.122 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:39:23.122 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:39:23.122 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:39:23.122 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.122 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.123 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.123 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6800", 2024-04-26T03:39:23.123 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377656090 2024-04-26T03:39:23.123 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.123 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.123 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.123 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6801", 2024-04-26T03:39:23.123 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377656090 2024-04-26T03:39:23.123 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.123 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.123 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.123 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:39:23.123 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.124 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.124 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.124 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6802", 2024-04-26T03:39:23.124 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377656090 2024-04-26T03:39:23.124 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.124 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.124 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.124 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6803", 2024-04-26T03:39:23.124 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377656090 2024-04-26T03:39:23.124 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.124 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.124 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.124 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:39:23.124 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.125 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.125 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.125 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6806", 2024-04-26T03:39:23.125 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377656090 2024-04-26T03:39:23.125 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.125 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.125 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.125 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6807", 2024-04-26T03:39:23.125 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377656090 2024-04-26T03:39:23.125 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.125 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.125 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.125 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:39:23.125 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.126 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.126 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.126 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6804", 2024-04-26T03:39:23.126 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377656090 2024-04-26T03:39:23.126 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.126 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.126 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.126 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6805", 2024-04-26T03:39:23.126 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377656090 2024-04-26T03:39:23.126 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.126 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.126 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.127 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.28:6801/2377656090", 2024-04-26T03:39:23.127 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.28:6803/2377656090", 2024-04-26T03:39:23.127 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.28:6807/2377656090", 2024-04-26T03:39:23.127 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.28:6805/2377656090", 2024-04-26T03:39:23.127 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:39:23.127 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:39:23.127 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:39:23.127 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.127 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.127 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.127 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 1, 2024-04-26T03:39:23.127 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "52819ace-a070-4822-b33e-3c54466e61ff", 2024-04-26T03:39:23.128 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:39:23.128 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:39:23.128 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:39:23.128 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:39:23.128 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:39:23.128 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:39:23.128 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 23, 2024-04-26T03:39:23.128 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 44, 2024-04-26T03:39:23.128 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:39:23.128 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:39:23.128 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:39:23.128 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.128 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.128 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.129 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6800", 2024-04-26T03:39:23.129 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1678150206 2024-04-26T03:39:23.129 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.129 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.129 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.129 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6801", 2024-04-26T03:39:23.129 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1678150206 2024-04-26T03:39:23.129 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.129 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.129 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.129 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:39:23.129 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.129 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.129 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.130 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6802", 2024-04-26T03:39:23.130 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1678150206 2024-04-26T03:39:23.130 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.130 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.130 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.130 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6803", 2024-04-26T03:39:23.130 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1678150206 2024-04-26T03:39:23.130 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.130 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.130 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.130 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:39:23.130 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.131 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.131 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.131 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6806", 2024-04-26T03:39:23.131 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1678150206 2024-04-26T03:39:23.131 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.131 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.131 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.131 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6807", 2024-04-26T03:39:23.131 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1678150206 2024-04-26T03:39:23.131 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.131 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.131 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.131 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:39:23.131 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.132 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.132 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.132 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6804", 2024-04-26T03:39:23.132 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1678150206 2024-04-26T03:39:23.132 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.132 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.132 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.132 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6805", 2024-04-26T03:39:23.132 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1678150206 2024-04-26T03:39:23.132 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.132 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.132 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.132 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.42:6801/1678150206", 2024-04-26T03:39:23.133 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.42:6803/1678150206", 2024-04-26T03:39:23.133 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.42:6807/1678150206", 2024-04-26T03:39:23.133 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.42:6805/1678150206", 2024-04-26T03:39:23.133 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:39:23.133 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:39:23.133 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:39:23.133 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.133 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.133 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.133 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 2, 2024-04-26T03:39:23.133 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "858f8ac1-2edc-43d5-914c-c1914bdc0c9a", 2024-04-26T03:39:23.133 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:39:23.133 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:39:23.133 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:39:23.134 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:39:23.134 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:39:23.134 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:39:23.134 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 23, 2024-04-26T03:39:23.134 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 0, 2024-04-26T03:39:23.134 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:39:23.134 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:39:23.134 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:39:23.134 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.134 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.134 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.134 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6800", 2024-04-26T03:39:23.134 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 714535415 2024-04-26T03:39:23.134 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.135 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.135 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.135 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6801", 2024-04-26T03:39:23.135 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 714535415 2024-04-26T03:39:23.135 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.135 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.135 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.135 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:39:23.135 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.135 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.135 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.135 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6802", 2024-04-26T03:39:23.135 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 714535415 2024-04-26T03:39:23.136 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.136 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.136 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.136 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6803", 2024-04-26T03:39:23.136 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 714535415 2024-04-26T03:39:23.136 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.136 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.136 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.136 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:39:23.136 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.136 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.136 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.136 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6806", 2024-04-26T03:39:23.136 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 714535415 2024-04-26T03:39:23.137 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.137 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.137 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.137 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6807", 2024-04-26T03:39:23.137 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 714535415 2024-04-26T03:39:23.137 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.137 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.137 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.137 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:39:23.137 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.137 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.137 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.137 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6804", 2024-04-26T03:39:23.138 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 714535415 2024-04-26T03:39:23.138 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.138 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.138 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.138 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6805", 2024-04-26T03:39:23.138 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 714535415 2024-04-26T03:39:23.138 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.138 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.138 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.138 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.66:6801/714535415", 2024-04-26T03:39:23.138 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.66:6803/714535415", 2024-04-26T03:39:23.138 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.66:6807/714535415", 2024-04-26T03:39:23.139 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.66:6805/714535415", 2024-04-26T03:39:23.139 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:39:23.139 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:39:23.139 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:39:23.139 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.139 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.139 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.139 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 3, 2024-04-26T03:39:23.139 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "531a2e30-3351-4f84-804b-5fb5050b84ed", 2024-04-26T03:39:23.139 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:39:23.139 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:39:23.139 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:39:23.139 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:39:23.140 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:39:23.140 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:39:23.140 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 24, 2024-04-26T03:39:23.140 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 32, 2024-04-26T03:39:23.140 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:39:23.140 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:39:23.140 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:39:23.140 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.140 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.140 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.140 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6810", 2024-04-26T03:39:23.140 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377983625 2024-04-26T03:39:23.140 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.140 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.141 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.141 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6811", 2024-04-26T03:39:23.141 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377983625 2024-04-26T03:39:23.141 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.141 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.141 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.141 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:39:23.141 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.141 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.141 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.141 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6812", 2024-04-26T03:39:23.141 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377983625 2024-04-26T03:39:23.141 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.141 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.141 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.142 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6813", 2024-04-26T03:39:23.142 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377983625 2024-04-26T03:39:23.142 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.142 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.142 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.142 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:39:23.142 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.142 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.142 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.142 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6816", 2024-04-26T03:39:23.142 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377983625 2024-04-26T03:39:23.142 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.142 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.142 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.143 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6817", 2024-04-26T03:39:23.143 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377983625 2024-04-26T03:39:23.143 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.143 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.143 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.143 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:39:23.143 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.143 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.143 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.143 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6814", 2024-04-26T03:39:23.143 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377983625 2024-04-26T03:39:23.143 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.143 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.143 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.144 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6815", 2024-04-26T03:39:23.144 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2377983625 2024-04-26T03:39:23.144 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.144 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.144 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.144 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.92:6811/2377983625", 2024-04-26T03:39:23.144 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.92:6813/2377983625", 2024-04-26T03:39:23.144 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.92:6817/2377983625", 2024-04-26T03:39:23.144 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.92:6815/2377983625", 2024-04-26T03:39:23.144 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:39:23.144 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:39:23.144 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:39:23.144 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.144 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.145 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.145 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 4, 2024-04-26T03:39:23.145 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "6d38ffda-40eb-4fd7-9300-2a09f2673e66", 2024-04-26T03:39:23.145 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:39:23.145 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:39:23.145 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:39:23.145 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:39:23.145 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:39:23.145 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:39:23.145 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 23, 2024-04-26T03:39:23.145 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 65, 2024-04-26T03:39:23.145 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:39:23.145 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:39:23.145 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:39:23.145 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.146 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.146 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.146 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6808", 2024-04-26T03:39:23.146 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 183752455 2024-04-26T03:39:23.146 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.146 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.146 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.146 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6809", 2024-04-26T03:39:23.146 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 183752455 2024-04-26T03:39:23.146 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.146 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.146 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.146 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:39:23.147 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.147 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.147 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.147 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6810", 2024-04-26T03:39:23.147 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 183752455 2024-04-26T03:39:23.147 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.147 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.147 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.147 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6811", 2024-04-26T03:39:23.147 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 183752455 2024-04-26T03:39:23.147 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.147 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.147 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.147 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:39:23.147 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.148 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.148 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.148 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6814", 2024-04-26T03:39:23.148 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 183752455 2024-04-26T03:39:23.148 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.148 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.148 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.148 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6815", 2024-04-26T03:39:23.148 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 183752455 2024-04-26T03:39:23.148 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.149 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.149 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.149 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:39:23.149 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.149 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.149 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.149 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6812", 2024-04-26T03:39:23.149 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 183752455 2024-04-26T03:39:23.149 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.149 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.149 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.149 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6813", 2024-04-26T03:39:23.149 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 183752455 2024-04-26T03:39:23.149 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.150 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.150 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.150 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.28:6809/183752455", 2024-04-26T03:39:23.150 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.28:6811/183752455", 2024-04-26T03:39:23.150 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.28:6815/183752455", 2024-04-26T03:39:23.150 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.28:6813/183752455", 2024-04-26T03:39:23.150 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:39:23.150 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:39:23.150 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:39:23.150 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.150 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.150 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.150 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 5, 2024-04-26T03:39:23.151 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "361aa40d-e53b-42af-afd4-81fbcac28725", 2024-04-26T03:39:23.151 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:39:23.151 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:39:23.151 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:39:23.151 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:39:23.151 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:39:23.151 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:39:23.151 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 24, 2024-04-26T03:39:23.151 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 51, 2024-04-26T03:39:23.151 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:39:23.151 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:39:23.151 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:39:23.151 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.151 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.152 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.152 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6808", 2024-04-26T03:39:23.152 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2012261939 2024-04-26T03:39:23.152 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.152 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.152 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.152 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6809", 2024-04-26T03:39:23.152 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2012261939 2024-04-26T03:39:23.152 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.152 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.152 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.152 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:39:23.152 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.152 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.153 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.153 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6810", 2024-04-26T03:39:23.153 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2012261939 2024-04-26T03:39:23.153 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.153 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.153 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.153 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6811", 2024-04-26T03:39:23.153 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2012261939 2024-04-26T03:39:23.153 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.153 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.153 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.153 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:39:23.153 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.154 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.154 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.154 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6814", 2024-04-26T03:39:23.154 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2012261939 2024-04-26T03:39:23.154 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.154 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.154 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.154 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6815", 2024-04-26T03:39:23.154 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2012261939 2024-04-26T03:39:23.154 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.154 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.154 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.154 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:39:23.154 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.155 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.155 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.155 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6812", 2024-04-26T03:39:23.155 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2012261939 2024-04-26T03:39:23.155 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.155 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.155 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.155 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6813", 2024-04-26T03:39:23.155 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2012261939 2024-04-26T03:39:23.155 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.155 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.155 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.155 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.42:6809/2012261939", 2024-04-26T03:39:23.155 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.42:6811/2012261939", 2024-04-26T03:39:23.156 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.42:6815/2012261939", 2024-04-26T03:39:23.156 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.42:6813/2012261939", 2024-04-26T03:39:23.156 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:39:23.156 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:39:23.156 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:39:23.156 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.156 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.156 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.157 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 6, 2024-04-26T03:39:23.157 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "f610ad3e-1fd8-42bd-af4b-0d79c3bb9cf8", 2024-04-26T03:39:23.157 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:39:23.157 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:39:23.157 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:39:23.157 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:39:23.157 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:39:23.157 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:39:23.157 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 22, 2024-04-26T03:39:23.157 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 63, 2024-04-26T03:39:23.157 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:39:23.157 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:39:23.157 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:39:23.158 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.158 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.158 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.158 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6808", 2024-04-26T03:39:23.158 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3533460882 2024-04-26T03:39:23.158 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.158 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.158 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.158 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6809", 2024-04-26T03:39:23.158 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3533460882 2024-04-26T03:39:23.158 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.158 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.158 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.159 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:39:23.159 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.159 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.159 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.159 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6810", 2024-04-26T03:39:23.159 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3533460882 2024-04-26T03:39:23.159 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.159 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.159 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.159 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6811", 2024-04-26T03:39:23.159 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3533460882 2024-04-26T03:39:23.159 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.159 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.159 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.160 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:39:23.160 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.160 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.160 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.160 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6814", 2024-04-26T03:39:23.160 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3533460882 2024-04-26T03:39:23.160 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.160 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.160 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.160 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6815", 2024-04-26T03:39:23.160 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3533460882 2024-04-26T03:39:23.160 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.160 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.160 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.161 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:39:23.161 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.161 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.161 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.161 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6812", 2024-04-26T03:39:23.161 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3533460882 2024-04-26T03:39:23.161 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.161 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.161 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.161 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6813", 2024-04-26T03:39:23.161 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3533460882 2024-04-26T03:39:23.161 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.161 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.161 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.162 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.66:6809/3533460882", 2024-04-26T03:39:23.162 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.66:6811/3533460882", 2024-04-26T03:39:23.162 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.66:6815/3533460882", 2024-04-26T03:39:23.162 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.66:6813/3533460882", 2024-04-26T03:39:23.162 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:39:23.162 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:39:23.162 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:39:23.162 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.162 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.162 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.162 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 7, 2024-04-26T03:39:23.162 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "a4bae6fb-e087-4563-a129-6938e18eed8c", 2024-04-26T03:39:23.162 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:39:23.163 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:39:23.163 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:39:23.163 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:39:23.163 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:39:23.163 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:39:23.163 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 22, 2024-04-26T03:39:23.163 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 32, 2024-04-26T03:39:23.163 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:39:23.163 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:39:23.163 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:39:23.163 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.163 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.163 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.164 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6802", 2024-04-26T03:39:23.164 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1760163933 2024-04-26T03:39:23.164 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.164 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.164 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.164 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6803", 2024-04-26T03:39:23.164 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1760163933 2024-04-26T03:39:23.164 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.164 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.164 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.164 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:39:23.164 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.164 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.164 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.165 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6804", 2024-04-26T03:39:23.165 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1760163933 2024-04-26T03:39:23.165 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.165 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.165 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.165 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6805", 2024-04-26T03:39:23.165 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1760163933 2024-04-26T03:39:23.165 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.165 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.165 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.165 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:39:23.165 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.165 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.165 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.166 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6808", 2024-04-26T03:39:23.166 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1760163933 2024-04-26T03:39:23.166 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.166 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.166 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.166 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6809", 2024-04-26T03:39:23.166 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1760163933 2024-04-26T03:39:23.166 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.166 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.166 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.166 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:39:23.166 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.166 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.167 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.167 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6806", 2024-04-26T03:39:23.167 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1760163933 2024-04-26T03:39:23.167 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.167 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.167 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.167 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6807", 2024-04-26T03:39:23.167 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1760163933 2024-04-26T03:39:23.167 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.167 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.167 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.167 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.92:6803/1760163933", 2024-04-26T03:39:23.167 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.92:6805/1760163933", 2024-04-26T03:39:23.167 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.92:6809/1760163933", 2024-04-26T03:39:23.168 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.92:6807/1760163933", 2024-04-26T03:39:23.168 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:39:23.168 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:39:23.168 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:39:23.168 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.168 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.168 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.168 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 8, 2024-04-26T03:39:23.168 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "479f3885-452e-4231-bdc2-b79668384ba6", 2024-04-26T03:39:23.168 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:39:23.168 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:39:23.168 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:39:23.169 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:39:23.169 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:39:23.169 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:39:23.169 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 22, 2024-04-26T03:39:23.169 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 44, 2024-04-26T03:39:23.169 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:39:23.169 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:39:23.169 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:39:23.169 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.169 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.169 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.169 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6816", 2024-04-26T03:39:23.170 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4127602407 2024-04-26T03:39:23.170 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.170 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.170 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.170 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6817", 2024-04-26T03:39:23.170 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4127602407 2024-04-26T03:39:23.170 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.170 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.170 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.170 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:39:23.171 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.171 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.171 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.171 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6818", 2024-04-26T03:39:23.171 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4127602407 2024-04-26T03:39:23.171 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.172 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.172 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.172 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6819", 2024-04-26T03:39:23.172 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4127602407 2024-04-26T03:39:23.172 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.172 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.172 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.172 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:39:23.172 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.172 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.172 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.172 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6822", 2024-04-26T03:39:23.173 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4127602407 2024-04-26T03:39:23.173 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.173 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.173 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.173 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6823", 2024-04-26T03:39:23.173 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4127602407 2024-04-26T03:39:23.173 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.173 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.173 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.173 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:39:23.173 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.173 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.173 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.174 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6820", 2024-04-26T03:39:23.174 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4127602407 2024-04-26T03:39:23.174 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.174 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.174 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.174 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6821", 2024-04-26T03:39:23.174 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4127602407 2024-04-26T03:39:23.175 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.175 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.175 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.175 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.28:6817/4127602407", 2024-04-26T03:39:23.175 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.28:6819/4127602407", 2024-04-26T03:39:23.175 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.28:6823/4127602407", 2024-04-26T03:39:23.175 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.28:6821/4127602407", 2024-04-26T03:39:23.175 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:39:23.175 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:39:23.175 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:39:23.175 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.175 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.176 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.176 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 9, 2024-04-26T03:39:23.176 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "9a603a47-7c5a-47d4-a298-56b3ac991892", 2024-04-26T03:39:23.176 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:39:23.176 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:39:23.176 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:39:23.176 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:39:23.176 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:39:23.176 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:39:23.176 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 22, 2024-04-26T03:39:23.176 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 26, 2024-04-26T03:39:23.176 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:39:23.176 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:39:23.177 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:39:23.177 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.177 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.177 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.177 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6816", 2024-04-26T03:39:23.177 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4168729845 2024-04-26T03:39:23.177 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.177 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.177 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.177 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6818", 2024-04-26T03:39:23.177 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4168729845 2024-04-26T03:39:23.177 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.177 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.178 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.178 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:39:23.178 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.178 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.178 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.178 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6820", 2024-04-26T03:39:23.178 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4168729845 2024-04-26T03:39:23.178 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.178 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.178 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.178 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6822", 2024-04-26T03:39:23.178 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4168729845 2024-04-26T03:39:23.179 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.179 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.179 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.179 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:39:23.179 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.179 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.179 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.179 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6828", 2024-04-26T03:39:23.179 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4168729845 2024-04-26T03:39:23.180 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.180 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.180 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.180 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6830", 2024-04-26T03:39:23.180 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4168729845 2024-04-26T03:39:23.180 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.180 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.180 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.180 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:39:23.180 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.180 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.180 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.180 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6824", 2024-04-26T03:39:23.181 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4168729845 2024-04-26T03:39:23.181 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.181 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.181 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.181 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6826", 2024-04-26T03:39:23.181 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4168729845 2024-04-26T03:39:23.181 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.181 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.181 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.181 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.42:6818/4168729845", 2024-04-26T03:39:23.181 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.42:6822/4168729845", 2024-04-26T03:39:23.181 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.42:6830/4168729845", 2024-04-26T03:39:23.181 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.42:6826/4168729845", 2024-04-26T03:39:23.182 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:39:23.182 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:39:23.182 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:39:23.182 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.182 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.182 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.182 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 10, 2024-04-26T03:39:23.182 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "1cd42945-6ee1-49f0-b17b-4093dd36b4ec", 2024-04-26T03:39:23.182 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:39:23.182 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:39:23.182 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:39:23.183 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:39:23.183 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:39:23.183 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:39:23.183 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 22, 2024-04-26T03:39:23.183 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 44, 2024-04-26T03:39:23.183 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:39:23.183 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:39:23.183 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:39:23.183 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.183 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.183 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.183 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6816", 2024-04-26T03:39:23.183 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4180572718 2024-04-26T03:39:23.183 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.184 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.184 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.184 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6817", 2024-04-26T03:39:23.184 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4180572718 2024-04-26T03:39:23.184 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.184 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.184 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.184 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:39:23.184 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.184 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.184 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.184 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6818", 2024-04-26T03:39:23.184 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4180572718 2024-04-26T03:39:23.185 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.185 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.185 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.185 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6819", 2024-04-26T03:39:23.185 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4180572718 2024-04-26T03:39:23.185 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.185 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.185 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.185 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:39:23.185 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.185 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.185 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.185 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6822", 2024-04-26T03:39:23.185 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4180572718 2024-04-26T03:39:23.186 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.186 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.186 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.186 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6823", 2024-04-26T03:39:23.186 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4180572718 2024-04-26T03:39:23.186 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.186 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.186 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.186 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:39:23.186 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.186 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.186 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.186 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6820", 2024-04-26T03:39:23.187 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4180572718 2024-04-26T03:39:23.187 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.187 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.187 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.187 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6821", 2024-04-26T03:39:23.187 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4180572718 2024-04-26T03:39:23.187 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.187 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.187 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.187 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.66:6817/4180572718", 2024-04-26T03:39:23.187 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.66:6819/4180572718", 2024-04-26T03:39:23.187 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.66:6823/4180572718", 2024-04-26T03:39:23.187 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.66:6821/4180572718", 2024-04-26T03:39:23.187 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:39:23.188 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:39:23.188 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:39:23.188 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.188 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.188 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.188 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 11, 2024-04-26T03:39:23.188 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "b41e8100-1d4f-4b7a-991a-7a27bc29feec", 2024-04-26T03:39:23.188 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:39:23.188 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:39:23.188 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:39:23.189 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:39:23.189 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:39:23.189 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:39:23.189 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 21, 2024-04-26T03:39:23.189 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 62, 2024-04-26T03:39:23.189 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:39:23.189 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:39:23.189 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:39:23.189 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.189 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.189 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.189 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6818", 2024-04-26T03:39:23.189 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 295713815 2024-04-26T03:39:23.189 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.190 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.190 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.190 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6819", 2024-04-26T03:39:23.190 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 295713815 2024-04-26T03:39:23.190 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.190 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.190 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.190 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:39:23.190 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.190 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.190 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.190 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6820", 2024-04-26T03:39:23.190 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 295713815 2024-04-26T03:39:23.191 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.191 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.191 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.191 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6821", 2024-04-26T03:39:23.191 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 295713815 2024-04-26T03:39:23.191 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.191 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.191 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.191 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:39:23.192 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.192 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.192 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.192 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6824", 2024-04-26T03:39:23.192 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 295713815 2024-04-26T03:39:23.192 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.192 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.192 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.193 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6825", 2024-04-26T03:39:23.193 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 295713815 2024-04-26T03:39:23.193 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.193 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.193 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.193 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:39:23.193 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.193 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.193 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.193 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6822", 2024-04-26T03:39:23.193 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 295713815 2024-04-26T03:39:23.194 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.194 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.194 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.194 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6823", 2024-04-26T03:39:23.194 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 295713815 2024-04-26T03:39:23.194 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.194 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.194 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.194 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.92:6819/295713815", 2024-04-26T03:39:23.194 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.92:6821/295713815", 2024-04-26T03:39:23.194 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.92:6825/295713815", 2024-04-26T03:39:23.194 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.92:6823/295713815", 2024-04-26T03:39:23.194 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:39:23.195 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:39:23.195 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:39:23.195 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.195 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.195 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.195 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 12, 2024-04-26T03:39:23.195 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "458cf150-32e1-449e-b533-ac6ab5226f85", 2024-04-26T03:39:23.195 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:39:23.195 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:39:23.195 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:39:23.195 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:39:23.196 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:39:23.196 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:39:23.196 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 22, 2024-04-26T03:39:23.196 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 0, 2024-04-26T03:39:23.196 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:39:23.196 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:39:23.196 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:39:23.196 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.196 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.197 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.197 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6824", 2024-04-26T03:39:23.197 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2081141953 2024-04-26T03:39:23.197 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.197 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.197 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.197 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6825", 2024-04-26T03:39:23.197 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2081141953 2024-04-26T03:39:23.197 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.197 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.197 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.197 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:39:23.197 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.198 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.198 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.198 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6826", 2024-04-26T03:39:23.198 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2081141953 2024-04-26T03:39:23.198 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.198 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.198 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.199 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6827", 2024-04-26T03:39:23.199 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2081141953 2024-04-26T03:39:23.199 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.199 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.199 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.199 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:39:23.199 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.200 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.200 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.200 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6830", 2024-04-26T03:39:23.200 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2081141953 2024-04-26T03:39:23.200 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.200 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.200 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.200 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6831", 2024-04-26T03:39:23.201 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2081141953 2024-04-26T03:39:23.201 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.201 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.201 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.201 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:39:23.201 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.201 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.201 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.202 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6828", 2024-04-26T03:39:23.202 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2081141953 2024-04-26T03:39:23.202 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.202 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.202 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.202 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.28:6829", 2024-04-26T03:39:23.202 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2081141953 2024-04-26T03:39:23.203 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.203 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.203 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.203 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.28:6825/2081141953", 2024-04-26T03:39:23.203 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.28:6827/2081141953", 2024-04-26T03:39:23.203 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.28:6831/2081141953", 2024-04-26T03:39:23.203 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.28:6829/2081141953", 2024-04-26T03:39:23.204 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:39:23.204 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:39:23.204 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:39:23.204 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.204 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.204 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.204 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 13, 2024-04-26T03:39:23.205 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "f008462e-a735-4845-9ecf-dd7d4a6941af", 2024-04-26T03:39:23.205 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:39:23.205 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:39:23.205 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:39:23.205 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:39:23.205 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:39:23.205 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:39:23.206 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 22, 2024-04-26T03:39:23.206 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 59, 2024-04-26T03:39:23.206 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:39:23.206 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:39:23.206 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:39:23.206 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.206 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.206 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.207 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6817", 2024-04-26T03:39:23.207 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 301360893 2024-04-26T03:39:23.207 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.207 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.207 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.207 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6819", 2024-04-26T03:39:23.207 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 301360893 2024-04-26T03:39:23.207 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.208 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.208 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.208 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:39:23.208 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.208 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.208 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.208 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6821", 2024-04-26T03:39:23.209 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 301360893 2024-04-26T03:39:23.209 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.209 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.209 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.209 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6823", 2024-04-26T03:39:23.209 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 301360893 2024-04-26T03:39:23.209 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.210 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.210 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.210 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:39:23.210 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.210 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.210 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.210 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6829", 2024-04-26T03:39:23.211 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 301360893 2024-04-26T03:39:23.211 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.211 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.211 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.211 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6831", 2024-04-26T03:39:23.211 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 301360893 2024-04-26T03:39:23.211 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.211 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.212 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.212 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:39:23.212 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.212 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.212 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.212 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6825", 2024-04-26T03:39:23.212 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 301360893 2024-04-26T03:39:23.212 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.213 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.213 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.213 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.42:6827", 2024-04-26T03:39:23.213 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 301360893 2024-04-26T03:39:23.213 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.213 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.213 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.213 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.42:6819/301360893", 2024-04-26T03:39:23.214 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.42:6823/301360893", 2024-04-26T03:39:23.214 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.42:6831/301360893", 2024-04-26T03:39:23.214 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.42:6827/301360893", 2024-04-26T03:39:23.214 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:39:23.214 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:39:23.214 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:39:23.214 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.214 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.215 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.215 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 14, 2024-04-26T03:39:23.215 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "6f66ae97-eb8a-48eb-9bcc-20f65fbf690b", 2024-04-26T03:39:23.215 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:39:23.215 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:39:23.215 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:39:23.215 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:39:23.215 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:39:23.216 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:39:23.216 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 22, 2024-04-26T03:39:23.216 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 53, 2024-04-26T03:39:23.216 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:39:23.216 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:39:23.216 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:39:23.216 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.216 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.217 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.217 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6824", 2024-04-26T03:39:23.217 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1534905602 2024-04-26T03:39:23.217 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.217 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.217 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.217 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6825", 2024-04-26T03:39:23.217 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1534905602 2024-04-26T03:39:23.217 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.218 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.218 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.218 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:39:23.218 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.218 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.218 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.218 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6826", 2024-04-26T03:39:23.218 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1534905602 2024-04-26T03:39:23.218 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.218 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.218 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.218 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6827", 2024-04-26T03:39:23.218 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1534905602 2024-04-26T03:39:23.218 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.219 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.219 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.219 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:39:23.219 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.219 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.219 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.219 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6830", 2024-04-26T03:39:23.219 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1534905602 2024-04-26T03:39:23.219 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.219 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.219 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.219 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6831", 2024-04-26T03:39:23.219 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1534905602 2024-04-26T03:39:23.219 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.220 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.220 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.220 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:39:23.220 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.220 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.220 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.220 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6828", 2024-04-26T03:39:23.220 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1534905602 2024-04-26T03:39:23.220 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.220 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.220 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.220 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.66:6829", 2024-04-26T03:39:23.220 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1534905602 2024-04-26T03:39:23.221 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.221 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.221 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.221 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.66:6825/1534905602", 2024-04-26T03:39:23.221 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.66:6827/1534905602", 2024-04-26T03:39:23.221 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.66:6831/1534905602", 2024-04-26T03:39:23.221 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.66:6829/1534905602", 2024-04-26T03:39:23.221 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:39:23.221 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:39:23.221 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:39:23.221 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.221 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.221 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.222 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 15, 2024-04-26T03:39:23.222 INFO:teuthology.orchestra.run.smithi028.stdout: "uuid": "b856bfc8-253b-4112-be5c-f67d61d01768", 2024-04-26T03:39:23.222 INFO:teuthology.orchestra.run.smithi028.stdout: "up": 1, 2024-04-26T03:39:23.222 INFO:teuthology.orchestra.run.smithi028.stdout: "in": 1, 2024-04-26T03:39:23.222 INFO:teuthology.orchestra.run.smithi028.stdout: "weight": 1, 2024-04-26T03:39:23.222 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_affinity": 1, 2024-04-26T03:39:23.222 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_begin": 0, 2024-04-26T03:39:23.222 INFO:teuthology.orchestra.run.smithi028.stdout: "last_clean_end": 0, 2024-04-26T03:39:23.222 INFO:teuthology.orchestra.run.smithi028.stdout: "up_from": 22, 2024-04-26T03:39:23.222 INFO:teuthology.orchestra.run.smithi028.stdout: "up_thru": 44, 2024-04-26T03:39:23.222 INFO:teuthology.orchestra.run.smithi028.stdout: "down_at": 0, 2024-04-26T03:39:23.222 INFO:teuthology.orchestra.run.smithi028.stdout: "lost_at": 0, 2024-04-26T03:39:23.223 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addrs": { 2024-04-26T03:39:23.223 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.223 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.223 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.223 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6826", 2024-04-26T03:39:23.223 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3982634323 2024-04-26T03:39:23.223 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.223 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.223 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.223 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6827", 2024-04-26T03:39:23.223 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3982634323 2024-04-26T03:39:23.223 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.223 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.224 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.224 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addrs": { 2024-04-26T03:39:23.224 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.224 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.224 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.224 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6828", 2024-04-26T03:39:23.224 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3982634323 2024-04-26T03:39:23.224 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.224 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.224 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.224 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6829", 2024-04-26T03:39:23.224 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3982634323 2024-04-26T03:39:23.224 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.224 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.225 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.225 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addrs": { 2024-04-26T03:39:23.225 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.225 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.225 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.225 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6832", 2024-04-26T03:39:23.225 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3982634323 2024-04-26T03:39:23.225 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.225 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.225 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.225 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6833", 2024-04-26T03:39:23.225 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3982634323 2024-04-26T03:39:23.225 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.226 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.226 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.226 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addrs": { 2024-04-26T03:39:23.226 INFO:teuthology.orchestra.run.smithi028.stdout: "addrvec": [ 2024-04-26T03:39:23.226 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.226 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v2", 2024-04-26T03:39:23.226 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6830", 2024-04-26T03:39:23.226 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3982634323 2024-04-26T03:39:23.226 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.226 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.226 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T03:39:23.226 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "172.21.15.92:6831", 2024-04-26T03:39:23.226 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3982634323 2024-04-26T03:39:23.227 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.227 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.227 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.227 INFO:teuthology.orchestra.run.smithi028.stdout: "public_addr": "172.21.15.92:6827/3982634323", 2024-04-26T03:39:23.227 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_addr": "172.21.15.92:6829/3982634323", 2024-04-26T03:39:23.227 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_back_addr": "172.21.15.92:6833/3982634323", 2024-04-26T03:39:23.227 INFO:teuthology.orchestra.run.smithi028.stdout: "heartbeat_front_addr": "172.21.15.92:6831/3982634323", 2024-04-26T03:39:23.227 INFO:teuthology.orchestra.run.smithi028.stdout: "state": [ 2024-04-26T03:39:23.227 INFO:teuthology.orchestra.run.smithi028.stdout: "exists", 2024-04-26T03:39:23.227 INFO:teuthology.orchestra.run.smithi028.stdout: "up" 2024-04-26T03:39:23.227 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.227 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.227 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-04-26T03:39:23.228 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_xinfo": [ 2024-04-26T03:39:23.228 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.228 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 0, 2024-04-26T03:39:23.228 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:39:23.228 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:39:23.228 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:39:23.228 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:39:23.228 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:39:23.228 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.265053+0000", 2024-04-26T03:39:23.228 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:39:23.228 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.228 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.228 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 1, 2024-04-26T03:39:23.228 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:39:23.229 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:39:23.229 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:39:23.229 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:39:23.229 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:39:23.229 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.243502+0000", 2024-04-26T03:39:23.229 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:39:23.229 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.229 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.229 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 2, 2024-04-26T03:39:23.229 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:39:23.229 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:39:23.229 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:39:23.229 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:39:23.230 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:39:23.230 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.252440+0000", 2024-04-26T03:39:23.230 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:39:23.230 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.230 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.230 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 3, 2024-04-26T03:39:23.230 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:39:23.230 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:39:23.230 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:39:23.230 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:39:23.231 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:39:23.231 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.295331+0000", 2024-04-26T03:39:23.231 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:39:23.231 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.231 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.231 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 4, 2024-04-26T03:39:23.231 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:39:23.231 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:39:23.231 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:39:23.231 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:39:23.231 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:39:23.231 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.213076+0000", 2024-04-26T03:39:23.231 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:39:23.231 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.232 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.232 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 5, 2024-04-26T03:39:23.232 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:39:23.232 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:39:23.232 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:39:23.232 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:39:23.232 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:39:23.232 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.281193+0000", 2024-04-26T03:39:23.232 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:39:23.232 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.232 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.232 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 6, 2024-04-26T03:39:23.232 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:39:23.233 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:39:23.233 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:39:23.233 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:39:23.233 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:39:23.233 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.315801+0000", 2024-04-26T03:39:23.233 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:39:23.233 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.233 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.233 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 7, 2024-04-26T03:39:23.233 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:39:23.233 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:39:23.233 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:39:23.233 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:39:23.233 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:39:23.233 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.329538+0000", 2024-04-26T03:39:23.234 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:39:23.234 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.234 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.234 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 8, 2024-04-26T03:39:23.234 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:39:23.234 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:39:23.234 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:39:23.234 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:39:23.234 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:39:23.234 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.293303+0000", 2024-04-26T03:39:23.234 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:39:23.234 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.235 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.235 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 9, 2024-04-26T03:39:23.235 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:39:23.235 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:39:23.235 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:39:23.235 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:39:23.235 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:39:23.235 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.334962+0000", 2024-04-26T03:39:23.235 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:39:23.235 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.235 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.235 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 10, 2024-04-26T03:39:23.235 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:39:23.235 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:39:23.236 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:39:23.236 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:39:23.236 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:39:23.236 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.233032+0000", 2024-04-26T03:39:23.236 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:39:23.236 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.236 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.236 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 11, 2024-04-26T03:39:23.236 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:39:23.236 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:39:23.236 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:39:23.236 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:39:23.236 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:39:23.237 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.305138+0000", 2024-04-26T03:39:23.237 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:39:23.237 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.237 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.237 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 12, 2024-04-26T03:39:23.237 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:39:23.237 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:39:23.237 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:39:23.237 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:39:23.237 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:39:23.237 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.351964+0000", 2024-04-26T03:39:23.237 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:39:23.237 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.237 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.238 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 13, 2024-04-26T03:39:23.238 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:39:23.238 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:39:23.238 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:39:23.238 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:39:23.238 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:39:23.238 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.241086+0000", 2024-04-26T03:39:23.238 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:39:23.239 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.239 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.239 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 14, 2024-04-26T03:39:23.239 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:39:23.239 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:39:23.239 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:39:23.239 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:39:23.239 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:39:23.239 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.352515+0000", 2024-04-26T03:39:23.239 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:39:23.239 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.239 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.240 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": 15, 2024-04-26T03:39:23.240 INFO:teuthology.orchestra.run.smithi028.stdout: "down_stamp": "0.000000", 2024-04-26T03:39:23.240 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_probability": 0, 2024-04-26T03:39:23.240 INFO:teuthology.orchestra.run.smithi028.stdout: "laggy_interval": 0, 2024-04-26T03:39:23.240 INFO:teuthology.orchestra.run.smithi028.stdout: "features": 4540701547738038271, 2024-04-26T03:39:23.240 INFO:teuthology.orchestra.run.smithi028.stdout: "old_weight": 0, 2024-04-26T03:39:23.240 INFO:teuthology.orchestra.run.smithi028.stdout: "last_purged_snaps_scrub": "2024-04-26T03:38:29.357462+0000", 2024-04-26T03:39:23.240 INFO:teuthology.orchestra.run.smithi028.stdout: "dead_epoch": 0 2024-04-26T03:39:23.240 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.240 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-04-26T03:39:23.240 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_upmap": [], 2024-04-26T03:39:23.240 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_upmap_items": [ 2024-04-26T03:39:23.240 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.240 INFO:teuthology.orchestra.run.smithi028.stdout: "pgid": "2.2", 2024-04-26T03:39:23.241 INFO:teuthology.orchestra.run.smithi028.stdout: "mappings": [ 2024-04-26T03:39:23.241 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.241 INFO:teuthology.orchestra.run.smithi028.stdout: "from": 3, 2024-04-26T03:39:23.241 INFO:teuthology.orchestra.run.smithi028.stdout: "to": 11 2024-04-26T03:39:23.241 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.241 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.241 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.241 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.241 INFO:teuthology.orchestra.run.smithi028.stdout: "pgid": "3.f", 2024-04-26T03:39:23.241 INFO:teuthology.orchestra.run.smithi028.stdout: "mappings": [ 2024-04-26T03:39:23.241 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.242 INFO:teuthology.orchestra.run.smithi028.stdout: "from": 2, 2024-04-26T03:39:23.242 INFO:teuthology.orchestra.run.smithi028.stdout: "to": 9 2024-04-26T03:39:23.242 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.242 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.242 INFO:teuthology.orchestra.run.smithi028.stdout: "from": 5, 2024-04-26T03:39:23.242 INFO:teuthology.orchestra.run.smithi028.stdout: "to": 14 2024-04-26T03:39:23.242 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.242 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.242 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.242 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-04-26T03:39:23.242 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_upmap_primaries": [], 2024-04-26T03:39:23.243 INFO:teuthology.orchestra.run.smithi028.stdout: "pg_temp": [], 2024-04-26T03:39:23.243 INFO:teuthology.orchestra.run.smithi028.stdout: "primary_temp": [], 2024-04-26T03:39:23.243 INFO:teuthology.orchestra.run.smithi028.stdout: "blocklist": {}, 2024-04-26T03:39:23.243 INFO:teuthology.orchestra.run.smithi028.stdout: "range_blocklist": {}, 2024-04-26T03:39:23.243 INFO:teuthology.orchestra.run.smithi028.stdout: "erasure_code_profiles": { 2024-04-26T03:39:23.243 INFO:teuthology.orchestra.run.smithi028.stdout: "default": { 2024-04-26T03:39:23.243 INFO:teuthology.orchestra.run.smithi028.stdout: "crush-failure-domain": "osd", 2024-04-26T03:39:23.243 INFO:teuthology.orchestra.run.smithi028.stdout: "k": "2", 2024-04-26T03:39:23.243 INFO:teuthology.orchestra.run.smithi028.stdout: "m": "1", 2024-04-26T03:39:23.243 INFO:teuthology.orchestra.run.smithi028.stdout: "plugin": "jerasure", 2024-04-26T03:39:23.244 INFO:teuthology.orchestra.run.smithi028.stdout: "technique": "reed_sol_van" 2024-04-26T03:39:23.244 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.244 INFO:teuthology.orchestra.run.smithi028.stdout: "shecprofile": { 2024-04-26T03:39:23.244 INFO:teuthology.orchestra.run.smithi028.stdout: "c": "2", 2024-04-26T03:39:23.244 INFO:teuthology.orchestra.run.smithi028.stdout: "crush-device-class": "", 2024-04-26T03:39:23.244 INFO:teuthology.orchestra.run.smithi028.stdout: "crush-failure-domain": "osd", 2024-04-26T03:39:23.244 INFO:teuthology.orchestra.run.smithi028.stdout: "crush-num-failure-domains": "0", 2024-04-26T03:39:23.244 INFO:teuthology.orchestra.run.smithi028.stdout: "crush-osds-per-failure-domain": "0", 2024-04-26T03:39:23.244 INFO:teuthology.orchestra.run.smithi028.stdout: "crush-root": "default", 2024-04-26T03:39:23.244 INFO:teuthology.orchestra.run.smithi028.stdout: "k": "4", 2024-04-26T03:39:23.244 INFO:teuthology.orchestra.run.smithi028.stdout: "m": "3", 2024-04-26T03:39:23.244 INFO:teuthology.orchestra.run.smithi028.stdout: "name": "shecprofile", 2024-04-26T03:39:23.244 INFO:teuthology.orchestra.run.smithi028.stdout: "plugin": "shec", 2024-04-26T03:39:23.244 INFO:teuthology.orchestra.run.smithi028.stdout: "technique": "multiple" 2024-04-26T03:39:23.245 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.245 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.245 INFO:teuthology.orchestra.run.smithi028.stdout: "removed_snaps_queue": [ 2024-04-26T03:39:23.245 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.245 INFO:teuthology.orchestra.run.smithi028.stdout: "pool": 3, 2024-04-26T03:39:23.245 INFO:teuthology.orchestra.run.smithi028.stdout: "snaps": [ 2024-04-26T03:39:23.245 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.245 INFO:teuthology.orchestra.run.smithi028.stdout: "begin": 10, 2024-04-26T03:39:23.245 INFO:teuthology.orchestra.run.smithi028.stdout: "length": 2 2024-04-26T03:39:23.245 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.245 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.245 INFO:teuthology.orchestra.run.smithi028.stdout: "begin": 15, 2024-04-26T03:39:23.246 INFO:teuthology.orchestra.run.smithi028.stdout: "length": 1 2024-04-26T03:39:23.246 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:23.246 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:23.246 INFO:teuthology.orchestra.run.smithi028.stdout: "begin": 17, 2024-04-26T03:39:23.246 INFO:teuthology.orchestra.run.smithi028.stdout: "length": 1 2024-04-26T03:39:23.246 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.246 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:23.246 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.247 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2024-04-26T03:39:23.247 INFO:teuthology.orchestra.run.smithi028.stdout: "new_removed_snaps": [], 2024-04-26T03:39:23.247 INFO:teuthology.orchestra.run.smithi028.stdout: "new_purged_snaps": [], 2024-04-26T03:39:23.247 INFO:teuthology.orchestra.run.smithi028.stdout: "crush_node_flags": {}, 2024-04-26T03:39:23.247 INFO:teuthology.orchestra.run.smithi028.stdout: "device_class_flags": {}, 2024-04-26T03:39:23.247 INFO:teuthology.orchestra.run.smithi028.stdout: "stretch_mode": { 2024-04-26T03:39:23.248 INFO:teuthology.orchestra.run.smithi028.stdout: "stretch_mode_enabled": false, 2024-04-26T03:39:23.248 INFO:teuthology.orchestra.run.smithi028.stdout: "stretch_bucket_count": 0, 2024-04-26T03:39:23.248 INFO:teuthology.orchestra.run.smithi028.stdout: "degraded_stretch_mode": 0, 2024-04-26T03:39:23.248 INFO:teuthology.orchestra.run.smithi028.stdout: "recovering_stretch_mode": 0, 2024-04-26T03:39:23.248 INFO:teuthology.orchestra.run.smithi028.stdout: "stretch_mode_bucket": 0 2024-04-26T03:39:23.248 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:23.249 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:39:23.251 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:39:23.249+0000 7f60d3c4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31757) UID: 0 2024-04-26T03:39:23.252 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:23.252 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:23.254 INFO:tasks.thrashosds.thrasher:j is {'epoch': 67, 'fsid': 'c5a13c99-5c9a-4b8b-a893-4f6f4211677a', 'created': '2024-04-26T03:38:19.896648+0000', 'modified': '2024-04-26T03:39:22.686418+0000', 'last_up_change': '2024-04-26T03:38:33.308561+0000', 'last_in_change': '2024-04-26T03:38:24.150336+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 16, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T03:38:32.306696+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 15.380000114440918, 'score_stable': 15.380000114440918, 'optimal_score': 0.12999999523162842, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-04-26T03:38:34.592633+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '29', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 4, 'score_stable': 4, 'optimal_score': 0.75, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'unique_pool_0', 'create_time': '2024-04-26T03:38:47.497176+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 3, 'size': 7, 'min_size': 5, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 23, 'pg_placement_num': 20, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 23, 'last_pg_merge_meta': {'source_pgid': '3.17', 'ready_epoch': 67, 'last_epoch_started': 66, 'last_epoch_clean': 66, 'source_version': "43'6", 'target_version': "49'10"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '67', 'last_force_op_resend_preluminous': '65', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 20, 'snap_epoch': 67, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'shecprofile', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 16384, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': '31b0e7a9-455d-4874-9c60-26efac8d0b1e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6800', 'nonce': 2377656090}, {'type': 'v1', 'addr': '172.21.15.28:6801', 'nonce': 2377656090}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6802', 'nonce': 2377656090}, {'type': 'v1', 'addr': '172.21.15.28:6803', 'nonce': 2377656090}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6806', 'nonce': 2377656090}, {'type': 'v1', 'addr': '172.21.15.28:6807', 'nonce': 2377656090}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6804', 'nonce': 2377656090}, {'type': 'v1', 'addr': '172.21.15.28:6805', 'nonce': 2377656090}]}, 'public_addr': '172.21.15.28:6801/2377656090', 'cluster_addr': '172.21.15.28:6803/2377656090', 'heartbeat_back_addr': '172.21.15.28:6807/2377656090', 'heartbeat_front_addr': '172.21.15.28:6805/2377656090', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '52819ace-a070-4822-b33e-3c54466e61ff', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6800', 'nonce': 1678150206}, {'type': 'v1', 'addr': '172.21.15.42:6801', 'nonce': 1678150206}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6802', 'nonce': 1678150206}, {'type': 'v1', 'addr': '172.21.15.42:6803', 'nonce': 1678150206}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6806', 'nonce': 1678150206}, {'type': 'v1', 'addr': '172.21.15.42:6807', 'nonce': 1678150206}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6804', 'nonce': 1678150206}, {'type': 'v1', 'addr': '172.21.15.42:6805', 'nonce': 1678150206}]}, 'public_addr': '172.21.15.42:6801/1678150206', 'cluster_addr': '172.21.15.42:6803/1678150206', 'heartbeat_back_addr': '172.21.15.42:6807/1678150206', 'heartbeat_front_addr': '172.21.15.42:6805/1678150206', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '858f8ac1-2edc-43d5-914c-c1914bdc0c9a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6800', 'nonce': 714535415}, {'type': 'v1', 'addr': '172.21.15.66:6801', 'nonce': 714535415}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6802', 'nonce': 714535415}, {'type': 'v1', 'addr': '172.21.15.66:6803', 'nonce': 714535415}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6806', 'nonce': 714535415}, {'type': 'v1', 'addr': '172.21.15.66:6807', 'nonce': 714535415}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6804', 'nonce': 714535415}, {'type': 'v1', 'addr': '172.21.15.66:6805', 'nonce': 714535415}]}, 'public_addr': '172.21.15.66:6801/714535415', 'cluster_addr': '172.21.15.66:6803/714535415', 'heartbeat_back_addr': '172.21.15.66:6807/714535415', 'heartbeat_front_addr': '172.21.15.66:6805/714535415', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '531a2e30-3351-4f84-804b-5fb5050b84ed', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 24, 'up_thru': 32, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6810', 'nonce': 2377983625}, {'type': 'v1', 'addr': '172.21.15.92:6811', 'nonce': 2377983625}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6812', 'nonce': 2377983625}, {'type': 'v1', 'addr': '172.21.15.92:6813', 'nonce': 2377983625}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6816', 'nonce': 2377983625}, {'type': 'v1', 'addr': '172.21.15.92:6817', 'nonce': 2377983625}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6814', 'nonce': 2377983625}, {'type': 'v1', 'addr': '172.21.15.92:6815', 'nonce': 2377983625}]}, 'public_addr': '172.21.15.92:6811/2377983625', 'cluster_addr': '172.21.15.92:6813/2377983625', 'heartbeat_back_addr': '172.21.15.92:6817/2377983625', 'heartbeat_front_addr': '172.21.15.92:6815/2377983625', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '6d38ffda-40eb-4fd7-9300-2a09f2673e66', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 65, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6808', 'nonce': 183752455}, {'type': 'v1', 'addr': '172.21.15.28:6809', 'nonce': 183752455}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6810', 'nonce': 183752455}, {'type': 'v1', 'addr': '172.21.15.28:6811', 'nonce': 183752455}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6814', 'nonce': 183752455}, {'type': 'v1', 'addr': '172.21.15.28:6815', 'nonce': 183752455}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6812', 'nonce': 183752455}, {'type': 'v1', 'addr': '172.21.15.28:6813', 'nonce': 183752455}]}, 'public_addr': '172.21.15.28:6809/183752455', 'cluster_addr': '172.21.15.28:6811/183752455', 'heartbeat_back_addr': '172.21.15.28:6815/183752455', 'heartbeat_front_addr': '172.21.15.28:6813/183752455', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '361aa40d-e53b-42af-afd4-81fbcac28725', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 24, 'up_thru': 51, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6808', 'nonce': 2012261939}, {'type': 'v1', 'addr': '172.21.15.42:6809', 'nonce': 2012261939}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6810', 'nonce': 2012261939}, {'type': 'v1', 'addr': '172.21.15.42:6811', 'nonce': 2012261939}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6814', 'nonce': 2012261939}, {'type': 'v1', 'addr': '172.21.15.42:6815', 'nonce': 2012261939}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6812', 'nonce': 2012261939}, {'type': 'v1', 'addr': '172.21.15.42:6813', 'nonce': 2012261939}]}, 'public_addr': '172.21.15.42:6809/2012261939', 'cluster_addr': '172.21.15.42:6811/2012261939', 'heartbeat_back_addr': '172.21.15.42:6815/2012261939', 'heartbeat_front_addr': '172.21.15.42:6813/2012261939', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'f610ad3e-1fd8-42bd-af4b-0d79c3bb9cf8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 63, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6808', 'nonce': 3533460882}, {'type': 'v1', 'addr': '172.21.15.66:6809', 'nonce': 3533460882}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6810', 'nonce': 3533460882}, {'type': 'v1', 'addr': '172.21.15.66:6811', 'nonce': 3533460882}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6814', 'nonce': 3533460882}, {'type': 'v1', 'addr': '172.21.15.66:6815', 'nonce': 3533460882}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6812', 'nonce': 3533460882}, {'type': 'v1', 'addr': '172.21.15.66:6813', 'nonce': 3533460882}]}, 'public_addr': '172.21.15.66:6809/3533460882', 'cluster_addr': '172.21.15.66:6811/3533460882', 'heartbeat_back_addr': '172.21.15.66:6815/3533460882', 'heartbeat_front_addr': '172.21.15.66:6813/3533460882', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'a4bae6fb-e087-4563-a129-6938e18eed8c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 32, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6802', 'nonce': 1760163933}, {'type': 'v1', 'addr': '172.21.15.92:6803', 'nonce': 1760163933}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6804', 'nonce': 1760163933}, {'type': 'v1', 'addr': '172.21.15.92:6805', 'nonce': 1760163933}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6808', 'nonce': 1760163933}, {'type': 'v1', 'addr': '172.21.15.92:6809', 'nonce': 1760163933}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6806', 'nonce': 1760163933}, {'type': 'v1', 'addr': '172.21.15.92:6807', 'nonce': 1760163933}]}, 'public_addr': '172.21.15.92:6803/1760163933', 'cluster_addr': '172.21.15.92:6805/1760163933', 'heartbeat_back_addr': '172.21.15.92:6809/1760163933', 'heartbeat_front_addr': '172.21.15.92:6807/1760163933', 'state': ['exists', 'up']}, {'osd': 8, 'uuid': '479f3885-452e-4231-bdc2-b79668384ba6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6816', 'nonce': 4127602407}, {'type': 'v1', 'addr': '172.21.15.28:6817', 'nonce': 4127602407}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6818', 'nonce': 4127602407}, {'type': 'v1', 'addr': '172.21.15.28:6819', 'nonce': 4127602407}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6822', 'nonce': 4127602407}, {'type': 'v1', 'addr': '172.21.15.28:6823', 'nonce': 4127602407}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6820', 'nonce': 4127602407}, {'type': 'v1', 'addr': '172.21.15.28:6821', 'nonce': 4127602407}]}, 'public_addr': '172.21.15.28:6817/4127602407', 'cluster_addr': '172.21.15.28:6819/4127602407', 'heartbeat_back_addr': '172.21.15.28:6823/4127602407', 'heartbeat_front_addr': '172.21.15.28:6821/4127602407', 'state': ['exists', 'up']}, {'osd': 9, 'uuid': '9a603a47-7c5a-47d4-a298-56b3ac991892', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 26, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6816', 'nonce': 4168729845}, {'type': 'v1', 'addr': '172.21.15.42:6818', 'nonce': 4168729845}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6820', 'nonce': 4168729845}, {'type': 'v1', 'addr': '172.21.15.42:6822', 'nonce': 4168729845}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6828', 'nonce': 4168729845}, {'type': 'v1', 'addr': '172.21.15.42:6830', 'nonce': 4168729845}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6824', 'nonce': 4168729845}, {'type': 'v1', 'addr': '172.21.15.42:6826', 'nonce': 4168729845}]}, 'public_addr': '172.21.15.42:6818/4168729845', 'cluster_addr': '172.21.15.42:6822/4168729845', 'heartbeat_back_addr': '172.21.15.42:6830/4168729845', 'heartbeat_front_addr': '172.21.15.42:6826/4168729845', 'state': ['exists', 'up']}, {'osd': 10, 'uuid': '1cd42945-6ee1-49f0-b17b-4093dd36b4ec', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6816', 'nonce': 4180572718}, {'type': 'v1', 'addr': '172.21.15.66:6817', 'nonce': 4180572718}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6818', 'nonce': 4180572718}, {'type': 'v1', 'addr': '172.21.15.66:6819', 'nonce': 4180572718}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6822', 'nonce': 4180572718}, {'type': 'v1', 'addr': '172.21.15.66:6823', 'nonce': 4180572718}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6820', 'nonce': 4180572718}, {'type': 'v1', 'addr': '172.21.15.66:6821', 'nonce': 4180572718}]}, 'public_addr': '172.21.15.66:6817/4180572718', 'cluster_addr': '172.21.15.66:6819/4180572718', 'heartbeat_back_addr': '172.21.15.66:6823/4180572718', 'heartbeat_front_addr': '172.21.15.66:6821/4180572718', 'state': ['exists', 'up']}, {'osd': 11, 'uuid': 'b41e8100-1d4f-4b7a-991a-7a27bc29feec', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 62, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6818', 'nonce': 295713815}, {'type': 'v1', 'addr': '172.21.15.92:6819', 'nonce': 295713815}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6820', 'nonce': 295713815}, {'type': 'v1', 'addr': '172.21.15.92:6821', 'nonce': 295713815}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6824', 'nonce': 295713815}, {'type': 'v1', 'addr': '172.21.15.92:6825', 'nonce': 295713815}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6822', 'nonce': 295713815}, {'type': 'v1', 'addr': '172.21.15.92:6823', 'nonce': 295713815}]}, 'public_addr': '172.21.15.92:6819/295713815', 'cluster_addr': '172.21.15.92:6821/295713815', 'heartbeat_back_addr': '172.21.15.92:6825/295713815', 'heartbeat_front_addr': '172.21.15.92:6823/295713815', 'state': ['exists', 'up']}, {'osd': 12, 'uuid': '458cf150-32e1-449e-b533-ac6ab5226f85', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6824', 'nonce': 2081141953}, {'type': 'v1', 'addr': '172.21.15.28:6825', 'nonce': 2081141953}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6826', 'nonce': 2081141953}, {'type': 'v1', 'addr': '172.21.15.28:6827', 'nonce': 2081141953}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6830', 'nonce': 2081141953}, {'type': 'v1', 'addr': '172.21.15.28:6831', 'nonce': 2081141953}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6828', 'nonce': 2081141953}, {'type': 'v1', 'addr': '172.21.15.28:6829', 'nonce': 2081141953}]}, 'public_addr': '172.21.15.28:6825/2081141953', 'cluster_addr': '172.21.15.28:6827/2081141953', 'heartbeat_back_addr': '172.21.15.28:6831/2081141953', 'heartbeat_front_addr': '172.21.15.28:6829/2081141953', 'state': ['exists', 'up']}, {'osd': 13, 'uuid': 'f008462e-a735-4845-9ecf-dd7d4a6941af', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 59, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6817', 'nonce': 301360893}, {'type': 'v1', 'addr': '172.21.15.42:6819', 'nonce': 301360893}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6821', 'nonce': 301360893}, {'type': 'v1', 'addr': '172.21.15.42:6823', 'nonce': 301360893}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6829', 'nonce': 301360893}, {'type': 'v1', 'addr': '172.21.15.42:6831', 'nonce': 301360893}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6825', 'nonce': 301360893}, {'type': 'v1', 'addr': '172.21.15.42:6827', 'nonce': 301360893}]}, 'public_addr': '172.21.15.42:6819/301360893', 'cluster_addr': '172.21.15.42:6823/301360893', 'heartbeat_back_addr': '172.21.15.42:6831/301360893', 'heartbeat_front_addr': '172.21.15.42:6827/301360893', 'state': ['exists', 'up']}, {'osd': 14, 'uuid': '6f66ae97-eb8a-48eb-9bcc-20f65fbf690b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 53, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6824', 'nonce': 1534905602}, {'type': 'v1', 'addr': '172.21.15.66:6825', 'nonce': 1534905602}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6826', 'nonce': 1534905602}, {'type': 'v1', 'addr': '172.21.15.66:6827', 'nonce': 1534905602}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6830', 'nonce': 1534905602}, {'type': 'v1', 'addr': '172.21.15.66:6831', 'nonce': 1534905602}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6828', 'nonce': 1534905602}, {'type': 'v1', 'addr': '172.21.15.66:6829', 'nonce': 1534905602}]}, 'public_addr': '172.21.15.66:6825/1534905602', 'cluster_addr': '172.21.15.66:6827/1534905602', 'heartbeat_back_addr': '172.21.15.66:6831/1534905602', 'heartbeat_front_addr': '172.21.15.66:6829/1534905602', 'state': ['exists', 'up']}, {'osd': 15, 'uuid': 'b856bfc8-253b-4112-be5c-f67d61d01768', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6826', 'nonce': 3982634323}, {'type': 'v1', 'addr': '172.21.15.92:6827', 'nonce': 3982634323}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6828', 'nonce': 3982634323}, {'type': 'v1', 'addr': '172.21.15.92:6829', 'nonce': 3982634323}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6832', 'nonce': 3982634323}, {'type': 'v1', 'addr': '172.21.15.92:6833', 'nonce': 3982634323}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6830', 'nonce': 3982634323}, {'type': 'v1', 'addr': '172.21.15.92:6831', 'nonce': 3982634323}]}, 'public_addr': '172.21.15.92:6827/3982634323', 'cluster_addr': '172.21.15.92:6829/3982634323', 'heartbeat_back_addr': '172.21.15.92:6833/3982634323', 'heartbeat_front_addr': '172.21.15.92:6831/3982634323', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.265053+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.243502+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.252440+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.295331+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.213076+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.281193+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.315801+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.329538+0000', 'dead_epoch': 0}, {'osd': 8, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.293303+0000', 'dead_epoch': 0}, {'osd': 9, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.334962+0000', 'dead_epoch': 0}, {'osd': 10, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.233032+0000', 'dead_epoch': 0}, {'osd': 11, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.305138+0000', 'dead_epoch': 0}, {'osd': 12, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.351964+0000', 'dead_epoch': 0}, {'osd': 13, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.241086+0000', 'dead_epoch': 0}, {'osd': 14, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.352515+0000', 'dead_epoch': 0}, {'osd': 15, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T03:38:29.357462+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '2.2', 'mappings': [{'from': 3, 'to': 11}]}, {'pgid': '3.f', 'mappings': [{'from': 2, 'to': 9}, {'from': 5, 'to': 14}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}, 'shecprofile': {'c': '2', 'crush-device-class': '', 'crush-failure-domain': 'osd', 'crush-num-failure-domains': '0', 'crush-osds-per-failure-domain': '0', 'crush-root': 'default', 'k': '4', 'm': '3', 'name': 'shecprofile', 'plugin': 'shec', 'technique': 'multiple'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 10, 'length': 2}, {'begin': 15, 'length': 1}, {'begin': 17, 'length': 1}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-04-26T03:39:23.255 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-04-26T03:39:23.255 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-04-26T03:39:23.352 INFO:tasks.ceph.osd.4.smithi028.stderr:2024-04-26T03:39:23.349+0000 7fd9511b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31963) UID: 0 2024-04-26T03:39:23.363 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:23.363 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:23.372 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-04-26T03:39:23.454 INFO:tasks.ceph.osd.4.smithi028.stderr:2024-04-26T03:39:23.451+0000 7fd9511b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31963) UID: 0 2024-04-26T03:39:23.480 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:23.480 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:23.488 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-04-26T03:39:23.554 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:39:23.553+0000 7f60d3c4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31757) UID: 0 2024-04-26T03:39:23.596 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:23.596 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:23.605 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-04-26T03:39:23.654 INFO:tasks.ceph.osd.4.smithi028.stderr:2024-04-26T03:39:23.653+0000 7fd9511b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31963) UID: 0 2024-04-26T03:39:23.692 INFO:teuthology.orchestra.run.smithi028.stderr:noscrub is set 2024-04-26T03:39:23.694 INFO:tasks.rados.rados.0.smithi092.stdout:310: done (0 left) 2024-04-26T03:39:23.694 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 45 current snap is 20 2024-04-26T03:39:23.694 INFO:tasks.rados.rados.0.smithi092.stdout:311: seq_num 145 ranges {753664=32768} 2024-04-26T03:39:23.698 INFO:tasks.rados.rados.0.smithi092.stdout:311: writing smithi09235267-45 from 753664 to 786432 tid 1 2024-04-26T03:39:23.698 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 47 current snap is 20 2024-04-26T03:39:23.699 INFO:tasks.rados.rados.0.smithi092.stdout:312: seq_num 146 ranges {2113536=49152} 2024-04-26T03:39:23.711 INFO:tasks.rados.rados.0.smithi092.stdout:312: writing smithi09235267-47 from 2113536 to 2162688 tid 1 2024-04-26T03:39:23.711 INFO:tasks.rados.rados.0.smithi092.stdout:313: copy_from oid 37 from oid 41 current snap is 20 2024-04-26T03:39:23.711 INFO:tasks.rados.rados.0.smithi092.stdout:311: finishing write tid 1 to smithi09235267-45 2024-04-26T03:39:23.711 INFO:tasks.rados.rados.0.smithi092.stdout:311: finishing write tid 2 to smithi09235267-45 2024-04-26T03:39:23.711 INFO:tasks.rados.rados.0.smithi092.stdout:311: finishing write tid 3 to smithi09235267-45 2024-04-26T03:39:23.712 INFO:tasks.rados.rados.0.smithi092.stdout:311: oid 45 updating version 0 to 88 2024-04-26T03:39:23.712 INFO:tasks.rados.rados.0.smithi092.stdout:311: oid 45 version 88 is already newer than 87 2024-04-26T03:39:23.712 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 45 v 88 (ObjNum 145 snap 20 seq_num 145) dirty exists 2024-04-26T03:39:23.712 INFO:tasks.rados.rados.0.smithi092.stdout:311: left oid 45 (ObjNum 145 snap 20 seq_num 145) 2024-04-26T03:39:23.712 INFO:tasks.rados.rados.0.smithi092.stdout:311: done (2 left) 2024-04-26T03:39:23.712 INFO:tasks.rados.rados.0.smithi092.stdout:314: rollback oid 38 current snap is 20 2024-04-26T03:39:23.712 INFO:tasks.rados.rados.0.smithi092.stdout:rollback oid 38 to 10 2024-04-26T03:39:23.712 INFO:tasks.rados.rados.0.smithi092.stdout:315: rollback oid 31 current snap is 20 2024-04-26T03:39:23.712 INFO:tasks.rados.rados.0.smithi092.stdout:rollback oid 31 to 14 2024-04-26T03:39:23.712 INFO:tasks.rados.rados.0.smithi092.stdout:316: snap_remove snap 11 2024-04-26T03:39:23.716 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:23.716 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:23.727 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-04-26T03:39:23.755 INFO:tasks.ceph.osd.15.smithi092.stderr:2024-04-26T03:39:23.755+0000 7f6d12dec640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 31824) UID: 0 2024-04-26T03:39:23.833 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:23.834 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:23.842 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-04-26T03:39:23.886 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-04-26T03:39:23.885+0000 7fe9223c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31954) UID: 0 2024-04-26T03:39:23.949 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:23.949 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:23.957 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-26T03:39:23.958 INFO:tasks.ceph.osd.15.smithi092.stderr:2024-04-26T03:39:23.955+0000 7f6d12dec640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 31824) UID: 0 2024-04-26T03:39:24.058 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-04-26T03:39:24.057+0000 7fe9223c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31954) UID: 0 2024-04-26T03:39:24.077 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:24.077 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:24.086 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-26T03:39:24.158 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-04-26T03:39:24.157+0000 7fee562fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31759) UID: 0 2024-04-26T03:39:24.195 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:24.195 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:24.203 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-26T03:39:24.259 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-04-26T03:39:24.257+0000 7fdc2d21c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31819) UID: 0 2024-04-26T03:39:24.355 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:24.355 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:24.359 INFO:tasks.ceph.osd.14.smithi066.stderr:2024-04-26T03:39:24.359+0000 7f5395f08640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 31765) UID: 0 2024-04-26T03:39:24.362 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-26T03:39:24.461 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:39:24.460+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:39:24.509 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:24.509 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:24.517 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-26T03:39:24.561 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:39:24.560+0000 7fb78c544640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31956) UID: 0 2024-04-26T03:39:24.624 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:24.624 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:24.633 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-26T03:39:24.663 INFO:tasks.ceph.osd.15.smithi092.stderr:2024-04-26T03:39:24.660+0000 7f6d12dec640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 31824) UID: 0 2024-04-26T03:39:24.693 INFO:tasks.rados.rados.0.smithi092.stdout:316: done (4 left) 2024-04-26T03:39:24.693 INFO:tasks.rados.rados.0.smithi092.stdout:317: read oid 40 snap -1 2024-04-26T03:39:24.693 INFO:tasks.rados.rados.0.smithi092.stdout:317: expect (ObjNum 60 snap 0 seq_num 60) 2024-04-26T03:39:24.693 INFO:tasks.rados.rados.0.smithi092.stdout:318: snap_create 2024-04-26T03:39:24.695 INFO:tasks.rados.rados.0.smithi092.stdout:314: finishing rollback tid 0 to smithi09235267-38 2024-04-26T03:39:24.695 INFO:tasks.rados.rados.0.smithi092.stdout:315: finishing rollback tid 1 to smithi09235267-31 2024-04-26T03:39:24.695 INFO:tasks.rados.rados.0.smithi092.stdout:315: finishing rollback tid 2 to smithi09235267-31 2024-04-26T03:39:24.695 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 31 v 82 (ObjNum 93 snap 7 seq_num 93) dirty exists 2024-04-26T03:39:24.695 INFO:tasks.rados.rados.0.smithi092.stdout:314: finishing rollback tid 1 to smithi09235267-38 2024-04-26T03:39:24.695 INFO:tasks.rados.rados.0.smithi092.stdout:312: finishing write tid 1 to smithi09235267-47 2024-04-26T03:39:24.695 INFO:tasks.rados.rados.0.smithi092.stdout:314: finishing rollback tid 2 to smithi09235267-38 2024-04-26T03:39:24.695 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 38 v 67 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-04-26T03:39:24.695 INFO:tasks.rados.rados.0.smithi092.stdout:312: finishing write tid 2 to smithi09235267-47 2024-04-26T03:39:24.696 INFO:tasks.rados.rados.0.smithi092.stdout:312: finishing write tid 3 to smithi09235267-47 2024-04-26T03:39:24.696 INFO:tasks.rados.rados.0.smithi092.stdout:312: oid 47 updating version 0 to 48 2024-04-26T03:39:24.696 INFO:tasks.rados.rados.0.smithi092.stdout:312: oid 47 version 48 is already newer than 47 2024-04-26T03:39:24.696 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 47 v 48 (ObjNum 146 snap 20 seq_num 146) dirty exists 2024-04-26T03:39:24.696 INFO:tasks.rados.rados.0.smithi092.stdout:312: left oid 47 (ObjNum 146 snap 20 seq_num 146) 2024-04-26T03:39:24.696 INFO:tasks.rados.rados.0.smithi092.stdout:313: finishing copy_from racing read to smithi09235267-37 2024-04-26T03:39:24.696 INFO:tasks.rados.rados.0.smithi092.stdout:313: finishing copy_from to smithi09235267-37 2024-04-26T03:39:24.696 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 37 v 10 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-04-26T03:39:24.718 INFO:tasks.rados.rados.0.smithi092.stdout:317: expect (ObjNum 60 snap 0 seq_num 60) 2024-04-26T03:39:24.753 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:24.754 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:24.763 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-04-26T03:39:24.763 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-04-26T03:39:24.760+0000 7fdc2d21c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31819) UID: 0 2024-04-26T03:39:24.863 INFO:tasks.ceph.osd.9.smithi042.stderr:2024-04-26T03:39:24.862+0000 7fdac63b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 31960) UID: 0 2024-04-26T03:39:24.869 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-26T03:39:24.870 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:24.870 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:24.878 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-04-26T03:39:24.964 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-04-26T03:39:24.962+0000 7f745118d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 31821) UID: 0 2024-04-26T03:39:24.985 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:24.985 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:24.995 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-04-26T03:39:25.003 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0: {} 2024-04-26T03:39:25.004 INFO:teuthology.orchestra.run.smithi028.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T03:39:25.023 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1: {} 2024-04-26T03:39:25.023 INFO:teuthology.orchestra.run.smithi028.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T03:39:25.040 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2: {} 2024-04-26T03:39:25.041 INFO:teuthology.orchestra.run.smithi028.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T03:39:25.058 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3: {} 2024-04-26T03:39:25.059 INFO:teuthology.orchestra.run.smithi028.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T03:39:25.065 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-04-26T03:39:25.063+0000 7f745118d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 31821) UID: 0 2024-04-26T03:39:25.075 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4: {} 2024-04-26T03:39:25.075 INFO:teuthology.orchestra.run.smithi028.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T03:39:25.093 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5: {} 2024-04-26T03:39:25.093 INFO:teuthology.orchestra.run.smithi028.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T03:39:25.101 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:25.101 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:25.110 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-04-26T03:39:25.111 INFO:teuthology.orchestra.run.smithi028.stdout:osd.6: {} 2024-04-26T03:39:25.111 INFO:teuthology.orchestra.run.smithi028.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T03:39:25.128 INFO:teuthology.orchestra.run.smithi028.stdout:osd.7: {} 2024-04-26T03:39:25.129 INFO:teuthology.orchestra.run.smithi028.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T03:39:25.145 INFO:teuthology.orchestra.run.smithi028.stdout:osd.8: {} 2024-04-26T03:39:25.146 INFO:teuthology.orchestra.run.smithi028.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T03:39:25.163 INFO:teuthology.orchestra.run.smithi028.stdout:osd.9: {} 2024-04-26T03:39:25.163 INFO:teuthology.orchestra.run.smithi028.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T03:39:25.166 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-04-26T03:39:25.163+0000 7fdc2d21c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31819) UID: 0 2024-04-26T03:39:25.183 INFO:teuthology.orchestra.run.smithi028.stdout:osd.10: {} 2024-04-26T03:39:25.183 INFO:teuthology.orchestra.run.smithi028.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T03:39:25.219 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:25.219 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:25.229 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-04-26T03:39:25.272 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:39:25.271+0000 7f60d3c4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31757) UID: 0 2024-04-26T03:39:25.335 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:25.336 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:25.346 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-04-26T03:39:25.367 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-04-26T03:39:25.366+0000 7fe9223c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31954) UID: 0 2024-04-26T03:39:25.402 INFO:teuthology.orchestra.run.smithi028.stdout:osd.11: {} 2024-04-26T03:39:25.402 INFO:teuthology.orchestra.run.smithi028.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T03:39:25.419 INFO:teuthology.orchestra.run.smithi028.stdout:osd.12: {} 2024-04-26T03:39:25.419 INFO:teuthology.orchestra.run.smithi028.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T03:39:25.437 INFO:teuthology.orchestra.run.smithi028.stdout:osd.13: {} 2024-04-26T03:39:25.437 INFO:teuthology.orchestra.run.smithi028.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T03:39:25.454 INFO:teuthology.orchestra.run.smithi028.stdout:osd.14: {} 2024-04-26T03:39:25.455 INFO:teuthology.orchestra.run.smithi028.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T03:39:25.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:39:25.453+0000 7f6152266640 0 -- 172.21.15.28:0/3415531933 >> [v2:172.21.15.92:6826/3982634323,v1:172.21.15.92:6827/3982634323] conn(0x7f614c223200 msgr2=0x7f614c2255f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-26T03:39:25.468 INFO:tasks.ceph.osd.12.smithi028.stderr:2024-04-26T03:39:25.466+0000 7f114fe14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 31967) UID: 0 2024-04-26T03:39:25.495 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:39:25.495 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-04-26T03:39:25.495 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-04-26T03:39:25.495 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-04-26T03:39:25.495 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.496 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:3 3.2s0 3:42fcabaa:::smithi09235267-1:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:25.496 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.480913+0000", 2024-04-26T03:39:25.496 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 34.011775790000002, 2024-04-26T03:39:25.496 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.012266038999999999, 2024-04-26T03:39:25.496 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:25.496 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:25.496 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:25.496 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:25.496 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:25.496 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:25.496 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 3 2024-04-26T03:39:25.496 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.496 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:25.496 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.497 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:25.497 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.480913+0000", 2024-04-26T03:39:25.497 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:25.497 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.497 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.497 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:25.497 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.480913+0000", 2024-04-26T03:39:25.497 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:25.497 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.497 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.497 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:25.497 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.480914+0000", 2024-04-26T03:39:25.497 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.7999999999999996e-07 2024-04-26T03:39:25.497 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.498 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.498 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:25.498 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.480917+0000", 2024-04-26T03:39:25.498 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.366e-06 2024-04-26T03:39:25.498 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.498 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.498 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:25.498 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.480925+0000", 2024-04-26T03:39:25.498 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9759999999999995e-06 2024-04-26T03:39:25.498 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.498 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.498 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:25.498 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.480932+0000", 2024-04-26T03:39:25.499 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.0339999999999999e-06 2024-04-26T03:39:25.499 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.499 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.499 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:25.499 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.483324+0000", 2024-04-26T03:39:25.499 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.002392323 2024-04-26T03:39:25.499 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.499 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.499 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:25.499 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.483402+0000", 2024-04-26T03:39:25.499 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.7745999999999997e-05 2024-04-26T03:39:25.499 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.499 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.499 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:25.500 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.487467+0000", 2024-04-26T03:39:25.500 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0040650310000000002 2024-04-26T03:39:25.500 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.500 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.500 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:25.500 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.487555+0000", 2024-04-26T03:39:25.500 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.8271999999999998e-05 2024-04-26T03:39:25.500 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.500 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.500 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:25.500 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.493179+0000", 2024-04-26T03:39:25.500 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0056238110000000003 2024-04-26T03:39:25.500 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:25.500 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:25.501 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:25.501 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.501 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.501 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:79 3.2s0 3:41b9d80c:::smithi09235267-27:head [append 0~442368,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:25.501 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.531542+0000", 2024-04-26T03:39:25.501 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 33.961146737, 2024-04-26T03:39:25.501 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.013543507999999999, 2024-04-26T03:39:25.501 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:25.501 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:25.501 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:25.501 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:25.501 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:25.501 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:25.501 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 79 2024-04-26T03:39:25.502 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.502 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:25.502 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.502 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:25.502 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.531542+0000", 2024-04-26T03:39:25.502 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:25.502 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.502 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.502 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:25.502 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.531542+0000", 2024-04-26T03:39:25.502 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:25.502 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.502 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.502 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:25.502 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.531544+0000", 2024-04-26T03:39:25.503 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9719999999999999e-06 2024-04-26T03:39:25.503 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.503 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.503 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:25.503 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.531978+0000", 2024-04-26T03:39:25.503 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00043396899999999998 2024-04-26T03:39:25.503 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.503 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.503 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:25.503 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.531990+0000", 2024-04-26T03:39:25.503 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2150999999999999e-05 2024-04-26T03:39:25.503 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.503 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.503 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:25.504 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.531997+0000", 2024-04-26T03:39:25.504 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.6320000000000002e-06 2024-04-26T03:39:25.504 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.504 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.504 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:25.504 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532044+0000", 2024-04-26T03:39:25.504 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.7095999999999998e-05 2024-04-26T03:39:25.504 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.504 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.504 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:25.504 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532156+0000", 2024-04-26T03:39:25.504 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000112131 2024-04-26T03:39:25.505 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.505 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.505 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:25.505 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532990+0000", 2024-04-26T03:39:25.505 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00083437099999999998 2024-04-26T03:39:25.505 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.505 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.505 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:25.505 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.537471+0000", 2024-04-26T03:39:25.505 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0044802160000000004 2024-04-26T03:39:25.505 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.505 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.505 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:25.506 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.545008+0000", 2024-04-26T03:39:25.506 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0075372089999999996 2024-04-26T03:39:25.506 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.506 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.506 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:25.506 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.545086+0000", 2024-04-26T03:39:25.506 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.7761000000000003e-05 2024-04-26T03:39:25.506 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:25.506 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:25.506 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:25.506 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.506 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.506 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:80 3.2s0 3:41b9d80c:::smithi09235267-27:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:25.507 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.532011+0000", 2024-04-26T03:39:25.507 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 33.960677408999999, 2024-04-26T03:39:25.507 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.013370922, 2024-04-26T03:39:25.507 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:25.507 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:25.507 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:25.507 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:25.507 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:25.507 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:25.507 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 80 2024-04-26T03:39:25.507 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.507 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:25.507 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.507 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:25.508 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532011+0000", 2024-04-26T03:39:25.508 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:25.508 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.508 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.508 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:25.508 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532011+0000", 2024-04-26T03:39:25.508 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:25.508 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.508 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.508 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:25.508 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532012+0000", 2024-04-26T03:39:25.508 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.1900000000000003e-07 2024-04-26T03:39:25.509 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.509 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.509 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:25.509 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532015+0000", 2024-04-26T03:39:25.509 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5919999999999999e-06 2024-04-26T03:39:25.509 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.509 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.509 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:25.509 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532025+0000", 2024-04-26T03:39:25.509 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0132999999999999e-05 2024-04-26T03:39:25.509 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.509 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.509 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:25.509 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532029+0000", 2024-04-26T03:39:25.510 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9820000000000002e-06 2024-04-26T03:39:25.510 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.510 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.510 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:25.510 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.536416+0000", 2024-04-26T03:39:25.510 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0043876480000000001 2024-04-26T03:39:25.510 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.510 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.510 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:25.510 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.536558+0000", 2024-04-26T03:39:25.510 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000141994 2024-04-26T03:39:25.510 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.510 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.511 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:25.511 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.537081+0000", 2024-04-26T03:39:25.511 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00052260200000000005 2024-04-26T03:39:25.511 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.511 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.511 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:25.511 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.537579+0000", 2024-04-26T03:39:25.511 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00049790100000000003 2024-04-26T03:39:25.511 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.511 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.511 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:25.511 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.545304+0000", 2024-04-26T03:39:25.511 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.007724936 2024-04-26T03:39:25.511 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.512 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.512 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:25.512 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.545382+0000", 2024-04-26T03:39:25.512 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8615000000000005e-05 2024-04-26T03:39:25.512 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:25.512 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:25.512 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:25.512 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.512 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.512 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:81 3.2s0 3:41b9d80c:::smithi09235267-27:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:25.512 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.532034+0000", 2024-04-26T03:39:25.513 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 33.960654495, 2024-04-26T03:39:25.513 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.015088236, 2024-04-26T03:39:25.513 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:25.513 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:25.513 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:25.513 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:25.513 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:25.513 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:25.513 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 81 2024-04-26T03:39:25.513 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.513 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:25.513 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.513 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:25.513 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532034+0000", 2024-04-26T03:39:25.514 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:25.514 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.514 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.514 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:25.514 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532034+0000", 2024-04-26T03:39:25.514 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:25.514 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.514 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.514 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:25.514 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532035+0000", 2024-04-26T03:39:25.514 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.22e-07 2024-04-26T03:39:25.514 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.514 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.515 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:25.515 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532036+0000", 2024-04-26T03:39:25.515 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6279999999999999e-06 2024-04-26T03:39:25.515 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.515 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.515 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:25.515 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532041+0000", 2024-04-26T03:39:25.515 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.391e-06 2024-04-26T03:39:25.515 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.515 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.515 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:25.515 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532046+0000", 2024-04-26T03:39:25.515 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.4090000000000002e-06 2024-04-26T03:39:25.515 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.515 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.516 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:25.516 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.537306+0000", 2024-04-26T03:39:25.516 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0052594319999999996 2024-04-26T03:39:25.516 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.516 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.516 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:25.516 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.537400+0000", 2024-04-26T03:39:25.516 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.4753999999999994e-05 2024-04-26T03:39:25.516 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.516 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.516 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:25.516 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.545720+0000", 2024-04-26T03:39:25.516 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.008319557 2024-04-26T03:39:25.516 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.517 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.517 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:25.517 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.545802+0000", 2024-04-26T03:39:25.517 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.1898999999999996e-05 2024-04-26T03:39:25.517 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.517 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.517 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:25.517 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.547123+0000", 2024-04-26T03:39:25.517 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0013208440000000001 2024-04-26T03:39:25.517 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:25.517 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:25.517 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:25.517 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.517 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.517 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:139 3.2s0 3:450a1cc5:::smithi09235267-47:head [append 0~786432,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:25.518 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.619094+0000", 2024-04-26T03:39:25.518 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 33.873594449000002, 2024-04-26T03:39:25.518 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.015918081000000001, 2024-04-26T03:39:25.518 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:25.518 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:25.518 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:25.518 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:25.518 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:25.518 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:25.518 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 139 2024-04-26T03:39:25.518 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.518 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:25.518 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.519 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:25.519 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619094+0000", 2024-04-26T03:39:25.519 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:25.519 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.519 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.519 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:25.519 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619094+0000", 2024-04-26T03:39:25.519 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:25.519 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.519 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.519 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:25.519 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619096+0000", 2024-04-26T03:39:25.519 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6559999999999999e-06 2024-04-26T03:39:25.520 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.520 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.520 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:25.520 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619809+0000", 2024-04-26T03:39:25.520 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00071288600000000003 2024-04-26T03:39:25.520 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.520 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.520 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:25.520 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619821+0000", 2024-04-26T03:39:25.520 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1586000000000001e-05 2024-04-26T03:39:25.520 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.520 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.520 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:25.520 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619826+0000", 2024-04-26T03:39:25.521 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.3870000000000003e-06 2024-04-26T03:39:25.521 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.521 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.521 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:25.521 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619879+0000", 2024-04-26T03:39:25.521 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.2660000000000001e-05 2024-04-26T03:39:25.521 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.521 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.521 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:25.521 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619966+0000", 2024-04-26T03:39:25.521 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7777999999999999e-05 2024-04-26T03:39:25.521 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.521 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.521 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:25.522 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.620812+0000", 2024-04-26T03:39:25.522 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00084571199999999998 2024-04-26T03:39:25.522 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.522 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.522 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:25.522 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.627482+0000", 2024-04-26T03:39:25.522 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0066701590000000002 2024-04-26T03:39:25.522 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.522 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.522 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:25.522 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.634919+0000", 2024-04-26T03:39:25.522 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.007437056 2024-04-26T03:39:25.523 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.523 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.523 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:25.523 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.635013+0000", 2024-04-26T03:39:25.523 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.3201000000000004e-05 2024-04-26T03:39:25.523 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:25.523 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:25.523 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:25.523 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.523 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.523 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:140 3.2s0 3:450a1cc5:::smithi09235267-47:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:25.523 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.619840+0000", 2024-04-26T03:39:25.523 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 33.872849387999999, 2024-04-26T03:39:25.523 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.015356137000000001, 2024-04-26T03:39:25.524 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:25.524 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:25.524 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:25.524 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:25.524 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:25.524 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:25.524 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 140 2024-04-26T03:39:25.524 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.524 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:25.524 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.524 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:25.524 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619840+0000", 2024-04-26T03:39:25.524 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:25.524 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.525 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.525 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:25.525 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619840+0000", 2024-04-26T03:39:25.525 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:25.525 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.525 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.525 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:25.525 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619840+0000", 2024-04-26T03:39:25.525 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.8100000000000002e-07 2024-04-26T03:39:25.525 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.525 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.525 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:25.525 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619843+0000", 2024-04-26T03:39:25.525 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.785e-06 2024-04-26T03:39:25.526 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.526 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.526 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:25.526 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619848+0000", 2024-04-26T03:39:25.526 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.2490000000000001e-06 2024-04-26T03:39:25.526 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.526 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.526 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:25.526 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619852+0000", 2024-04-26T03:39:25.526 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.1869999999999999e-06 2024-04-26T03:39:25.526 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.527 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.527 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:25.527 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.626570+0000", 2024-04-26T03:39:25.527 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0067177679999999998 2024-04-26T03:39:25.527 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.527 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.527 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:25.527 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.626669+0000", 2024-04-26T03:39:25.527 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.9177000000000003e-05 2024-04-26T03:39:25.527 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.527 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.527 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:25.527 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.627020+0000", 2024-04-26T03:39:25.527 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00035101199999999999 2024-04-26T03:39:25.528 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.528 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.528 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:25.528 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.627486+0000", 2024-04-26T03:39:25.528 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00046560299999999998 2024-04-26T03:39:25.528 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.528 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.528 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:25.528 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.635101+0000", 2024-04-26T03:39:25.528 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0076154680000000002 2024-04-26T03:39:25.528 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.528 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.528 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:25.528 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.635196+0000", 2024-04-26T03:39:25.529 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.4207000000000001e-05 2024-04-26T03:39:25.529 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:25.529 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:25.529 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:25.529 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.529 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.529 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:141 3.2s0 3:450a1cc5:::smithi09235267-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:25.529 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.619858+0000", 2024-04-26T03:39:25.529 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 33.872831056000003, 2024-04-26T03:39:25.529 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.017889577, 2024-04-26T03:39:25.529 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:25.529 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:25.529 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:25.529 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:25.530 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:25.530 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:25.530 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 141 2024-04-26T03:39:25.530 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.530 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:25.530 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.530 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:25.530 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619858+0000", 2024-04-26T03:39:25.530 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:25.530 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.530 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.530 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:25.530 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619858+0000", 2024-04-26T03:39:25.531 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:25.531 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.531 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.531 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:25.531 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619858+0000", 2024-04-26T03:39:25.531 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7000000000000001e-07 2024-04-26T03:39:25.531 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.531 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.531 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:25.531 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619860+0000", 2024-04-26T03:39:25.531 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1090000000000001e-06 2024-04-26T03:39:25.531 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.531 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.532 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:25.532 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619864+0000", 2024-04-26T03:39:25.532 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.1330000000000001e-06 2024-04-26T03:39:25.532 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.532 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.532 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:25.532 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619867+0000", 2024-04-26T03:39:25.533 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.9100000000000001e-06 2024-04-26T03:39:25.533 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.533 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.533 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:25.533 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.627233+0000", 2024-04-26T03:39:25.533 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0073655659999999996 2024-04-26T03:39:25.533 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.533 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.533 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:25.533 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.627322+0000", 2024-04-26T03:39:25.533 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.9456999999999994e-05 2024-04-26T03:39:25.534 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.534 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.534 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:25.534 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.635530+0000", 2024-04-26T03:39:25.534 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0082076349999999996 2024-04-26T03:39:25.534 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.534 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.534 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:25.534 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.635609+0000", 2024-04-26T03:39:25.534 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8724e-05 2024-04-26T03:39:25.534 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.534 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.534 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:25.535 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.637747+0000", 2024-04-26T03:39:25.535 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0021387730000000001 2024-04-26T03:39:25.535 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:25.535 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:25.535 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:25.535 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.535 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.535 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:223 3.2s0 3:42fcabaa:::smithi09235267-1:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:25.535 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.838898+0000", 2024-04-26T03:39:25.535 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 33.653790532000002, 2024-04-26T03:39:25.535 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.012225915, 2024-04-26T03:39:25.535 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:25.535 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:25.535 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:25.536 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:25.536 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:25.536 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:25.536 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 223 2024-04-26T03:39:25.536 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.536 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:25.536 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.536 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:25.536 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.838898+0000", 2024-04-26T03:39:25.536 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:25.536 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.536 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.536 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:25.537 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.838898+0000", 2024-04-26T03:39:25.537 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:25.537 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.537 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.537 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:25.537 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.838899+0000", 2024-04-26T03:39:25.537 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.7e-07 2024-04-26T03:39:25.537 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.537 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.537 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:25.537 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.838901+0000", 2024-04-26T03:39:25.537 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5799999999999999e-06 2024-04-26T03:39:25.537 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.538 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.538 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:25.538 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.838910+0000", 2024-04-26T03:39:25.538 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.1189999999999997e-06 2024-04-26T03:39:25.538 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.538 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.538 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:25.538 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.838916+0000", 2024-04-26T03:39:25.538 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.8630000000000002e-06 2024-04-26T03:39:25.538 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.538 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.538 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:25.539 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.845205+0000", 2024-04-26T03:39:25.539 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0062883449999999999 2024-04-26T03:39:25.539 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.539 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.539 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:25.539 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.845254+0000", 2024-04-26T03:39:25.539 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.9101000000000003e-05 2024-04-26T03:39:25.539 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.539 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.539 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:25.539 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.850264+0000", 2024-04-26T03:39:25.539 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0050106079999999997 2024-04-26T03:39:25.539 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.539 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.540 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:25.540 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.850341+0000", 2024-04-26T03:39:25.540 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.6654000000000002e-05 2024-04-26T03:39:25.540 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.540 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.540 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:25.540 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.851124+0000", 2024-04-26T03:39:25.540 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000783175 2024-04-26T03:39:25.540 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:25.540 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:25.540 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:25.540 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.540 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.541 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.es1 43/32 ECSubWrite(tid=48, reqid=client.4556.0:301, at_version=43'16, trim_to=0'0, roll_forward_to=35'14))", 2024-04-26T03:39:25.541 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:59.531315+0000", 2024-04-26T03:39:25.541 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 25.961373581, 2024-04-26T03:39:25.541 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.014140454, 2024-04-26T03:39:25.541 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:25.541 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:25.541 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:25.541 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:25.541 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.541 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:25.541 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.531315+0000", 2024-04-26T03:39:25.541 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:25.541 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.541 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.542 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:25.542 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.531315+0000", 2024-04-26T03:39:25.542 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:25.542 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.542 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.542 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:25.542 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.531316+0000", 2024-04-26T03:39:25.542 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1799999999999999e-07 2024-04-26T03:39:25.542 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.542 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.542 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:25.542 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.531691+0000", 2024-04-26T03:39:25.542 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00037520700000000001 2024-04-26T03:39:25.543 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.543 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.543 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:25.543 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.531705+0000", 2024-04-26T03:39:25.543 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.395e-05 2024-04-26T03:39:25.543 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.543 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.543 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:25.543 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.531713+0000", 2024-04-26T03:39:25.543 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.0150000000000003e-06 2024-04-26T03:39:25.543 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.543 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.543 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:25.543 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.531798+0000", 2024-04-26T03:39:25.544 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5714e-05 2024-04-26T03:39:25.544 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.544 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.544 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:25.544 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.531811+0000", 2024-04-26T03:39:25.544 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2782000000000001e-05 2024-04-26T03:39:25.544 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.544 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.544 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:25.544 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.545405+0000", 2024-04-26T03:39:25.544 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.013593888 2024-04-26T03:39:25.544 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.544 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.544 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:25.545 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.545456+0000", 2024-04-26T03:39:25.545 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.0680000000000003e-05 2024-04-26T03:39:25.545 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:25.545 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:25.545 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:25.545 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.545 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.545 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.fs3 45/32 ECSubWrite(tid=49, reqid=client.4556.0:325, at_version=45'17, trim_to=0'0, roll_forward_to=42'15))", 2024-04-26T03:39:25.545 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:01.554312+0000", 2024-04-26T03:39:25.545 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 23.938376485999999, 2024-04-26T03:39:25.545 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.012529861, 2024-04-26T03:39:25.545 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:25.546 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:25.546 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:25.546 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:25.546 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.546 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:25.546 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.554312+0000", 2024-04-26T03:39:25.546 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:25.546 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.546 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.547 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:25.547 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.554312+0000", 2024-04-26T03:39:25.547 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:25.547 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.547 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.548 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:25.548 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.554313+0000", 2024-04-26T03:39:25.548 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.9999999999999996e-07 2024-04-26T03:39:25.548 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.548 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.548 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:25.548 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.554865+0000", 2024-04-26T03:39:25.548 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00055152399999999996 2024-04-26T03:39:25.548 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.548 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.548 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:25.548 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.554886+0000", 2024-04-26T03:39:25.549 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1418999999999999e-05 2024-04-26T03:39:25.549 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.549 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.549 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:25.549 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.554897+0000", 2024-04-26T03:39:25.549 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0994e-05 2024-04-26T03:39:25.549 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.549 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.549 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:25.549 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.554971+0000", 2024-04-26T03:39:25.549 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3516000000000006e-05 2024-04-26T03:39:25.549 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.549 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.549 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:25.550 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.554983+0000", 2024-04-26T03:39:25.550 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2133999999999999e-05 2024-04-26T03:39:25.550 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.550 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.550 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:25.550 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.566763+0000", 2024-04-26T03:39:25.550 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.011780067 2024-04-26T03:39:25.550 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.550 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.550 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:25.550 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.566842+0000", 2024-04-26T03:39:25.551 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9307000000000005e-05 2024-04-26T03:39:25.551 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:25.551 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:25.551 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:25.551 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.551 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.551 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:333 3.2s0 3:450a1cc5:::smithi09235267-47:head [copy-from ver 6] snapc 7=[7,6,5] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-26T03:39:25.551 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:01.558145+0000", 2024-04-26T03:39:25.551 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 23.934544147, 2024-04-26T03:39:25.551 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.017282072999999998, 2024-04-26T03:39:25.551 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:25.551 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:25.551 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:25.551 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:25.552 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:25.552 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:25.552 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 333 2024-04-26T03:39:25.552 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.552 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:25.552 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.552 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:25.552 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558145+0000", 2024-04-26T03:39:25.552 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:25.552 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.552 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.552 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:25.553 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558145+0000", 2024-04-26T03:39:25.553 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:25.553 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.553 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.553 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:25.553 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558147+0000", 2024-04-26T03:39:25.553 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4360000000000001e-06 2024-04-26T03:39:25.553 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.553 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.553 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:25.553 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558166+0000", 2024-04-26T03:39:25.553 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8411999999999999e-05 2024-04-26T03:39:25.554 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.554 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.554 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:25.554 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558181+0000", 2024-04-26T03:39:25.554 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4956e-05 2024-04-26T03:39:25.554 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.554 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.554 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:25.554 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558190+0000", 2024-04-26T03:39:25.554 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.7599999999999997e-06 2024-04-26T03:39:25.554 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.555 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.555 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:25.555 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558309+0000", 2024-04-26T03:39:25.555 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000118893 2024-04-26T03:39:25.555 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.555 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.555 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:25.555 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558491+0000", 2024-04-26T03:39:25.555 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00018157000000000001 2024-04-26T03:39:25.555 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.555 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.555 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:25.556 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.566556+0000", 2024-04-26T03:39:25.556 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0080647549999999998 2024-04-26T03:39:25.556 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.556 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.556 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:25.556 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.572889+0000", 2024-04-26T03:39:25.556 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0063337100000000002 2024-04-26T03:39:25.556 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.556 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.556 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:25.556 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.575342+0000", 2024-04-26T03:39:25.557 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0024523269999999998 2024-04-26T03:39:25.557 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.557 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.557 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:25.557 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.575427+0000", 2024-04-26T03:39:25.557 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5253999999999994e-05 2024-04-26T03:39:25.557 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:25.557 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:25.557 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:25.557 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.557 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.557 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:334 3.2s0 3:450a1cc5:::smithi09235267-47:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e45)", 2024-04-26T03:39:25.558 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:01.558215+0000", 2024-04-26T03:39:25.558 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 23.934473996000001, 2024-04-26T03:39:25.558 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.013358238999999999, 2024-04-26T03:39:25.558 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:25.558 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:25.558 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:25.558 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:25.558 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:25.558 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:25.558 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 334 2024-04-26T03:39:25.558 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.558 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:25.558 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.558 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:25.559 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558215+0000", 2024-04-26T03:39:25.559 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:25.559 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.559 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.559 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:25.559 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558215+0000", 2024-04-26T03:39:25.559 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:25.559 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.559 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.559 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:25.559 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558216+0000", 2024-04-26T03:39:25.559 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.0800000000000004e-07 2024-04-26T03:39:25.559 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.559 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.559 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:25.559 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558220+0000", 2024-04-26T03:39:25.560 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8410000000000002e-06 2024-04-26T03:39:25.560 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.560 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.560 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:25.560 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558233+0000", 2024-04-26T03:39:25.560 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3556e-05 2024-04-26T03:39:25.560 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.560 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.560 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:25.560 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558241+0000", 2024-04-26T03:39:25.560 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.4080000000000001e-06 2024-04-26T03:39:25.560 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.560 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.560 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:25.560 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558731+0000", 2024-04-26T03:39:25.561 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00049039899999999998 2024-04-26T03:39:25.561 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.561 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.561 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for blocked object", 2024-04-26T03:39:25.561 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558777+0000", 2024-04-26T03:39:25.561 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6301000000000003e-05 2024-04-26T03:39:25.561 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.561 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.561 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:25.561 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.571433+0000", 2024-04-26T03:39:25.561 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.012656137 2024-04-26T03:39:25.561 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.561 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.562 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:25.562 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.571501+0000", 2024-04-26T03:39:25.562 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7319999999999999e-05 2024-04-26T03:39:25.562 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.562 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.562 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:25.562 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.571573+0000", 2024-04-26T03:39:25.562 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.2469e-05 2024-04-26T03:39:25.562 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:25.562 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:25.562 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:25.563 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.563 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.563 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:433 3.2s0 3:42fcabaa:::smithi09235267-1:head [append 0~671744] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-04-26T03:39:25.563 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:04.692691+0000", 2024-04-26T03:39:25.563 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 20.799997505, 2024-04-26T03:39:25.563 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.026947377000000002, 2024-04-26T03:39:25.563 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:25.563 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:25.563 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:25.563 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:25.563 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:25.564 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:25.564 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 433 2024-04-26T03:39:25.564 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.564 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:25.564 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.564 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:25.564 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.692691+0000", 2024-04-26T03:39:25.564 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:25.564 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.564 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.564 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:25.564 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.692691+0000", 2024-04-26T03:39:25.564 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:25.565 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.565 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.565 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:25.565 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.692694+0000", 2024-04-26T03:39:25.565 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.216e-06 2024-04-26T03:39:25.565 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.565 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.565 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:25.565 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.693610+0000", 2024-04-26T03:39:25.565 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00091655899999999995 2024-04-26T03:39:25.565 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.565 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.565 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:25.565 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.693622+0000", 2024-04-26T03:39:25.566 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1976e-05 2024-04-26T03:39:25.566 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.566 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.566 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:25.566 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.693629+0000", 2024-04-26T03:39:25.566 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5749999999999997e-06 2024-04-26T03:39:25.566 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.566 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.566 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:25.566 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.693698+0000", 2024-04-26T03:39:25.566 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.9529000000000005e-05 2024-04-26T03:39:25.566 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.567 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.567 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op must wait for map", 2024-04-26T03:39:25.567 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.693708+0000", 2024-04-26T03:39:25.567 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.6649999999999998e-06 2024-04-26T03:39:25.567 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.567 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.567 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:25.567 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.698905+0000", 2024-04-26T03:39:25.567 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.005197279 2024-04-26T03:39:25.567 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.567 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.567 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:25.567 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.698993+0000", 2024-04-26T03:39:25.568 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.8029999999999996e-05 2024-04-26T03:39:25.568 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.568 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.568 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:25.568 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.699957+0000", 2024-04-26T03:39:25.568 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00096368199999999999 2024-04-26T03:39:25.568 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.568 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.568 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:25.568 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.718295+0000", 2024-04-26T03:39:25.568 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.018338255000000001 2024-04-26T03:39:25.568 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.568 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.568 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:25.569 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.719556+0000", 2024-04-26T03:39:25.569 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001261086 2024-04-26T03:39:25.569 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.569 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.569 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:25.569 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.719639+0000", 2024-04-26T03:39:25.569 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2525e-05 2024-04-26T03:39:25.569 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:25.569 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:25.569 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:25.569 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.569 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.569 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:434 3.2s0 3:42fcabaa:::smithi09235267-1:head [append 0~770048] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-04-26T03:39:25.569 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:04.699291+0000", 2024-04-26T03:39:25.569 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 20.793397808000002, 2024-04-26T03:39:25.570 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.029702725999999999, 2024-04-26T03:39:25.570 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:25.570 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:25.570 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:25.570 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:25.570 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:25.570 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:25.570 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 434 2024-04-26T03:39:25.570 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.570 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:25.570 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.570 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:25.570 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.699291+0000", 2024-04-26T03:39:25.570 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:25.571 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.571 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.571 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:25.571 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.699291+0000", 2024-04-26T03:39:25.571 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:25.571 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.571 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.571 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:25.571 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.699292+0000", 2024-04-26T03:39:25.571 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.192e-06 2024-04-26T03:39:25.571 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.571 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.572 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:25.572 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700019+0000", 2024-04-26T03:39:25.572 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00072667100000000002 2024-04-26T03:39:25.572 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.572 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.572 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:25.572 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700032+0000", 2024-04-26T03:39:25.572 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3064999999999999e-05 2024-04-26T03:39:25.572 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.572 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.572 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:25.572 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700040+0000", 2024-04-26T03:39:25.572 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5299999999999999e-06 2024-04-26T03:39:25.572 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.572 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.573 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:25.573 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.711078+0000", 2024-04-26T03:39:25.573 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.011038536 2024-04-26T03:39:25.573 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.573 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.573 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:25.573 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.711181+0000", 2024-04-26T03:39:25.573 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000102833 2024-04-26T03:39:25.573 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.573 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.573 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:25.573 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.712518+0000", 2024-04-26T03:39:25.574 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0013371769999999999 2024-04-26T03:39:25.574 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.574 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.574 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:25.574 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.719306+0000", 2024-04-26T03:39:25.574 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0067879009999999998 2024-04-26T03:39:25.574 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.574 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.574 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:25.574 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728932+0000", 2024-04-26T03:39:25.574 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0096258230000000004 2024-04-26T03:39:25.575 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.575 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.575 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:25.575 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728994+0000", 2024-04-26T03:39:25.575 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.1997999999999998e-05 2024-04-26T03:39:25.575 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:25.575 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:25.575 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:25.575 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.575 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.575 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:435 3.2s0 3:42fcabaa:::smithi09235267-1:head [setxattr _header (55) in=7b] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-04-26T03:39:25.575 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:04.700054+0000", 2024-04-26T03:39:25.575 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 20.792634550999999, 2024-04-26T03:39:25.575 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.029111279, 2024-04-26T03:39:25.576 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:25.576 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:25.576 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:25.576 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:25.576 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:25.576 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:25.576 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 435 2024-04-26T03:39:25.576 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.576 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:25.576 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.576 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:25.576 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700054+0000", 2024-04-26T03:39:25.577 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:25.577 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.577 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.577 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:25.577 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700054+0000", 2024-04-26T03:39:25.577 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:25.577 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.577 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.577 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:25.577 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700055+0000", 2024-04-26T03:39:25.577 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0470000000000001e-06 2024-04-26T03:39:25.577 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.577 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.578 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:25.578 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700059+0000", 2024-04-26T03:39:25.578 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8909999999999997e-06 2024-04-26T03:39:25.578 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.578 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.578 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:25.578 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700067+0000", 2024-04-26T03:39:25.578 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.447e-06 2024-04-26T03:39:25.578 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.578 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.579 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:25.579 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700072+0000", 2024-04-26T03:39:25.579 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7529999999999998e-06 2024-04-26T03:39:25.579 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.579 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.579 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:25.579 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.718645+0000", 2024-04-26T03:39:25.579 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.018572564 2024-04-26T03:39:25.579 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.579 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.579 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:25.580 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.718711+0000", 2024-04-26T03:39:25.580 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5743999999999999e-05 2024-04-26T03:39:25.580 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.580 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.580 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:25.580 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.718985+0000", 2024-04-26T03:39:25.580 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00027395400000000003 2024-04-26T03:39:25.580 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.580 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.580 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:25.580 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.719895+0000", 2024-04-26T03:39:25.580 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00091011499999999997 2024-04-26T03:39:25.580 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.580 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.580 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:25.580 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.729090+0000", 2024-04-26T03:39:25.581 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0091950920000000002 2024-04-26T03:39:25.581 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.581 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.581 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:25.581 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.729166+0000", 2024-04-26T03:39:25.581 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5672000000000003e-05 2024-04-26T03:39:25.581 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:25.581 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:25.581 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:25.581 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.581 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.581 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:436 3.2s0 3:42fcabaa:::smithi09235267-1:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e49)", 2024-04-26T03:39:25.581 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:04.700082+0000", 2024-04-26T03:39:25.582 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 20.792606906, 2024-04-26T03:39:25.582 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.032317973, 2024-04-26T03:39:25.582 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:25.582 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:25.582 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:25.582 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:25.582 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:25.582 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:25.582 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 436 2024-04-26T03:39:25.582 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.582 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:25.582 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.583 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:25.583 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700082+0000", 2024-04-26T03:39:25.583 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:25.583 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.583 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.583 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:25.583 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700082+0000", 2024-04-26T03:39:25.583 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:25.583 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.583 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.583 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:25.583 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700082+0000", 2024-04-26T03:39:25.583 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.3000000000000001e-07 2024-04-26T03:39:25.584 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.584 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.584 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:25.584 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700085+0000", 2024-04-26T03:39:25.584 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4739999999999999e-06 2024-04-26T03:39:25.584 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.584 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.584 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:25.584 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700092+0000", 2024-04-26T03:39:25.584 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3000000000000004e-06 2024-04-26T03:39:25.584 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.584 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.585 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:25.585 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700097+0000", 2024-04-26T03:39:25.585 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.2290000000000002e-06 2024-04-26T03:39:25.585 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.585 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.585 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:25.585 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.719230+0000", 2024-04-26T03:39:25.585 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.019132564000000001 2024-04-26T03:39:25.585 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.585 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.585 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:25.585 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.719296+0000", 2024-04-26T03:39:25.585 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.6370000000000003e-05 2024-04-26T03:39:25.585 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.585 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.586 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:25.586 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.729620+0000", 2024-04-26T03:39:25.586 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.010323183999999999 2024-04-26T03:39:25.586 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.586 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.586 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:25.586 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.729695+0000", 2024-04-26T03:39:25.586 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5500000000000006e-05 2024-04-26T03:39:25.586 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.586 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.586 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:25.586 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.732400+0000", 2024-04-26T03:39:25.586 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0027049219999999998 2024-04-26T03:39:25.586 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:25.586 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:25.587 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:25.587 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.587 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.587 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:478 3.2s0 3:450a1cc5:::smithi09235267-47:head [append 0~802816] snapc 8=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:25.587 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:05.910200+0000", 2024-04-26T03:39:25.587 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 19.582489223, 2024-04-26T03:39:25.587 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.013459199999999999, 2024-04-26T03:39:25.587 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:25.587 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:25.587 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:25.587 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:25.587 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:25.587 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:25.587 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 478 2024-04-26T03:39:25.587 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.588 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:25.588 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.588 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:25.588 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.910200+0000", 2024-04-26T03:39:25.588 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:25.588 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.588 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.588 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:25.588 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.910200+0000", 2024-04-26T03:39:25.588 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:25.588 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.588 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.588 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:25.588 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.910202+0000", 2024-04-26T03:39:25.588 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.683e-06 2024-04-26T03:39:25.588 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.588 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.589 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:25.589 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.910944+0000", 2024-04-26T03:39:25.589 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00074142299999999995 2024-04-26T03:39:25.589 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.589 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.589 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:25.589 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.910959+0000", 2024-04-26T03:39:25.589 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4909000000000001e-05 2024-04-26T03:39:25.589 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.589 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.589 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:25.589 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.910966+0000", 2024-04-26T03:39:25.589 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.827e-06 2024-04-26T03:39:25.589 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.589 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.590 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:25.590 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.911019+0000", 2024-04-26T03:39:25.590 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.3817999999999999e-05 2024-04-26T03:39:25.590 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.590 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.590 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:25.590 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.911094+0000", 2024-04-26T03:39:25.590 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5059000000000001e-05 2024-04-26T03:39:25.590 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.590 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.590 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:25.590 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.911822+0000", 2024-04-26T03:39:25.590 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00072724499999999995 2024-04-26T03:39:25.591 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.591 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.591 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:25.591 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.923328+0000", 2024-04-26T03:39:25.591 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.01150674 2024-04-26T03:39:25.591 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.591 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.591 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:25.591 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.923574+0000", 2024-04-26T03:39:25.591 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00024563200000000002 2024-04-26T03:39:25.591 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.591 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.591 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:25.591 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.923659+0000", 2024-04-26T03:39:25.591 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.4864000000000006e-05 2024-04-26T03:39:25.592 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:25.592 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:25.592 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:25.592 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.592 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.592 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:481 3.2s0 3:450a1cc5:::smithi09235267-47:head [setxattr _header (56) in=7b] snapc 8=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:25.592 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:05.914415+0000", 2024-04-26T03:39:25.592 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 19.578274233999998, 2024-04-26T03:39:25.592 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.011349236, 2024-04-26T03:39:25.592 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:25.592 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:25.592 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:25.592 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:25.592 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:25.592 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:25.593 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 481 2024-04-26T03:39:25.593 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.593 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:25.593 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.593 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:25.593 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.914415+0000", 2024-04-26T03:39:25.593 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:25.593 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.593 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.593 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:25.593 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.914415+0000", 2024-04-26T03:39:25.593 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:25.593 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.593 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.594 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:25.594 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.914415+0000", 2024-04-26T03:39:25.594 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.8800000000000003e-07 2024-04-26T03:39:25.594 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.594 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.594 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:25.594 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.914422+0000", 2024-04-26T03:39:25.594 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.9199999999999998e-06 2024-04-26T03:39:25.594 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.594 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.594 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:25.594 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.914431+0000", 2024-04-26T03:39:25.594 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.8559999999999996e-06 2024-04-26T03:39:25.595 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.595 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.595 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:25.595 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.914438+0000", 2024-04-26T03:39:25.595 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.9199999999999998e-06 2024-04-26T03:39:25.595 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.595 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.595 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:25.595 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.924215+0000", 2024-04-26T03:39:25.595 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0097767129999999994 2024-04-26T03:39:25.595 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.595 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.595 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:25.595 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.924272+0000", 2024-04-26T03:39:25.596 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7003000000000003e-05 2024-04-26T03:39:25.596 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.596 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.596 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:25.596 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.924552+0000", 2024-04-26T03:39:25.596 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00028005100000000002 2024-04-26T03:39:25.596 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.596 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.596 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:25.596 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.925009+0000", 2024-04-26T03:39:25.596 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00045688899999999998 2024-04-26T03:39:25.596 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.596 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.597 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:25.597 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.925717+0000", 2024-04-26T03:39:25.597 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000708339 2024-04-26T03:39:25.597 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.597 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.597 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:25.597 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.925764+0000", 2024-04-26T03:39:25.597 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.7057e-05 2024-04-26T03:39:25.597 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:25.597 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:25.597 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:25.597 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.597 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.598 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:482 3.2s0 3:450a1cc5:::smithi09235267-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:25.598 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:05.914446+0000", 2024-04-26T03:39:25.598 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 19.578242656, 2024-04-26T03:39:25.598 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.01384406, 2024-04-26T03:39:25.598 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:25.598 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:25.598 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:25.598 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:25.598 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:25.598 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:25.598 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 482 2024-04-26T03:39:25.598 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.598 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:25.598 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.599 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:25.599 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.914446+0000", 2024-04-26T03:39:25.599 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:25.599 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.599 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.599 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:25.599 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.914446+0000", 2024-04-26T03:39:25.599 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:25.599 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.599 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.599 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:25.599 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.914447+0000", 2024-04-26T03:39:25.599 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.1600000000000001e-07 2024-04-26T03:39:25.599 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.599 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.600 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:25.600 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.914449+0000", 2024-04-26T03:39:25.600 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4229999999999999e-06 2024-04-26T03:39:25.600 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.600 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.600 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:25.600 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.914458+0000", 2024-04-26T03:39:25.600 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.9299999999999992e-06 2024-04-26T03:39:25.600 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.600 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.600 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:25.600 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.914465+0000", 2024-04-26T03:39:25.600 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.6599999999999998e-06 2024-04-26T03:39:25.601 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.601 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.601 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:25.601 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.924755+0000", 2024-04-26T03:39:25.601 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.010289777 2024-04-26T03:39:25.601 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.601 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.601 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:25.601 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.924801+0000", 2024-04-26T03:39:25.601 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6109000000000002e-05 2024-04-26T03:39:25.601 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.601 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.601 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:25.602 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.926051+0000", 2024-04-26T03:39:25.602 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001250393 2024-04-26T03:39:25.602 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.602 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.602 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:25.602 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.926140+0000", 2024-04-26T03:39:25.602 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.9227999999999993e-05 2024-04-26T03:39:25.602 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.602 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.602 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:25.602 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.928290+0000", 2024-04-26T03:39:25.602 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0021500239999999999 2024-04-26T03:39:25.602 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:25.603 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:25.603 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:25.603 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.603 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.603 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.fs3 65/62 ECSubWrite(tid=178, reqid=client.4556.0:782, at_version=65'60, trim_to=0'0, roll_forward_to=64'57))", 2024-04-26T03:39:25.603 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:20.829001+0000", 2024-04-26T03:39:25.603 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 4.6636876889999996, 2024-04-26T03:39:25.603 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.012433502000000001, 2024-04-26T03:39:25.603 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:25.603 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:25.603 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:25.603 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:25.603 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.604 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:25.604 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.829001+0000", 2024-04-26T03:39:25.604 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:25.604 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.604 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.604 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:25.604 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.829001+0000", 2024-04-26T03:39:25.604 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:25.604 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.604 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.604 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:25.604 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.829002+0000", 2024-04-26T03:39:25.604 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.6000000000000003e-07 2024-04-26T03:39:25.604 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.605 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.605 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:25.605 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.829342+0000", 2024-04-26T03:39:25.605 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00033970400000000003 2024-04-26T03:39:25.605 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.605 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.605 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:25.605 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.829356+0000", 2024-04-26T03:39:25.605 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.434e-05 2024-04-26T03:39:25.605 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.605 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.605 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:25.606 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.829365+0000", 2024-04-26T03:39:25.606 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.0149999999999992e-06 2024-04-26T03:39:25.606 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.606 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.606 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:25.606 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.829431+0000", 2024-04-26T03:39:25.606 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.6080000000000004e-05 2024-04-26T03:39:25.606 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.606 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.606 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:25.606 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.829442+0000", 2024-04-26T03:39:25.606 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1439e-05 2024-04-26T03:39:25.607 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.607 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.607 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:25.607 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.841390+0000", 2024-04-26T03:39:25.607 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.01194714 2024-04-26T03:39:25.607 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:25.607 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:25.607 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:25.607 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.841435+0000", 2024-04-26T03:39:25.607 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.5123999999999997e-05 2024-04-26T03:39:25.607 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:25.607 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:25.608 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:25.608 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:25.608 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:25.608 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:39:25.610 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-26T03:39:25.610 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-04-26T03:39:25.608+0000 7fee562fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31759) UID: 0 2024-04-26T03:39:25.673 INFO:teuthology.orchestra.run.smithi028.stdout:osd.15: {} 2024-04-26T03:39:25.674 INFO:teuthology.orchestra.run.smithi028.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T03:39:25.690 INFO:tasks.rados.rados.0.smithi092.stdout:312: done (5 left) 2024-04-26T03:39:25.691 INFO:tasks.rados.rados.0.smithi092.stdout:313: done (4 left) 2024-04-26T03:39:25.691 INFO:tasks.rados.rados.0.smithi092.stdout:314: done (3 left) 2024-04-26T03:39:25.691 INFO:tasks.rados.rados.0.smithi092.stdout:315: done (2 left) 2024-04-26T03:39:25.691 INFO:tasks.rados.rados.0.smithi092.stdout:317: done (1 left) 2024-04-26T03:39:25.691 INFO:tasks.rados.rados.0.smithi092.stdout:318: done (0 left) 2024-04-26T03:39:25.691 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 17 current snap is 21 2024-04-26T03:39:25.691 INFO:tasks.rados.rados.0.smithi092.stdout:319: seq_num 147 ranges {1409024=65536} 2024-04-26T03:39:25.699 INFO:tasks.rados.rados.0.smithi092.stdout:319: writing smithi09235267-17 from 1409024 to 1474560 tid 1 2024-04-26T03:39:25.699 INFO:tasks.rados.rados.0.smithi092.stdout:320: rollback oid 22 current snap is 21 2024-04-26T03:39:25.699 INFO:tasks.rados.rados.0.smithi092.stdout:rollback oid 22 to 18 2024-04-26T03:39:25.699 INFO:tasks.rados.rados.0.smithi092.stdout:321: snap_remove snap 20 2024-04-26T03:39:25.710 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:39:25.709+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:39:25.714 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-26T03:39:25.723 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:25.723 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-26T03:39:25.723 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-26T03:39:25.723 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:25.732 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-26T03:39:25.811 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-04-26T03:39:25.810+0000 7fe9223c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31954) UID: 0 2024-04-26T03:39:25.833 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:25.833 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-26T03:39:25.833 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-26T03:39:25.833 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-26T03:39:25.833 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:25.842 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-26T03:39:25.912 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:39:25.910+0000 7fba0d2ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31961) UID: 0 2024-04-26T03:39:25.947 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:25.948 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-26T03:39:25.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-26T03:39:25.948 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-26T03:39:25.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.948 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:20 3.as0 3:5f4e6bd0:::smithi09235267-7:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:25.948 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.485002+0000", 2024-04-26T03:39:25.948 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.460069169, 2024-04-26T03:39:25.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017259429, 2024-04-26T03:39:25.948 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:25.948 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:25.948 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:25.948 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:25.948 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:25.948 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:25.949 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 20 2024-04-26T03:39:25.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.949 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:25.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:25.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485002+0000", 2024-04-26T03:39:25.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:25.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:25.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485002+0000", 2024-04-26T03:39:25.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:25.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:25.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485003+0000", 2024-04-26T03:39:25.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6400000000000002e-07 2024-04-26T03:39:25.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:25.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485007+0000", 2024-04-26T03:39:25.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4390000000000002e-06 2024-04-26T03:39:25.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:25.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485015+0000", 2024-04-26T03:39:25.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9389999999999997e-06 2024-04-26T03:39:25.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:25.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485243+0000", 2024-04-26T03:39:25.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000227345 2024-04-26T03:39:25.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:25.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.487060+0000", 2024-04-26T03:39:25.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001817117 2024-04-26T03:39:25.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:25.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.487148+0000", 2024-04-26T03:39:25.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8712e-05 2024-04-26T03:39:25.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:25.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.487611+0000", 2024-04-26T03:39:25.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000462223 2024-04-26T03:39:25.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:25.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.491670+0000", 2024-04-26T03:39:25.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004058842 2024-04-26T03:39:25.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:25.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.502181+0000", 2024-04-26T03:39:25.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010511360000000001 2024-04-26T03:39:25.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:25.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.502262+0000", 2024-04-26T03:39:25.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0888000000000002e-05 2024-04-26T03:39:25.953 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:25.953 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:25.953 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:25.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.953 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:21 3.as0 3:5f4e6bd0:::smithi09235267-7:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:25.953 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.485256+0000", 2024-04-26T03:39:25.953 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.459815532, 2024-04-26T03:39:25.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026302736, 2024-04-26T03:39:25.954 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:25.954 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:25.954 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:25.954 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:25.954 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:25.954 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:25.954 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 21 2024-04-26T03:39:25.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.954 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:25.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:25.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485256+0000", 2024-04-26T03:39:25.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:25.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:25.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485256+0000", 2024-04-26T03:39:25.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:25.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:25.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485257+0000", 2024-04-26T03:39:25.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4000000000000001e-07 2024-04-26T03:39:25.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:25.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485261+0000", 2024-04-26T03:39:25.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2910000000000001e-06 2024-04-26T03:39:25.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:25.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485269+0000", 2024-04-26T03:39:25.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1780000000000003e-06 2024-04-26T03:39:25.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:25.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485275+0000", 2024-04-26T03:39:25.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2419999999999997e-06 2024-04-26T03:39:25.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:25.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.487806+0000", 2024-04-26T03:39:25.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025310369999999999 2024-04-26T03:39:25.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:25.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.488019+0000", 2024-04-26T03:39:25.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021297800000000001 2024-04-26T03:39:25.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:25.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.502612+0000", 2024-04-26T03:39:25.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014592327 2024-04-26T03:39:25.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:25.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.502711+0000", 2024-04-26T03:39:25.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9432999999999995e-05 2024-04-26T03:39:25.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:25.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.511559+0000", 2024-04-26T03:39:25.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0088476100000000005 2024-04-26T03:39:25.958 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:25.958 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:25.958 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:25.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.959 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:75 3.as0 3:50cd80b4:::smithi09235267-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:25.959 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.524874+0000", 2024-04-26T03:39:25.959 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.420197690000002, 2024-04-26T03:39:25.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017219323000000002, 2024-04-26T03:39:25.959 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:25.959 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:25.959 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:25.959 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:25.959 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:25.959 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:25.959 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 75 2024-04-26T03:39:25.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.961 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:25.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:25.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.524874+0000", 2024-04-26T03:39:25.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:25.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:25.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.524874+0000", 2024-04-26T03:39:25.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:25.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:25.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.524874+0000", 2024-04-26T03:39:25.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9200000000000001e-07 2024-04-26T03:39:25.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:25.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.524877+0000", 2024-04-26T03:39:25.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6079999999999998e-06 2024-04-26T03:39:25.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:25.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.524883+0000", 2024-04-26T03:39:25.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0039999999999996e-06 2024-04-26T03:39:25.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:25.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.524888+0000", 2024-04-26T03:39:25.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3070000000000002e-06 2024-04-26T03:39:25.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:25.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.530308+0000", 2024-04-26T03:39:25.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054201850000000001 2024-04-26T03:39:25.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:25.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.530412+0000", 2024-04-26T03:39:25.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010306699999999999 2024-04-26T03:39:25.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:25.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.534668+0000", 2024-04-26T03:39:25.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042560949999999997 2024-04-26T03:39:25.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:25.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.534740+0000", 2024-04-26T03:39:25.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2082000000000005e-05 2024-04-26T03:39:25.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:25.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.542093+0000", 2024-04-26T03:39:25.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0073534830000000001 2024-04-26T03:39:25.965 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:25.965 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:25.965 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:25.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.965 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:105 3.as0 3:527f9dfd:::smithi09235267-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:25.965 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.562626+0000", 2024-04-26T03:39:25.965 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.382445851, 2024-04-26T03:39:25.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.023769061000000001, 2024-04-26T03:39:25.966 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:25.966 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:25.966 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:25.966 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:25.966 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:25.966 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:25.966 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 105 2024-04-26T03:39:25.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.966 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:25.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:25.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.562626+0000", 2024-04-26T03:39:25.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:25.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:25.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.562626+0000", 2024-04-26T03:39:25.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:25.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:25.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.562626+0000", 2024-04-26T03:39:25.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7800000000000002e-07 2024-04-26T03:39:25.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:25.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.562629+0000", 2024-04-26T03:39:25.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7779999999999999e-06 2024-04-26T03:39:25.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:25.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.562636+0000", 2024-04-26T03:39:25.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3520000000000001e-06 2024-04-26T03:39:25.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:25.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.562641+0000", 2024-04-26T03:39:25.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4789999999999998e-06 2024-04-26T03:39:25.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:25.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.568071+0000", 2024-04-26T03:39:25.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054305719999999998 2024-04-26T03:39:25.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:25.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.568158+0000", 2024-04-26T03:39:25.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6726999999999998e-05 2024-04-26T03:39:25.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:25.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.575494+0000", 2024-04-26T03:39:25.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0073362180000000003 2024-04-26T03:39:25.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:25.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.575571+0000", 2024-04-26T03:39:25.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7134999999999999e-05 2024-04-26T03:39:25.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:25.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.586395+0000", 2024-04-26T03:39:25.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010823322 2024-04-26T03:39:25.970 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:25.970 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:25.970 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:25.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.970 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:117 3.as0 3:58923ed4:::smithi09235267-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:25.970 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.578829+0000", 2024-04-26T03:39:25.970 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.366242733, 2024-04-26T03:39:25.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017648347000000002, 2024-04-26T03:39:25.971 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:25.971 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:25.971 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:25.971 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:25.971 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:25.971 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:25.971 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 117 2024-04-26T03:39:25.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.971 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:25.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:25.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.578829+0000", 2024-04-26T03:39:25.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:25.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:25.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.578829+0000", 2024-04-26T03:39:25.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:25.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:25.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.578829+0000", 2024-04-26T03:39:25.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4900000000000001e-07 2024-04-26T03:39:25.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:25.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.578831+0000", 2024-04-26T03:39:25.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0700000000000001e-06 2024-04-26T03:39:25.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:25.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.578838+0000", 2024-04-26T03:39:25.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2689999999999996e-06 2024-04-26T03:39:25.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:25.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.578843+0000", 2024-04-26T03:39:25.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5300000000000004e-06 2024-04-26T03:39:25.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:25.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.586870+0000", 2024-04-26T03:39:25.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0080264919999999997 2024-04-26T03:39:25.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:25.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.586955+0000", 2024-04-26T03:39:25.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5753000000000004e-05 2024-04-26T03:39:25.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:25.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.589337+0000", 2024-04-26T03:39:25.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023813010000000002 2024-04-26T03:39:25.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:25.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.589419+0000", 2024-04-26T03:39:25.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2078999999999998e-05 2024-04-26T03:39:25.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:25.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.596477+0000", 2024-04-26T03:39:25.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0070584039999999999 2024-04-26T03:39:25.975 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:25.975 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:25.975 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:25.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.975 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:150 3.as0 3:5930de0f:::smithi09235267-50:head [append 0~458752,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:25.976 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.634461+0000", 2024-04-26T03:39:25.976 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.310610244000003, 2024-04-26T03:39:25.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017493549000000001, 2024-04-26T03:39:25.976 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:25.976 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:25.976 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:25.976 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:25.976 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:25.976 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:25.976 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 150 2024-04-26T03:39:25.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.976 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:25.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:25.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.634461+0000", 2024-04-26T03:39:25.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:25.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:25.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.634461+0000", 2024-04-26T03:39:25.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:25.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:25.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.634463+0000", 2024-04-26T03:39:25.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.618e-06 2024-04-26T03:39:25.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:25.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.634611+0000", 2024-04-26T03:39:25.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000148214 2024-04-26T03:39:25.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:25.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.634628+0000", 2024-04-26T03:39:25.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7105999999999999e-05 2024-04-26T03:39:25.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:25.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.634636+0000", 2024-04-26T03:39:25.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7829999999999999e-06 2024-04-26T03:39:25.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:25.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.634700+0000", 2024-04-26T03:39:25.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4286000000000001e-05 2024-04-26T03:39:25.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:25.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.634839+0000", 2024-04-26T03:39:25.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013907200000000001 2024-04-26T03:39:25.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:25.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.635607+0000", 2024-04-26T03:39:25.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00076726300000000002 2024-04-26T03:39:25.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:25.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.647963+0000", 2024-04-26T03:39:25.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012356525 2024-04-26T03:39:25.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:25.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.651885+0000", 2024-04-26T03:39:25.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039220180000000002 2024-04-26T03:39:25.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:25.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.651955+0000", 2024-04-26T03:39:25.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9664000000000003e-05 2024-04-26T03:39:25.981 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:25.981 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:25.981 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:25.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.981 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:152 3.as0 3:5930de0f:::smithi09235267-50:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:25.981 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.637409+0000", 2024-04-26T03:39:25.981 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.307662958000002, 2024-04-26T03:39:25.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016318039999999999, 2024-04-26T03:39:25.981 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:25.981 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:25.982 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:25.982 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:25.982 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:25.982 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:25.982 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 152 2024-04-26T03:39:25.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.982 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:25.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:25.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637409+0000", 2024-04-26T03:39:25.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:25.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:25.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637409+0000", 2024-04-26T03:39:25.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:25.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:25.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637409+0000", 2024-04-26T03:39:25.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5499999999999998e-07 2024-04-26T03:39:25.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:25.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637412+0000", 2024-04-26T03:39:25.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2959999999999999e-06 2024-04-26T03:39:25.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:25.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637419+0000", 2024-04-26T03:39:25.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4670000000000001e-06 2024-04-26T03:39:25.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:25.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637424+0000", 2024-04-26T03:39:25.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8720000000000001e-06 2024-04-26T03:39:25.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:25.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.643035+0000", 2024-04-26T03:39:25.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056115899999999996 2024-04-26T03:39:25.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:25.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.643126+0000", 2024-04-26T03:39:25.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0525999999999994e-05 2024-04-26T03:39:25.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:25.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.643545+0000", 2024-04-26T03:39:25.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00041870200000000002 2024-04-26T03:39:25.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:25.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.647965+0000", 2024-04-26T03:39:25.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0044201730000000003 2024-04-26T03:39:25.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:25.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.653647+0000", 2024-04-26T03:39:25.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056822670000000004 2024-04-26T03:39:25.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:25.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.653727+0000", 2024-04-26T03:39:25.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9592000000000006e-05 2024-04-26T03:39:25.986 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:25.986 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:25.986 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:25.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.987 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:153 3.as0 3:5930de0f:::smithi09235267-50:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:25.987 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.637431+0000", 2024-04-26T03:39:25.987 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.307640683000002, 2024-04-26T03:39:25.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019190048000000001, 2024-04-26T03:39:25.987 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:25.987 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:25.987 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:25.987 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:25.987 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:25.987 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:25.987 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 153 2024-04-26T03:39:25.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.988 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:25.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:25.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637431+0000", 2024-04-26T03:39:25.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:25.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:25.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637431+0000", 2024-04-26T03:39:25.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:25.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:25.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637431+0000", 2024-04-26T03:39:25.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2024-04-26T03:39:25.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:25.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637434+0000", 2024-04-26T03:39:25.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7690000000000001e-06 2024-04-26T03:39:25.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:25.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637439+0000", 2024-04-26T03:39:25.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4190000000000001e-06 2024-04-26T03:39:25.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:25.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637444+0000", 2024-04-26T03:39:25.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6800000000000001e-06 2024-04-26T03:39:25.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:25.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.643745+0000", 2024-04-26T03:39:25.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0063006570000000003 2024-04-26T03:39:25.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:25.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.643818+0000", 2024-04-26T03:39:25.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3558000000000003e-05 2024-04-26T03:39:25.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:25.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.654106+0000", 2024-04-26T03:39:25.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010287436 2024-04-26T03:39:25.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:25.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.654189+0000", 2024-04-26T03:39:25.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2719999999999994e-05 2024-04-26T03:39:25.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:25.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.656621+0000", 2024-04-26T03:39:25.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024323690000000002 2024-04-26T03:39:25.992 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:25.992 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:25.992 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:25.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.992 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:310 3.as0 3:50cd80b4:::smithi09235267-25:head [copy-from ver 3] snapc 6=[6,5] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-26T03:39:25.992 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:59.536617+0000", 2024-04-26T03:39:25.992 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.408454090999999, 2024-04-26T03:39:25.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022665805000000001, 2024-04-26T03:39:25.992 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:25.992 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:25.992 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:25.992 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:25.992 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:25.993 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:25.993 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 310 2024-04-26T03:39:25.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.993 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:25.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:25.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536617+0000", 2024-04-26T03:39:25.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:25.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:25.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536617+0000", 2024-04-26T03:39:25.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:25.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:25.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536620+0000", 2024-04-26T03:39:25.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0540000000000002e-06 2024-04-26T03:39:25.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:25.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536636+0000", 2024-04-26T03:39:25.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6963e-05 2024-04-26T03:39:25.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:25.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536710+0000", 2024-04-26T03:39:25.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3711e-05 2024-04-26T03:39:25.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:25.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536719+0000", 2024-04-26T03:39:25.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9889999999999998e-06 2024-04-26T03:39:25.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:25.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536789+0000", 2024-04-26T03:39:25.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9884999999999999e-05 2024-04-26T03:39:25.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-04-26T03:39:25.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536797+0000", 2024-04-26T03:39:25.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5909999999999998e-06 2024-04-26T03:39:25.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:25.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.538587+0000", 2024-04-26T03:39:25.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017899249999999999 2024-04-26T03:39:25.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:25.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.538669+0000", 2024-04-26T03:39:25.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2045000000000005e-05 2024-04-26T03:39:25.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:25.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.549606+0000", 2024-04-26T03:39:25.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010937448000000001 2024-04-26T03:39:25.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:25.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.558484+0000", 2024-04-26T03:39:25.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00887808 2024-04-26T03:39:25.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:25.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.559195+0000", 2024-04-26T03:39:25.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00071067799999999996 2024-04-26T03:39:25.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:25.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.559283+0000", 2024-04-26T03:39:25.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8436000000000005e-05 2024-04-26T03:39:25.998 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:25.998 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:25.998 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:25.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.998 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:311 3.as0 3:50cd80b4:::smithi09235267-25:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e43)", 2024-04-26T03:39:25.998 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:59.536742+0000", 2024-04-26T03:39:25.998 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.408329199000001, 2024-04-26T03:39:25.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021335742000000001, 2024-04-26T03:39:25.998 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:25.998 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:25.998 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:25.999 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:25.999 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:25.999 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:25.999 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 311 2024-04-26T03:39:25.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.999 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:25.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:25.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536742+0000", 2024-04-26T03:39:25.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:25.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:25.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:25.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:25.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536742+0000", 2024-04-26T03:39:25.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:26.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536743+0000", 2024-04-26T03:39:26.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5000000000000003e-07 2024-04-26T03:39:26.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:26.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536746+0000", 2024-04-26T03:39:26.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8720000000000002e-06 2024-04-26T03:39:26.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:26.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536771+0000", 2024-04-26T03:39:26.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4949000000000001e-05 2024-04-26T03:39:26.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:26.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536776+0000", 2024-04-26T03:39:26.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1089999999999997e-06 2024-04-26T03:39:26.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536853+0000", 2024-04-26T03:39:26.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7348000000000006e-05 2024-04-26T03:39:26.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-04-26T03:39:26.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536859+0000", 2024-04-26T03:39:26.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.784e-06 2024-04-26T03:39:26.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.538806+0000", 2024-04-26T03:39:26.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019471029999999999 2024-04-26T03:39:26.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for blocked object", 2024-04-26T03:39:26.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.538849+0000", 2024-04-26T03:39:26.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2901000000000001e-05 2024-04-26T03:39:26.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.557853+0000", 2024-04-26T03:39:26.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019003955999999999 2024-04-26T03:39:26.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:26.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.557954+0000", 2024-04-26T03:39:26.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010153399999999999 2024-04-26T03:39:26.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:26.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.558078+0000", 2024-04-26T03:39:26.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012363599999999999 2024-04-26T03:39:26.003 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.003 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:26.004 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.004 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.2s2 49/44 ECSubWrite(tid=34, reqid=client.4556.0:434, at_version=49'23, trim_to=0'0, roll_forward_to=48'19))", 2024-04-26T03:39:26.004 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:04.712444+0000", 2024-04-26T03:39:26.004 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.232627275999999, 2024-04-26T03:39:26.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016297222, 2024-04-26T03:39:26.004 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:26.004 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:26.004 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:26.004 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:26.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:26.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.712444+0000", 2024-04-26T03:39:26.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:26.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.712444+0000", 2024-04-26T03:39:26.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:26.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.712445+0000", 2024-04-26T03:39:26.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4399999999999999e-07 2024-04-26T03:39:26.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:26.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.712821+0000", 2024-04-26T03:39:26.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000376194 2024-04-26T03:39:26.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:26.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.712838+0000", 2024-04-26T03:39:26.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7076999999999999e-05 2024-04-26T03:39:26.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:26.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.712854+0000", 2024-04-26T03:39:26.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5852999999999999e-05 2024-04-26T03:39:26.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.712940+0000", 2024-04-26T03:39:26.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6061000000000002e-05 2024-04-26T03:39:26.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:26.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.712952+0000", 2024-04-26T03:39:26.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1810999999999999e-05 2024-04-26T03:39:26.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:26.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.728691+0000", 2024-04-26T03:39:26.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015739306000000002 2024-04-26T03:39:26.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:26.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:04.728741+0000", 2024-04-26T03:39:26.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0076e-05 2024-04-26T03:39:26.008 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.008 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:26.008 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.008 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:454 3.as0 3:5930de0f:::smithi09235267-50:head [append 0~737280] snapc 8=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:26.008 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:05.710804+0000", 2024-04-26T03:39:26.008 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.234267932000002, 2024-04-26T03:39:26.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017146651999999998, 2024-04-26T03:39:26.008 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:26.008 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:26.008 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:26.009 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:26.009 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:26.009 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:26.009 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 454 2024-04-26T03:39:26.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.009 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:26.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:26.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.710804+0000", 2024-04-26T03:39:26.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:26.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.710804+0000", 2024-04-26T03:39:26.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:26.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.710805+0000", 2024-04-26T03:39:26.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8509999999999999e-06 2024-04-26T03:39:26.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:26.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.711242+0000", 2024-04-26T03:39:26.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000436091 2024-04-26T03:39:26.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:26.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.711259+0000", 2024-04-26T03:39:26.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7533e-05 2024-04-26T03:39:26.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:26.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.711269+0000", 2024-04-26T03:39:26.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6760000000000002e-06 2024-04-26T03:39:26.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.711382+0000", 2024-04-26T03:39:26.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011296 2024-04-26T03:39:26.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:26.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.711483+0000", 2024-04-26T03:39:26.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101048 2024-04-26T03:39:26.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:26.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.713492+0000", 2024-04-26T03:39:26.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020094900000000001 2024-04-26T03:39:26.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:26.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.727613+0000", 2024-04-26T03:39:26.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014120819 2024-04-26T03:39:26.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:26.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.727890+0000", 2024-04-26T03:39:26.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027681299999999999 2024-04-26T03:39:26.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:26.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.727950+0000", 2024-04-26T03:39:26.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0371000000000002e-05 2024-04-26T03:39:26.013 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.013 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:26.013 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.013 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:457 3.as0 3:5930de0f:::smithi09235267-50:head [setxattr _header (56) in=7b] snapc 8=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:26.013 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:05.716766+0000", 2024-04-26T03:39:26.014 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.228305734999999, 2024-04-26T03:39:26.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017528595000000001, 2024-04-26T03:39:26.014 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:26.014 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:26.014 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:26.014 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:26.014 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:26.014 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:26.014 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 457 2024-04-26T03:39:26.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.014 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:26.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:26.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716766+0000", 2024-04-26T03:39:26.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:26.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716766+0000", 2024-04-26T03:39:26.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:26.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716766+0000", 2024-04-26T03:39:26.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6300000000000005e-07 2024-04-26T03:39:26.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:26.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716770+0000", 2024-04-26T03:39:26.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.377e-06 2024-04-26T03:39:26.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:26.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716776+0000", 2024-04-26T03:39:26.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6850000000000004e-06 2024-04-26T03:39:26.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:26.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716782+0000", 2024-04-26T03:39:26.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4779999999999999e-06 2024-04-26T03:39:26.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.730442+0000", 2024-04-26T03:39:26.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013660284 2024-04-26T03:39:26.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:26.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.730571+0000", 2024-04-26T03:39:26.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012850700000000001 2024-04-26T03:39:26.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:26.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.731245+0000", 2024-04-26T03:39:26.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00067431000000000001 2024-04-26T03:39:26.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:26.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.732606+0000", 2024-04-26T03:39:26.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013605850000000001 2024-04-26T03:39:26.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:26.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.734232+0000", 2024-04-26T03:39:26.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0016260809999999999 2024-04-26T03:39:26.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:26.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.734294+0000", 2024-04-26T03:39:26.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2725000000000007e-05 2024-04-26T03:39:26.019 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.019 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:26.019 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.019 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:458 3.as0 3:5930de0f:::smithi09235267-50:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:26.019 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:05.716790+0000", 2024-04-26T03:39:26.019 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.228281980999999, 2024-04-26T03:39:26.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019815810999999999, 2024-04-26T03:39:26.019 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:26.020 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:26.020 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:26.020 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:26.020 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:26.020 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:26.020 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 458 2024-04-26T03:39:26.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.020 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:26.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:26.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716790+0000", 2024-04-26T03:39:26.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:26.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716790+0000", 2024-04-26T03:39:26.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:26.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716790+0000", 2024-04-26T03:39:26.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-04-26T03:39:26.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:26.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716792+0000", 2024-04-26T03:39:26.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3379999999999999e-06 2024-04-26T03:39:26.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:26.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716798+0000", 2024-04-26T03:39:26.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.066e-06 2024-04-26T03:39:26.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:26.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716803+0000", 2024-04-26T03:39:26.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.673e-06 2024-04-26T03:39:26.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.732363+0000", 2024-04-26T03:39:26.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015560218000000001 2024-04-26T03:39:26.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:26.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.732421+0000", 2024-04-26T03:39:26.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7649000000000003e-05 2024-04-26T03:39:26.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.734770+0000", 2024-04-26T03:39:26.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002348761 2024-04-26T03:39:26.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:26.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.734838+0000", 2024-04-26T03:39:26.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8471000000000002e-05 2024-04-26T03:39:26.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:26.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.736605+0000", 2024-04-26T03:39:26.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017672250000000001 2024-04-26T03:39:26.024 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.025 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:26.025 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.025 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:760 3.as0 3:58923ed4:::smithi09235267-39:head [append 0~704512] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:26.025 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:20.745444+0000", 2024-04-26T03:39:26.025 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.1996275020000002, 2024-04-26T03:39:26.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021214666, 2024-04-26T03:39:26.025 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:26.025 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:26.025 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:26.025 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:26.025 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:26.026 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:26.026 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 760 2024-04-26T03:39:26.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.026 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:26.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:26.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.745444+0000", 2024-04-26T03:39:26.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:26.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.745444+0000", 2024-04-26T03:39:26.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:26.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.745446+0000", 2024-04-26T03:39:26.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.858e-06 2024-04-26T03:39:26.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:26.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.745741+0000", 2024-04-26T03:39:26.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029482400000000001 2024-04-26T03:39:26.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:26.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.745774+0000", 2024-04-26T03:39:26.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3124000000000003e-05 2024-04-26T03:39:26.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:26.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.745798+0000", 2024-04-26T03:39:26.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4528999999999998e-05 2024-04-26T03:39:26.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.745885+0000", 2024-04-26T03:39:26.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6878000000000004e-05 2024-04-26T03:39:26.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-04-26T03:39:26.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.745910+0000", 2024-04-26T03:39:26.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4935000000000001e-05 2024-04-26T03:39:26.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.748635+0000", 2024-04-26T03:39:26.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0027248569999999998 2024-04-26T03:39:26.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:26.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.748702+0000", 2024-04-26T03:39:26.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7076000000000006e-05 2024-04-26T03:39:26.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:26.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.749617+0000", 2024-04-26T03:39:26.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00091456800000000004 2024-04-26T03:39:26.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:26.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.765879+0000", 2024-04-26T03:39:26.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016262566999999999 2024-04-26T03:39:26.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:26.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.766500+0000", 2024-04-26T03:39:26.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00062033100000000001 2024-04-26T03:39:26.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:26.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.766659+0000", 2024-04-26T03:39:26.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015911899999999999 2024-04-26T03:39:26.031 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.031 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:26.031 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.031 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:761 3.as0 3:58923ed4:::smithi09235267-39:head [append 0~475136] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:26.031 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:20.748282+0000", 2024-04-26T03:39:26.032 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.1967899109999998, 2024-04-26T03:39:26.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019043616999999999, 2024-04-26T03:39:26.032 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:26.032 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:26.032 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:26.032 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:26.032 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:26.032 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:26.032 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 761 2024-04-26T03:39:26.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.032 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:26.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:26.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.748282+0000", 2024-04-26T03:39:26.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:26.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.748282+0000", 2024-04-26T03:39:26.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:26.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.748283+0000", 2024-04-26T03:39:26.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.133e-06 2024-04-26T03:39:26.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:26.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.748542+0000", 2024-04-26T03:39:26.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00025954400000000001 2024-04-26T03:39:26.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:26.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.748555+0000", 2024-04-26T03:39:26.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2483000000000001e-05 2024-04-26T03:39:26.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:26.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.748564+0000", 2024-04-26T03:39:26.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8109999999999992e-06 2024-04-26T03:39:26.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.753862+0000", 2024-04-26T03:39:26.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052979580000000002 2024-04-26T03:39:26.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:26.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.753977+0000", 2024-04-26T03:39:26.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115072 2024-04-26T03:39:26.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:26.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.754660+0000", 2024-04-26T03:39:26.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00068356000000000005 2024-04-26T03:39:26.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:26.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.765984+0000", 2024-04-26T03:39:26.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011324021 2024-04-26T03:39:26.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:26.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.767207+0000", 2024-04-26T03:39:26.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0012224340000000001 2024-04-26T03:39:26.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:26.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.767325+0000", 2024-04-26T03:39:26.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000118601 2024-04-26T03:39:26.036 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.037 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:26.037 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.037 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:763 3.as0 3:58923ed4:::smithi09235267-39:head [append 0~507904] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:26.037 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:20.754960+0000", 2024-04-26T03:39:26.037 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.1901110680000002, 2024-04-26T03:39:26.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020013485000000001, 2024-04-26T03:39:26.037 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:26.037 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:26.037 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:26.037 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:26.037 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:26.037 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:26.037 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 763 2024-04-26T03:39:26.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.038 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:26.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:26.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.754960+0000", 2024-04-26T03:39:26.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:26.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.754960+0000", 2024-04-26T03:39:26.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:26.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.754962+0000", 2024-04-26T03:39:26.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.66e-06 2024-04-26T03:39:26.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:26.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.755204+0000", 2024-04-26T03:39:26.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024158200000000001 2024-04-26T03:39:26.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:26.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.755216+0000", 2024-04-26T03:39:26.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1888999999999999e-05 2024-04-26T03:39:26.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:26.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.755223+0000", 2024-04-26T03:39:26.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6529999999999994e-06 2024-04-26T03:39:26.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.767902+0000", 2024-04-26T03:39:26.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012678833 2024-04-26T03:39:26.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:26.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.768010+0000", 2024-04-26T03:39:26.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107639 2024-04-26T03:39:26.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:26.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.768777+0000", 2024-04-26T03:39:26.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00076683800000000005 2024-04-26T03:39:26.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:26.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.774647+0000", 2024-04-26T03:39:26.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058705609999999998 2024-04-26T03:39:26.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:26.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.774790+0000", 2024-04-26T03:39:26.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000143017 2024-04-26T03:39:26.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:26.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.774974+0000", 2024-04-26T03:39:26.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018381300000000001 2024-04-26T03:39:26.042 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.042 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:26.042 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.042 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:764 3.as0 3:58923ed4:::smithi09235267-39:head [setxattr _header (57) in=7b] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:26.043 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:20.755241+0000", 2024-04-26T03:39:26.043 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.189830808, 2024-04-26T03:39:26.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020359479, 2024-04-26T03:39:26.043 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:26.043 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:26.043 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:26.043 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:26.043 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:26.043 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:26.043 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 764 2024-04-26T03:39:26.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.043 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:26.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:26.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.755241+0000", 2024-04-26T03:39:26.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:26.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.755241+0000", 2024-04-26T03:39:26.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:26.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.755241+0000", 2024-04-26T03:39:26.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3399999999999999e-07 2024-04-26T03:39:26.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:26.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.755245+0000", 2024-04-26T03:39:26.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5470000000000001e-06 2024-04-26T03:39:26.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:26.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.755251+0000", 2024-04-26T03:39:26.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2389999999999999e-06 2024-04-26T03:39:26.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:26.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.755257+0000", 2024-04-26T03:39:26.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3680000000000001e-06 2024-04-26T03:39:26.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.773298+0000", 2024-04-26T03:39:26.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018041303000000002 2024-04-26T03:39:26.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:26.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.773404+0000", 2024-04-26T03:39:26.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010583500000000001 2024-04-26T03:39:26.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:26.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.773767+0000", 2024-04-26T03:39:26.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000363786 2024-04-26T03:39:26.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:26.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.774979+0000", 2024-04-26T03:39:26.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0012112360000000001 2024-04-26T03:39:26.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:26.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.775421+0000", 2024-04-26T03:39:26.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00044187799999999999 2024-04-26T03:39:26.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:26.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.775600+0000", 2024-04-26T03:39:26.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000179653 2024-04-26T03:39:26.047 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.048 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:26.048 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.048 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:765 3.as0 3:58923ed4:::smithi09235267-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:26.048 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:20.755268+0000", 2024-04-26T03:39:26.048 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.1898033589999999, 2024-04-26T03:39:26.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021987618, 2024-04-26T03:39:26.048 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:26.048 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:26.048 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:26.048 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:26.048 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:26.049 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:26.049 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 765 2024-04-26T03:39:26.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.049 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:26.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:26.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.755268+0000", 2024-04-26T03:39:26.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:26.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.755268+0000", 2024-04-26T03:39:26.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:26.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.755269+0000", 2024-04-26T03:39:26.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5400000000000002e-07 2024-04-26T03:39:26.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:26.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.755271+0000", 2024-04-26T03:39:26.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7070000000000001e-06 2024-04-26T03:39:26.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:26.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.755280+0000", 2024-04-26T03:39:26.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4519999999999997e-06 2024-04-26T03:39:26.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:26.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.755285+0000", 2024-04-26T03:39:26.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6459999999999998e-06 2024-04-26T03:39:26.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.774477+0000", 2024-04-26T03:39:26.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019191144 2024-04-26T03:39:26.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:26.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.774591+0000", 2024-04-26T03:39:26.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000114602 2024-04-26T03:39:26.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.776170+0000", 2024-04-26T03:39:26.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0015785549999999999 2024-04-26T03:39:26.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:26.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.776262+0000", 2024-04-26T03:39:26.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1823000000000005e-05 2024-04-26T03:39:26.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:26.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.777256+0000", 2024-04-26T03:39:26.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000994235 2024-04-26T03:39:26.053 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.053 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:26.053 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.053 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:790 3.as0 3:50cd80b4:::smithi09235267-25:head [append 0~786432] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:26.053 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:20.850132+0000", 2024-04-26T03:39:26.053 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.0949390729999999, 2024-04-26T03:39:26.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019444847000000001, 2024-04-26T03:39:26.053 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:26.053 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:26.053 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:26.054 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:26.054 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:26.054 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:26.054 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 790 2024-04-26T03:39:26.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.054 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:26.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:26.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.850132+0000", 2024-04-26T03:39:26.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:26.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.850132+0000", 2024-04-26T03:39:26.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:26.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.850134+0000", 2024-04-26T03:39:26.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7409999999999999e-06 2024-04-26T03:39:26.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:26.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.850299+0000", 2024-04-26T03:39:26.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000165119 2024-04-26T03:39:26.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:26.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.850316+0000", 2024-04-26T03:39:26.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6419999999999999e-05 2024-04-26T03:39:26.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:26.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.850324+0000", 2024-04-26T03:39:26.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6670000000000002e-06 2024-04-26T03:39:26.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.850375+0000", 2024-04-26T03:39:26.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0498999999999999e-05 2024-04-26T03:39:26.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:26.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.850439+0000", 2024-04-26T03:39:26.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4017999999999996e-05 2024-04-26T03:39:26.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:26.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.851224+0000", 2024-04-26T03:39:26.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00078535300000000003 2024-04-26T03:39:26.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:26.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.869292+0000", 2024-04-26T03:39:26.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018067295000000001 2024-04-26T03:39:26.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:26.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.869498+0000", 2024-04-26T03:39:26.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020635599999999999 2024-04-26T03:39:26.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:26.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.869577+0000", 2024-04-26T03:39:26.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9379e-05 2024-04-26T03:39:26.058 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.058 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:26.058 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.058 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.058 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:26.058 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:26.060 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-26T03:39:26.060 INFO:tasks.ceph.osd.14.smithi066.stderr:2024-04-26T03:39:26.059+0000 7f5395f08640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 31765) UID: 0 2024-04-26T03:39:26.161 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:39:26.159+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:39:26.162 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:26.162 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-26T03:39:26.162 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-26T03:39:26.162 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:26.171 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-26T03:39:26.262 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:39:26.260+0000 7fba0d2ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31961) UID: 0 2024-04-26T03:39:26.273 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:26.273 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-26T03:39:26.273 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-26T03:39:26.273 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-26T03:39:26.273 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:26.282 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-26T03:39:26.363 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:39:26.362+0000 7f60d3c4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31757) UID: 0 2024-04-26T03:39:26.385 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:26.385 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-26T03:39:26.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-26T03:39:26.386 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-26T03:39:26.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.386 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:45 3.ds0 3:b4337c88:::smithi09235267-15:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:26.386 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.498959+0000", 2024-04-26T03:39:26.386 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.883364640000003, 2024-04-26T03:39:26.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.028116775, 2024-04-26T03:39:26.386 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:26.386 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:26.386 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:26.386 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:26.386 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:26.386 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:26.386 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45 2024-04-26T03:39:26.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.387 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:26.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:26.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498959+0000", 2024-04-26T03:39:26.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:26.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498959+0000", 2024-04-26T03:39:26.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:26.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498959+0000", 2024-04-26T03:39:26.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.58e-07 2024-04-26T03:39:26.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:26.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498963+0000", 2024-04-26T03:39:26.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2449999999999998e-06 2024-04-26T03:39:26.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:26.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498970+0000", 2024-04-26T03:39:26.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5780000000000001e-06 2024-04-26T03:39:26.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:26.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498976+0000", 2024-04-26T03:39:26.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6400000000000002e-06 2024-04-26T03:39:26.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.504199+0000", 2024-04-26T03:39:26.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052229190000000003 2024-04-26T03:39:26.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:26.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.504274+0000", 2024-04-26T03:39:26.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5187999999999999e-05 2024-04-26T03:39:26.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.512941+0000", 2024-04-26T03:39:26.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0086671879999999993 2024-04-26T03:39:26.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:26.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.513001+0000", 2024-04-26T03:39:26.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0204000000000002e-05 2024-04-26T03:39:26.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:26.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.527076+0000", 2024-04-26T03:39:26.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014074355 2024-04-26T03:39:26.391 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.391 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:26.391 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.391 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:52 3.ds0 3:b895fe71:::smithi09235267-18:head [append 0~294912,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:26.391 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.505560+0000", 2024-04-26T03:39:26.391 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.876763640999997, 2024-04-26T03:39:26.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016117464000000001, 2024-04-26T03:39:26.392 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:26.392 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:26.392 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:26.392 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:26.392 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:26.392 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:26.392 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 52 2024-04-26T03:39:26.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.392 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:26.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:26.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505560+0000", 2024-04-26T03:39:26.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:26.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505560+0000", 2024-04-26T03:39:26.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:26.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505562+0000", 2024-04-26T03:39:26.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0729999999999999e-06 2024-04-26T03:39:26.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:26.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505753+0000", 2024-04-26T03:39:26.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019100500000000001 2024-04-26T03:39:26.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:26.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505768+0000", 2024-04-26T03:39:26.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5213e-05 2024-04-26T03:39:26.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:26.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505776+0000", 2024-04-26T03:39:26.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7279999999999995e-06 2024-04-26T03:39:26.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505830+0000", 2024-04-26T03:39:26.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3749999999999999e-05 2024-04-26T03:39:26.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:26.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505998+0000", 2024-04-26T03:39:26.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000168259 2024-04-26T03:39:26.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:26.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.506585+0000", 2024-04-26T03:39:26.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00058713299999999999 2024-04-26T03:39:26.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:26.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.511525+0000", 2024-04-26T03:39:26.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004939789 2024-04-26T03:39:26.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:26.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.521601+0000", 2024-04-26T03:39:26.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010075766 2024-04-26T03:39:26.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:26.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.521677+0000", 2024-04-26T03:39:26.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6748000000000004e-05 2024-04-26T03:39:26.397 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.397 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:26.397 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.397 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:53 3.ds0 3:b895fe71:::smithi09235267-18:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:26.397 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.505792+0000", 2024-04-26T03:39:26.397 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.876531202000002, 2024-04-26T03:39:26.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016119725000000001, 2024-04-26T03:39:26.397 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:26.398 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:26.398 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:26.398 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:26.398 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:26.398 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:26.398 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 53 2024-04-26T03:39:26.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.398 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:26.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:26.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505792+0000", 2024-04-26T03:39:26.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:26.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505792+0000", 2024-04-26T03:39:26.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:26.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505793+0000", 2024-04-26T03:39:26.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5400000000000001e-07 2024-04-26T03:39:26.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:26.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505806+0000", 2024-04-26T03:39:26.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2733e-05 2024-04-26T03:39:26.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:26.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505814+0000", 2024-04-26T03:39:26.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0260000000000007e-06 2024-04-26T03:39:26.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:26.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505820+0000", 2024-04-26T03:39:26.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8769999999999998e-06 2024-04-26T03:39:26.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.509659+0000", 2024-04-26T03:39:26.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0038395149999999999 2024-04-26T03:39:26.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:26.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.509764+0000", 2024-04-26T03:39:26.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105159 2024-04-26T03:39:26.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:26.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.510189+0000", 2024-04-26T03:39:26.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00042430899999999999 2024-04-26T03:39:26.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:26.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.511526+0000", 2024-04-26T03:39:26.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001337934 2024-04-26T03:39:26.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:26.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.521837+0000", 2024-04-26T03:39:26.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010310503 2024-04-26T03:39:26.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:26.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.521912+0000", 2024-04-26T03:39:26.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5014999999999999e-05 2024-04-26T03:39:26.402 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.402 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:26.403 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.403 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:54 3.ds0 3:b895fe71:::smithi09235267-18:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:26.403 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.505829+0000", 2024-04-26T03:39:26.403 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.876494360999999, 2024-04-26T03:39:26.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021406122, 2024-04-26T03:39:26.403 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:26.403 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:26.403 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:26.403 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:26.403 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:26.403 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:26.404 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 54 2024-04-26T03:39:26.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.404 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:26.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:26.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505829+0000", 2024-04-26T03:39:26.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:26.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505829+0000", 2024-04-26T03:39:26.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:26.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505830+0000", 2024-04-26T03:39:26.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2e-07 2024-04-26T03:39:26.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:26.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505832+0000", 2024-04-26T03:39:26.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.509e-06 2024-04-26T03:39:26.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:26.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505839+0000", 2024-04-26T03:39:26.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9110000000000004e-06 2024-04-26T03:39:26.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:26.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505844+0000", 2024-04-26T03:39:26.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2109999999999997e-06 2024-04-26T03:39:26.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.510372+0000", 2024-04-26T03:39:26.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045275990000000002 2024-04-26T03:39:26.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:26.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.510419+0000", 2024-04-26T03:39:26.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7524000000000001e-05 2024-04-26T03:39:26.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.522258+0000", 2024-04-26T03:39:26.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011838448999999999 2024-04-26T03:39:26.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:26.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.522334+0000", 2024-04-26T03:39:26.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6134e-05 2024-04-26T03:39:26.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:26.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.527235+0000", 2024-04-26T03:39:26.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049012650000000001 2024-04-26T03:39:26.408 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.408 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:26.408 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.408 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:64 3.ds0 3:b3f958c3:::smithi09235267-22:head [append 0~360448,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:26.408 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.515665+0000", 2024-04-26T03:39:26.408 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.866658166000001, 2024-04-26T03:39:26.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.023538667999999999, 2024-04-26T03:39:26.408 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:26.408 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:26.409 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:26.409 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:26.409 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:26.409 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:26.409 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 64 2024-04-26T03:39:26.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.409 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:26.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:26.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515665+0000", 2024-04-26T03:39:26.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:26.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515665+0000", 2024-04-26T03:39:26.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:26.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515667+0000", 2024-04-26T03:39:26.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.652e-06 2024-04-26T03:39:26.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:26.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515857+0000", 2024-04-26T03:39:26.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000190447 2024-04-26T03:39:26.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:26.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515869+0000", 2024-04-26T03:39:26.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1450999999999999e-05 2024-04-26T03:39:26.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:26.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515876+0000", 2024-04-26T03:39:26.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2024-04-26T03:39:26.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.527263+0000", 2024-04-26T03:39:26.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011387061 2024-04-26T03:39:26.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:26.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.527389+0000", 2024-04-26T03:39:26.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000126172 2024-04-26T03:39:26.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:26.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.528593+0000", 2024-04-26T03:39:26.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00120386 2024-04-26T03:39:26.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:26.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.533773+0000", 2024-04-26T03:39:26.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005179973 2024-04-26T03:39:26.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:26.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.539131+0000", 2024-04-26T03:39:26.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053582279999999996 2024-04-26T03:39:26.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:26.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.539204+0000", 2024-04-26T03:39:26.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2809e-05 2024-04-26T03:39:26.413 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.413 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:26.413 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.414 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:65 3.ds0 3:b3f958c3:::smithi09235267-22:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:26.414 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.515889+0000", 2024-04-26T03:39:26.414 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.866434525999999, 2024-04-26T03:39:26.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.023486671000000001, 2024-04-26T03:39:26.414 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:26.414 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:26.414 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:26.414 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:26.414 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:26.414 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:26.414 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 65 2024-04-26T03:39:26.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.415 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:26.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:26.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515889+0000", 2024-04-26T03:39:26.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:26.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515889+0000", 2024-04-26T03:39:26.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:26.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515890+0000", 2024-04-26T03:39:26.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.13e-07 2024-04-26T03:39:26.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:26.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515894+0000", 2024-04-26T03:39:26.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.386e-06 2024-04-26T03:39:26.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:26.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515903+0000", 2024-04-26T03:39:26.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.166e-06 2024-04-26T03:39:26.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:26.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515909+0000", 2024-04-26T03:39:26.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.412e-06 2024-04-26T03:39:26.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.532799+0000", 2024-04-26T03:39:26.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016889990000000001 2024-04-26T03:39:26.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:26.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.532885+0000", 2024-04-26T03:39:26.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6336999999999997e-05 2024-04-26T03:39:26.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:26.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.533334+0000", 2024-04-26T03:39:26.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00044950799999999999 2024-04-26T03:39:26.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:26.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.538492+0000", 2024-04-26T03:39:26.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051571389999999998 2024-04-26T03:39:26.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:26.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.539303+0000", 2024-04-26T03:39:26.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000811325 2024-04-26T03:39:26.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:26.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.539376+0000", 2024-04-26T03:39:26.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2794999999999997e-05 2024-04-26T03:39:26.419 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.419 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:26.419 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.419 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:66 3.ds0 3:b3f958c3:::smithi09235267-22:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:26.419 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.515917+0000", 2024-04-26T03:39:26.419 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.866406951000002, 2024-04-26T03:39:26.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.024757049, 2024-04-26T03:39:26.419 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:26.420 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:26.420 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:26.420 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:26.420 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:26.420 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:26.420 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 66 2024-04-26T03:39:26.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.420 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:26.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:26.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515917+0000", 2024-04-26T03:39:26.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:26.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515917+0000", 2024-04-26T03:39:26.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:26.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515917+0000", 2024-04-26T03:39:26.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8699999999999995e-07 2024-04-26T03:39:26.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:26.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515920+0000", 2024-04-26T03:39:26.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.055e-06 2024-04-26T03:39:26.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:26.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515927+0000", 2024-04-26T03:39:26.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5150000000000003e-06 2024-04-26T03:39:26.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:26.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515936+0000", 2024-04-26T03:39:26.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7920000000000004e-06 2024-04-26T03:39:26.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.533517+0000", 2024-04-26T03:39:26.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017580535000000001 2024-04-26T03:39:26.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:26.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.533598+0000", 2024-04-26T03:39:26.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1354999999999996e-05 2024-04-26T03:39:26.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.539706+0000", 2024-04-26T03:39:26.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061075569999999996 2024-04-26T03:39:26.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:26.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.539758+0000", 2024-04-26T03:39:26.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2500000000000002e-05 2024-04-26T03:39:26.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:26.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.540674+0000", 2024-04-26T03:39:26.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00091525299999999999 2024-04-26T03:39:26.424 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.424 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:26.424 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.424 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:607 3.ds0 3:b4337c88:::smithi09235267-15:head [append 0~737280] snapc d=[d,c,b,a] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-26T03:39:26.425 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:11.722465+0000", 2024-04-26T03:39:26.425 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.659858373, 2024-04-26T03:39:26.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019589842999999999, 2024-04-26T03:39:26.425 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:26.425 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:26.425 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:26.425 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:26.425 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:26.425 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:26.425 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 607 2024-04-26T03:39:26.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.425 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:26.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:26.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.722465+0000", 2024-04-26T03:39:26.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:26.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.722465+0000", 2024-04-26T03:39:26.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:26.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.722467+0000", 2024-04-26T03:39:26.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.553e-06 2024-04-26T03:39:26.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:26.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.722788+0000", 2024-04-26T03:39:26.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00032146600000000002 2024-04-26T03:39:26.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:26.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.722805+0000", 2024-04-26T03:39:26.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6534000000000001e-05 2024-04-26T03:39:26.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:26.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.722815+0000", 2024-04-26T03:39:26.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0062e-05 2024-04-26T03:39:26.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.722890+0000", 2024-04-26T03:39:26.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5012000000000006e-05 2024-04-26T03:39:26.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:26.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.723013+0000", 2024-04-26T03:39:26.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000123693 2024-04-26T03:39:26.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:26.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.724487+0000", 2024-04-26T03:39:26.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001473427 2024-04-26T03:39:26.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:26.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.741651+0000", 2024-04-26T03:39:26.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017163754999999999 2024-04-26T03:39:26.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:26.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.741886+0000", 2024-04-26T03:39:26.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0002354 2024-04-26T03:39:26.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:26.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.742055+0000", 2024-04-26T03:39:26.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016894100000000001 2024-04-26T03:39:26.430 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.430 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:26.430 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.430 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:608 3.ds0 3:b4337c88:::smithi09235267-15:head [append 0~557056] snapc d=[d,c,b,a] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-26T03:39:26.430 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:11.726593+0000", 2024-04-26T03:39:26.430 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.655730137999999, 2024-04-26T03:39:26.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019055236999999999, 2024-04-26T03:39:26.431 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:26.431 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:26.431 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:26.431 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:26.431 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:26.431 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:26.431 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 608 2024-04-26T03:39:26.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.431 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:26.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:26.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.726593+0000", 2024-04-26T03:39:26.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:26.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.726593+0000", 2024-04-26T03:39:26.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:26.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.726596+0000", 2024-04-26T03:39:26.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3649999999999998e-06 2024-04-26T03:39:26.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:26.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.726927+0000", 2024-04-26T03:39:26.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00033128999999999998 2024-04-26T03:39:26.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:26.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.726950+0000", 2024-04-26T03:39:26.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.317e-05 2024-04-26T03:39:26.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:26.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.726961+0000", 2024-04-26T03:39:26.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0387999999999999e-05 2024-04-26T03:39:26.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.730805+0000", 2024-04-26T03:39:26.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003843993 2024-04-26T03:39:26.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:26.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.730917+0000", 2024-04-26T03:39:26.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011274599999999999 2024-04-26T03:39:26.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:26.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.731823+0000", 2024-04-26T03:39:26.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00090538700000000001 2024-04-26T03:39:26.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:26.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.742998+0000", 2024-04-26T03:39:26.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011175144 2024-04-26T03:39:26.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:26.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.745560+0000", 2024-04-26T03:39:26.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025625660000000001 2024-04-26T03:39:26.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:26.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.745649+0000", 2024-04-26T03:39:26.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8188000000000003e-05 2024-04-26T03:39:26.435 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.436 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:26.436 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.436 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:610 3.ds0 3:b4337c88:::smithi09235267-15:head [setxattr _header (57) in=7b] snapc d=[d,c,b,a] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-26T03:39:26.436 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:11.731469+0000", 2024-04-26T03:39:26.436 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.650854692999999, 2024-04-26T03:39:26.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015326482000000001, 2024-04-26T03:39:26.436 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:26.436 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:26.436 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:26.436 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:26.436 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:26.436 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:26.437 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 610 2024-04-26T03:39:26.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.437 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:26.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:26.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.731469+0000", 2024-04-26T03:39:26.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:26.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.731469+0000", 2024-04-26T03:39:26.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:26.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.731469+0000", 2024-04-26T03:39:26.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2800000000000002e-07 2024-04-26T03:39:26.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:26.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.731473+0000", 2024-04-26T03:39:26.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.456e-06 2024-04-26T03:39:26.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:26.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.731478+0000", 2024-04-26T03:39:26.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9860000000000002e-06 2024-04-26T03:39:26.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:26.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.731481+0000", 2024-04-26T03:39:26.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2590000000000001e-06 2024-04-26T03:39:26.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.742103+0000", 2024-04-26T03:39:26.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010621680999999999 2024-04-26T03:39:26.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:26.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.742180+0000", 2024-04-26T03:39:26.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7392999999999996e-05 2024-04-26T03:39:26.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:26.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.742694+0000", 2024-04-26T03:39:26.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000513654 2024-04-26T03:39:26.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:26.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.745660+0000", 2024-04-26T03:39:26.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002966836 2024-04-26T03:39:26.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:26.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.746708+0000", 2024-04-26T03:39:26.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010471619999999999 2024-04-26T03:39:26.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:26.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.746795+0000", 2024-04-26T03:39:26.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7627000000000007e-05 2024-04-26T03:39:26.441 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.441 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:26.441 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.441 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:611 3.ds0 3:b4337c88:::smithi09235267-15:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-04-26T03:39:26.441 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:11.731485+0000", 2024-04-26T03:39:26.442 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.650837993, 2024-04-26T03:39:26.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017052068, 2024-04-26T03:39:26.442 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:26.442 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:26.442 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:26.442 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:26.442 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:26.442 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:26.442 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 611 2024-04-26T03:39:26.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.442 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:26.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:26.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.731485+0000", 2024-04-26T03:39:26.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:26.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.731485+0000", 2024-04-26T03:39:26.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:26.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.731486+0000", 2024-04-26T03:39:26.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6899999999999999e-07 2024-04-26T03:39:26.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:26.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.731487+0000", 2024-04-26T03:39:26.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.44e-06 2024-04-26T03:39:26.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:26.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.731492+0000", 2024-04-26T03:39:26.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9930000000000003e-06 2024-04-26T03:39:26.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:26.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.731495+0000", 2024-04-26T03:39:26.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.022e-06 2024-04-26T03:39:26.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.743270+0000", 2024-04-26T03:39:26.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011774316 2024-04-26T03:39:26.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:26.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.743323+0000", 2024-04-26T03:39:26.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3463e-05 2024-04-26T03:39:26.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.747168+0000", 2024-04-26T03:39:26.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003844859 2024-04-26T03:39:26.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:26.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.747255+0000", 2024-04-26T03:39:26.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7052000000000006e-05 2024-04-26T03:39:26.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:26.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.748538+0000", 2024-04-26T03:39:26.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001282654 2024-04-26T03:39:26.447 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.447 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:26.447 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.447 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:690 3.ds0 3:b895fe71:::smithi09235267-18:head [append 0~720896] snapc f=[f,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-04-26T03:39:26.447 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:15.944310+0000", 2024-04-26T03:39:26.447 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.438013914000001, 2024-04-26T03:39:26.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016189867, 2024-04-26T03:39:26.447 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:26.447 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:26.447 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:26.447 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:26.447 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:26.447 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:26.448 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 690 2024-04-26T03:39:26.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.448 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:26.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:26.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.944310+0000", 2024-04-26T03:39:26.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:26.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.944310+0000", 2024-04-26T03:39:26.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:26.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.944311+0000", 2024-04-26T03:39:26.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2500000000000001e-06 2024-04-26T03:39:26.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:26.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.944635+0000", 2024-04-26T03:39:26.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000324271 2024-04-26T03:39:26.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:26.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.944644+0000", 2024-04-26T03:39:26.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1670000000000005e-06 2024-04-26T03:39:26.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:26.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.944651+0000", 2024-04-26T03:39:26.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4300000000000003e-06 2024-04-26T03:39:26.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.944689+0000", 2024-04-26T03:39:26.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8272000000000002e-05 2024-04-26T03:39:26.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:26.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.944788+0000", 2024-04-26T03:39:26.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8916e-05 2024-04-26T03:39:26.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:26.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.945668+0000", 2024-04-26T03:39:26.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00088055000000000002 2024-04-26T03:39:26.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:26.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.960314+0000", 2024-04-26T03:39:26.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014645467000000001 2024-04-26T03:39:26.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:26.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.960414+0000", 2024-04-26T03:39:26.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.98e-05 2024-04-26T03:39:26.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:26.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.960499+0000", 2024-04-26T03:39:26.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5743999999999998e-05 2024-04-26T03:39:26.452 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.452 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:26.452 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.452 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:694 3.ds0 3:b895fe71:::smithi09235267-18:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e60)", 2024-04-26T03:39:26.452 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:15.950694+0000", 2024-04-26T03:39:26.453 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.431629613, 2024-04-26T03:39:26.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018244219999999998, 2024-04-26T03:39:26.453 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:26.453 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:26.453 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:26.453 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:26.453 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:26.453 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:26.453 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 694 2024-04-26T03:39:26.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.453 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:26.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:26.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.950694+0000", 2024-04-26T03:39:26.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:26.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.950694+0000", 2024-04-26T03:39:26.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:26.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.950694+0000", 2024-04-26T03:39:26.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.82e-07 2024-04-26T03:39:26.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:26.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.950697+0000", 2024-04-26T03:39:26.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.559e-06 2024-04-26T03:39:26.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:26.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.950704+0000", 2024-04-26T03:39:26.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.917e-06 2024-04-26T03:39:26.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:26.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.950713+0000", 2024-04-26T03:39:26.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5419999999999994e-06 2024-04-26T03:39:26.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.962925+0000", 2024-04-26T03:39:26.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012211926999999999 2024-04-26T03:39:26.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:26.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.963013+0000", 2024-04-26T03:39:26.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7791e-05 2024-04-26T03:39:26.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.966400+0000", 2024-04-26T03:39:26.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003386816 2024-04-26T03:39:26.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:26.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.966566+0000", 2024-04-26T03:39:26.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000166525 2024-04-26T03:39:26.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:26.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.968938+0000", 2024-04-26T03:39:26.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023716610000000002 2024-04-26T03:39:26.457 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.457 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:26.457 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.458 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.10s2 60/48 ECSubWrite(tid=272, reqid=client.4556.0:703, at_version=60'20, trim_to=0'0, roll_forward_to=60'18))", 2024-04-26T03:39:26.458 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:15.958431+0000", 2024-04-26T03:39:26.458 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.423892669000001, 2024-04-26T03:39:26.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016173328000000001, 2024-04-26T03:39:26.458 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:26.458 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:26.458 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:26.458 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:26.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.458 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:26.458 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.958431+0000", 2024-04-26T03:39:26.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:26.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.958431+0000", 2024-04-26T03:39:26.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:26.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.958432+0000", 2024-04-26T03:39:26.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.031e-06 2024-04-26T03:39:26.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:26.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.962281+0000", 2024-04-26T03:39:26.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0038488630000000001 2024-04-26T03:39:26.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:26.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.962301+0000", 2024-04-26T03:39:26.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9828999999999999e-05 2024-04-26T03:39:26.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:26.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.962308+0000", 2024-04-26T03:39:26.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2620000000000001e-06 2024-04-26T03:39:26.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.962375+0000", 2024-04-26T03:39:26.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6729000000000004e-05 2024-04-26T03:39:26.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:26.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.962384+0000", 2024-04-26T03:39:26.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6390000000000004e-06 2024-04-26T03:39:26.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:26.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.974567+0000", 2024-04-26T03:39:26.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012182767000000001 2024-04-26T03:39:26.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:26.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.974604+0000", 2024-04-26T03:39:26.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7208e-05 2024-04-26T03:39:26.462 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.462 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:26.462 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.462 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:774 3.ds0 3:b3f958c3:::smithi09235267-22:head [append 0~737280] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:26.462 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:20.780689+0000", 2024-04-26T03:39:26.462 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.60163469, 2024-04-26T03:39:26.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021054691, 2024-04-26T03:39:26.462 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:26.462 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:26.462 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:26.463 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:26.463 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:26.463 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:26.463 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 774 2024-04-26T03:39:26.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.463 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:26.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:26.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.780689+0000", 2024-04-26T03:39:26.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:26.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.780689+0000", 2024-04-26T03:39:26.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:26.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.780690+0000", 2024-04-26T03:39:26.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3400000000000001e-06 2024-04-26T03:39:26.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:26.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.780841+0000", 2024-04-26T03:39:26.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000150743 2024-04-26T03:39:26.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:26.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.780853+0000", 2024-04-26T03:39:26.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1681e-05 2024-04-26T03:39:26.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:26.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.780858+0000", 2024-04-26T03:39:26.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8390000000000003e-06 2024-04-26T03:39:26.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.780915+0000", 2024-04-26T03:39:26.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6338000000000002e-05 2024-04-26T03:39:26.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:26.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.781021+0000", 2024-04-26T03:39:26.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106498 2024-04-26T03:39:26.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:26.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.782158+0000", 2024-04-26T03:39:26.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001136977 2024-04-26T03:39:26.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:26.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.795362+0000", 2024-04-26T03:39:26.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013204052000000001 2024-04-26T03:39:26.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:26.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.801661+0000", 2024-04-26T03:39:26.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0062988540000000004 2024-04-26T03:39:26.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:26.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.801743+0000", 2024-04-26T03:39:26.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2368999999999997e-05 2024-04-26T03:39:26.467 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.467 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:26.467 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.468 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:775 3.ds0 3:b3f958c3:::smithi09235267-22:head [append 0~770048] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:26.468 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:20.785167+0000", 2024-04-26T03:39:26.468 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.597156654, 2024-04-26T03:39:26.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017555833, 2024-04-26T03:39:26.468 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:26.468 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:26.468 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:26.468 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:26.468 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:26.468 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:26.468 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 775 2024-04-26T03:39:26.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.468 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:26.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:26.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.785167+0000", 2024-04-26T03:39:26.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:26.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.785167+0000", 2024-04-26T03:39:26.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:26.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.785168+0000", 2024-04-26T03:39:26.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.339e-06 2024-04-26T03:39:26.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:26.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.785488+0000", 2024-04-26T03:39:26.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031973100000000002 2024-04-26T03:39:26.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:26.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.785499+0000", 2024-04-26T03:39:26.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1409999999999999e-05 2024-04-26T03:39:26.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:26.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.785506+0000", 2024-04-26T03:39:26.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.224e-06 2024-04-26T03:39:26.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.789235+0000", 2024-04-26T03:39:26.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0037290069999999999 2024-04-26T03:39:26.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:26.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.789350+0000", 2024-04-26T03:39:26.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115327 2024-04-26T03:39:26.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:26.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.790178+0000", 2024-04-26T03:39:26.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00082851799999999998 2024-04-26T03:39:26.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:26.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.801460+0000", 2024-04-26T03:39:26.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011282003000000001 2024-04-26T03:39:26.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:26.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.802652+0000", 2024-04-26T03:39:26.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011913889999999999 2024-04-26T03:39:26.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:26.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.802723+0000", 2024-04-26T03:39:26.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0884999999999996e-05 2024-04-26T03:39:26.473 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.473 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:26.473 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.473 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:776 3.ds0 3:b3f958c3:::smithi09235267-22:head [append 0~540672] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:26.473 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:20.788324+0000", 2024-04-26T03:39:26.473 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.5939997159999999, 2024-04-26T03:39:26.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016617261000000001, 2024-04-26T03:39:26.473 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:26.473 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:26.473 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:26.473 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:26.474 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:26.474 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:26.474 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 776 2024-04-26T03:39:26.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.474 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:26.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:26.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.788324+0000", 2024-04-26T03:39:26.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:26.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.788324+0000", 2024-04-26T03:39:26.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:26.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.788325+0000", 2024-04-26T03:39:26.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7010000000000001e-06 2024-04-26T03:39:26.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:26.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.788621+0000", 2024-04-26T03:39:26.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029502800000000002 2024-04-26T03:39:26.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:26.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.788635+0000", 2024-04-26T03:39:26.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4741e-05 2024-04-26T03:39:26.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:26.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.788643+0000", 2024-04-26T03:39:26.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9610000000000005e-06 2024-04-26T03:39:26.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.795417+0000", 2024-04-26T03:39:26.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067738520000000003 2024-04-26T03:39:26.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:26.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.795523+0000", 2024-04-26T03:39:26.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010549599999999999 2024-04-26T03:39:26.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:26.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.796141+0000", 2024-04-26T03:39:26.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00061875999999999999 2024-04-26T03:39:26.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:26.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.801534+0000", 2024-04-26T03:39:26.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053927899999999997 2024-04-26T03:39:26.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:26.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.804742+0000", 2024-04-26T03:39:26.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003207522 2024-04-26T03:39:26.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:26.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.804941+0000", 2024-04-26T03:39:26.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019940999999999999 2024-04-26T03:39:26.478 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.478 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:26.478 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.479 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:777 3.ds0 3:b3f958c3:::smithi09235267-22:head [append 0~245760] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:26.479 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:20.789857+0000", 2024-04-26T03:39:26.479 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.5924661039999997, 2024-04-26T03:39:26.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015317147999999999, 2024-04-26T03:39:26.479 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:26.479 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:26.479 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:26.479 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:26.479 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:26.479 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:26.479 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 777 2024-04-26T03:39:26.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.479 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:26.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:26.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.789857+0000", 2024-04-26T03:39:26.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:26.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.789857+0000", 2024-04-26T03:39:26.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:26.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.789859+0000", 2024-04-26T03:39:26.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2869999999999999e-06 2024-04-26T03:39:26.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:26.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.789955+0000", 2024-04-26T03:39:26.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6141e-05 2024-04-26T03:39:26.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:26.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.789979+0000", 2024-04-26T03:39:26.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3713000000000001e-05 2024-04-26T03:39:26.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:26.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.789986+0000", 2024-04-26T03:39:26.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.131e-06 2024-04-26T03:39:26.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.482 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.799260+0000", 2024-04-26T03:39:26.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.009274572 2024-04-26T03:39:26.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.482 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:26.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.799331+0000", 2024-04-26T03:39:26.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0622000000000002e-05 2024-04-26T03:39:26.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.482 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:26.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.799793+0000", 2024-04-26T03:39:26.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00046205500000000001 2024-04-26T03:39:26.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:26.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.801843+0000", 2024-04-26T03:39:26.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020499139999999999 2024-04-26T03:39:26.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:26.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.805137+0000", 2024-04-26T03:39:26.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0032937779999999998 2024-04-26T03:39:26.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:26.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.805175+0000", 2024-04-26T03:39:26.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7935000000000002e-05 2024-04-26T03:39:26.484 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.484 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:26.484 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.484 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:778 3.ds0 3:b3f958c3:::smithi09235267-22:head [setxattr _header (57) in=7b] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:26.484 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:20.789998+0000", 2024-04-26T03:39:26.484 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.5923254880000002, 2024-04-26T03:39:26.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015643300999999998, 2024-04-26T03:39:26.484 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:26.484 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:26.484 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:26.485 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:26.485 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:26.485 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:26.485 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 778 2024-04-26T03:39:26.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.485 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:26.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:26.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.789998+0000", 2024-04-26T03:39:26.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:26.486 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.789998+0000", 2024-04-26T03:39:26.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.486 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:26.486 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.789999+0000", 2024-04-26T03:39:26.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9800000000000003e-07 2024-04-26T03:39:26.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.486 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:26.486 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.790002+0000", 2024-04-26T03:39:26.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.597e-06 2024-04-26T03:39:26.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.487 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:26.487 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.790009+0000", 2024-04-26T03:39:26.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9700000000000002e-06 2024-04-26T03:39:26.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.487 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:26.487 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.790015+0000", 2024-04-26T03:39:26.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6350000000000003e-06 2024-04-26T03:39:26.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.487 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.487 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.801932+0000", 2024-04-26T03:39:26.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011917669000000001 2024-04-26T03:39:26.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:26.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.801990+0000", 2024-04-26T03:39:26.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7822000000000003e-05 2024-04-26T03:39:26.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:26.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.802279+0000", 2024-04-26T03:39:26.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028822199999999998 2024-04-26T03:39:26.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:26.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.803223+0000", 2024-04-26T03:39:26.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00094406100000000005 2024-04-26T03:39:26.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.489 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:26.489 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.805407+0000", 2024-04-26T03:39:26.489 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0021844360000000001 2024-04-26T03:39:26.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.489 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:26.489 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.805641+0000", 2024-04-26T03:39:26.489 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000234291 2024-04-26T03:39:26.489 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.489 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:26.489 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.489 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:779 3.ds0 3:b3f958c3:::smithi09235267-22:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:26.490 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:20.790023+0000", 2024-04-26T03:39:26.490 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.5923005899999998, 2024-04-26T03:39:26.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017774486999999999, 2024-04-26T03:39:26.490 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:26.490 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:26.490 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:26.490 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:26.490 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:26.490 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:26.490 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 779 2024-04-26T03:39:26.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.490 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:26.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.490 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:26.491 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.790023+0000", 2024-04-26T03:39:26.491 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.491 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:26.491 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.790023+0000", 2024-04-26T03:39:26.491 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.491 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:26.491 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.790023+0000", 2024-04-26T03:39:26.491 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4200000000000001e-07 2024-04-26T03:39:26.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.491 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:26.492 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.790026+0000", 2024-04-26T03:39:26.492 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1159999999999999e-06 2024-04-26T03:39:26.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.492 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:26.492 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.790033+0000", 2024-04-26T03:39:26.492 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6100000000000002e-06 2024-04-26T03:39:26.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.492 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:26.492 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.790038+0000", 2024-04-26T03:39:26.492 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9439999999999996e-06 2024-04-26T03:39:26.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.492 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.493 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.802971+0000", 2024-04-26T03:39:26.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012933445 2024-04-26T03:39:26.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.493 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.493 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:26.493 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.803177+0000", 2024-04-26T03:39:26.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020553499999999999 2024-04-26T03:39:26.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.493 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.493 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.493 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.806365+0000", 2024-04-26T03:39:26.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003187615 2024-04-26T03:39:26.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.494 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.494 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:26.494 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.806453+0000", 2024-04-26T03:39:26.494 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8240999999999998e-05 2024-04-26T03:39:26.494 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.494 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.494 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:26.494 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.807797+0000", 2024-04-26T03:39:26.494 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013445390000000001 2024-04-26T03:39:26.494 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.494 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:26.494 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.494 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.495 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:26.495 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:26.496 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-04-26T03:39:26.497 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-04-26T03:39:26.494+0000 7f745118d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 31821) UID: 0 2024-04-26T03:39:26.597 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:39:26.595+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:39:26.639 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:26.640 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-26T03:39:26.640 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-26T03:39:26.640 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:26.648 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-04-26T03:39:26.689 INFO:teuthology.orchestra.run.smithi028.stderr:nodeep-scrub is set 2024-04-26T03:39:26.693 INFO:tasks.rados.rados.0.smithi092.stdout:321: done (2 left) 2024-04-26T03:39:26.694 INFO:tasks.rados.rados.0.smithi092.stdout:322: snap_create 2024-04-26T03:39:26.694 INFO:tasks.rados.rados.0.smithi092.stdout:320: finishing rollback tid 0 to smithi09235267-22 2024-04-26T03:39:26.694 INFO:tasks.rados.rados.0.smithi092.stdout:320: finishing rollback tid 1 to smithi09235267-22 2024-04-26T03:39:26.694 INFO:tasks.rados.rados.0.smithi092.stdout:320: finishing rollback tid 2 to smithi09235267-22 2024-04-26T03:39:26.694 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 22 v 63 (ObjNum 136 snap 17 seq_num 136) dirty exists 2024-04-26T03:39:26.694 INFO:tasks.rados.rados.0.smithi092.stdout:319: finishing write tid 1 to smithi09235267-17 2024-04-26T03:39:26.694 INFO:tasks.rados.rados.0.smithi092.stdout:319: finishing write tid 2 to smithi09235267-17 2024-04-26T03:39:26.694 INFO:tasks.rados.rados.0.smithi092.stdout:319: finishing write tid 3 to smithi09235267-17 2024-04-26T03:39:26.694 INFO:tasks.rados.rados.0.smithi092.stdout:319: oid 17 updating version 0 to 45 2024-04-26T03:39:26.694 INFO:tasks.rados.rados.0.smithi092.stdout:319: oid 17 updating version 45 to 46 2024-04-26T03:39:26.694 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 17 v 46 (ObjNum 147 snap 21 seq_num 147) dirty exists 2024-04-26T03:39:26.694 INFO:tasks.rados.rados.0.smithi092.stdout:319: left oid 17 (ObjNum 147 snap 21 seq_num 147) 2024-04-26T03:39:26.702 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:39:26.700+0000 7fba0d2ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31961) UID: 0 2024-04-26T03:39:26.756 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:26.756 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-26T03:39:26.757 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-26T03:39:26.757 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-26T03:39:26.757 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:26.765 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-04-26T03:39:26.801 INFO:tasks.ceph.osd.9.smithi042.stderr:2024-04-26T03:39:26.799+0000 7fdac63b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 31960) UID: 0 2024-04-26T03:39:26.901 INFO:tasks.ceph.osd.9.smithi042.stderr:2024-04-26T03:39:26.899+0000 7fdac63b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 31960) UID: 0 2024-04-26T03:39:26.912 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:26.912 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-26T03:39:26.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-26T03:39:26.912 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-26T03:39:26.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.912 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:12 3.cs0 3:3946a5f1:::smithi09235267-4:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:26.912 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.481722+0000", 2024-04-26T03:39:26.913 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.427939313000003, 2024-04-26T03:39:26.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039281273999999998, 2024-04-26T03:39:26.913 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:26.913 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:26.913 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:26.913 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:26.913 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:26.913 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:26.913 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 12 2024-04-26T03:39:26.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.914 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:26.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:26.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.481722+0000", 2024-04-26T03:39:26.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:26.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.481722+0000", 2024-04-26T03:39:26.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:26.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.481723+0000", 2024-04-26T03:39:26.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.75e-07 2024-04-26T03:39:26.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:26.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.481725+0000", 2024-04-26T03:39:26.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6979999999999998e-06 2024-04-26T03:39:26.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:26.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.481733+0000", 2024-04-26T03:39:26.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2899999999999997e-06 2024-04-26T03:39:26.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:26.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.481743+0000", 2024-04-26T03:39:26.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0845999999999999e-05 2024-04-26T03:39:26.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.484203+0000", 2024-04-26T03:39:26.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024598889999999998 2024-04-26T03:39:26.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:26.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.484353+0000", 2024-04-26T03:39:26.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015005199999999999 2024-04-26T03:39:26.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.497880+0000", 2024-04-26T03:39:26.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013526589 2024-04-26T03:39:26.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:26.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.497957+0000", 2024-04-26T03:39:26.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7126000000000007e-05 2024-04-26T03:39:26.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:26.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.521003+0000", 2024-04-26T03:39:26.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.023046309000000001 2024-04-26T03:39:26.918 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.918 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:26.918 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.918 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:31 3.cs0 3:3edefb40:::smithi09235267-11:head [append 0~180224,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:26.918 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.494104+0000", 2024-04-26T03:39:26.918 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.415557008999997, 2024-04-26T03:39:26.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025578053999999999, 2024-04-26T03:39:26.918 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:26.918 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:26.918 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:26.918 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:26.919 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:26.919 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:26.919 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 31 2024-04-26T03:39:26.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.919 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:26.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.919 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:26.919 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.494104+0000", 2024-04-26T03:39:26.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:26.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.494104+0000", 2024-04-26T03:39:26.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:26.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.494106+0000", 2024-04-26T03:39:26.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2100000000000001e-06 2024-04-26T03:39:26.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:26.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.494218+0000", 2024-04-26T03:39:26.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000112398 2024-04-26T03:39:26.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:26.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.494230+0000", 2024-04-26T03:39:26.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1688999999999999e-05 2024-04-26T03:39:26.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:26.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.494237+0000", 2024-04-26T03:39:26.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2049999999999996e-06 2024-04-26T03:39:26.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.494745+0000", 2024-04-26T03:39:26.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00050836900000000005 2024-04-26T03:39:26.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:26.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.494857+0000", 2024-04-26T03:39:26.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000112045 2024-04-26T03:39:26.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:26.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495764+0000", 2024-04-26T03:39:26.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00090692400000000003 2024-04-26T03:39:26.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:26.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519293+0000", 2024-04-26T03:39:26.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.023528830000000001 2024-04-26T03:39:26.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:26.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519597+0000", 2024-04-26T03:39:26.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030426100000000001 2024-04-26T03:39:26.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:26.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519682+0000", 2024-04-26T03:39:26.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5123000000000005e-05 2024-04-26T03:39:26.923 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.924 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:26.924 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.924 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:32 3.cs0 3:3edefb40:::smithi09235267-11:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:26.924 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.495390+0000", 2024-04-26T03:39:26.924 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.414271092999996, 2024-04-26T03:39:26.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027633438, 2024-04-26T03:39:26.924 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:26.924 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:26.924 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:26.924 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:26.924 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:26.924 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:26.925 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 32 2024-04-26T03:39:26.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.925 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:26.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:26.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495390+0000", 2024-04-26T03:39:26.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:26.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495390+0000", 2024-04-26T03:39:26.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:26.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495391+0000", 2024-04-26T03:39:26.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8999999999999995e-07 2024-04-26T03:39:26.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:26.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495396+0000", 2024-04-26T03:39:26.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4360000000000001e-06 2024-04-26T03:39:26.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:26.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495410+0000", 2024-04-26T03:39:26.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3482000000000001e-05 2024-04-26T03:39:26.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:26.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495419+0000", 2024-04-26T03:39:26.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9600000000000006e-06 2024-04-26T03:39:26.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518418+0000", 2024-04-26T03:39:26.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022999116999999999 2024-04-26T03:39:26.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:26.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518517+0000", 2024-04-26T03:39:26.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8542999999999995e-05 2024-04-26T03:39:26.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:26.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519020+0000", 2024-04-26T03:39:26.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00050364099999999998 2024-04-26T03:39:26.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:26.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519837+0000", 2024-04-26T03:39:26.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00081666699999999998 2024-04-26T03:39:26.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:26.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.522936+0000", 2024-04-26T03:39:26.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030991170000000002 2024-04-26T03:39:26.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:26.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.523024+0000", 2024-04-26T03:39:26.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7684999999999998e-05 2024-04-26T03:39:26.929 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.929 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:26.929 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.929 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:33 3.cs0 3:3edefb40:::smithi09235267-11:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:26.929 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.495433+0000", 2024-04-26T03:39:26.930 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.414228170999998, 2024-04-26T03:39:26.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032718572000000001, 2024-04-26T03:39:26.930 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:26.930 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:26.930 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:26.930 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:26.930 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:26.930 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:26.931 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 33 2024-04-26T03:39:26.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.931 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:26.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:26.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495433+0000", 2024-04-26T03:39:26.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:26.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495433+0000", 2024-04-26T03:39:26.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:26.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495434+0000", 2024-04-26T03:39:26.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2300000000000001e-07 2024-04-26T03:39:26.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:26.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495437+0000", 2024-04-26T03:39:26.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2399999999999999e-06 2024-04-26T03:39:26.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:26.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495444+0000", 2024-04-26T03:39:26.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3869999999999997e-06 2024-04-26T03:39:26.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:26.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495450+0000", 2024-04-26T03:39:26.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.412e-06 2024-04-26T03:39:26.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.520715+0000", 2024-04-26T03:39:26.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.02526488 2024-04-26T03:39:26.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:26.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.520783+0000", 2024-04-26T03:39:26.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8553000000000006e-05 2024-04-26T03:39:26.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.523315+0000", 2024-04-26T03:39:26.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025314299999999999 2024-04-26T03:39:26.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:26.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.523384+0000", 2024-04-26T03:39:26.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9503000000000002e-05 2024-04-26T03:39:26.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:26.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.528152+0000", 2024-04-26T03:39:26.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047676439999999997 2024-04-26T03:39:26.935 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.935 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:26.935 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.935 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.cs0 35/32 ECSubReadReply(tid=5, attrs_read=0))", 2024-04-26T03:39:26.935 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.498822+0000", 2024-04-26T03:39:26.935 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.410839527, 2024-04-26T03:39:26.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021016639, 2024-04-26T03:39:26.935 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:26.935 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:26.935 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:26.935 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:26.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:26.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498822+0000", 2024-04-26T03:39:26.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:26.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498822+0000", 2024-04-26T03:39:26.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:26.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498822+0000", 2024-04-26T03:39:26.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3099999999999997e-07 2024-04-26T03:39:26.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:26.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498834+0000", 2024-04-26T03:39:26.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1331e-05 2024-04-26T03:39:26.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:26.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498878+0000", 2024-04-26T03:39:26.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4092999999999999e-05 2024-04-26T03:39:26.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:26.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498888+0000", 2024-04-26T03:39:26.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0049e-05 2024-04-26T03:39:26.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519785+0000", 2024-04-26T03:39:26.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020896656 2024-04-26T03:39:26.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:26.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519838+0000", 2024-04-26T03:39:26.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3878999999999997e-05 2024-04-26T03:39:26.938 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.938 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:26.938 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.939 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 35/32 ECSubWriteReply(tid=4, last_complete=35'3, committed=1, applied=1))", 2024-04-26T03:39:26.939 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.499205+0000", 2024-04-26T03:39:26.939 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.410456519999997, 2024-04-26T03:39:26.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020089540999999999, 2024-04-26T03:39:26.939 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:26.939 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:26.939 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:26.939 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:26.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:26.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499205+0000", 2024-04-26T03:39:26.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:26.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499205+0000", 2024-04-26T03:39:26.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:26.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499205+0000", 2024-04-26T03:39:26.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5199999999999996e-07 2024-04-26T03:39:26.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:26.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499210+0000", 2024-04-26T03:39:26.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0690000000000003e-06 2024-04-26T03:39:26.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:26.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499223+0000", 2024-04-26T03:39:26.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3783e-05 2024-04-26T03:39:26.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:26.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499230+0000", 2024-04-26T03:39:26.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7970000000000003e-06 2024-04-26T03:39:26.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519273+0000", 2024-04-26T03:39:26.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020043068000000001 2024-04-26T03:39:26.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:26.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519294+0000", 2024-04-26T03:39:26.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1172e-05 2024-04-26T03:39:26.942 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.942 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:26.943 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.943 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 35/32 ECSubWriteReply(tid=4, last_complete=35'3, committed=1, applied=1))", 2024-04-26T03:39:26.943 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.499957+0000", 2024-04-26T03:39:26.943 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.409704734000002, 2024-04-26T03:39:26.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019482105, 2024-04-26T03:39:26.944 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:26.944 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:26.944 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:26.944 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:26.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:26.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499957+0000", 2024-04-26T03:39:26.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:26.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499957+0000", 2024-04-26T03:39:26.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:26.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499957+0000", 2024-04-26T03:39:26.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8700000000000005e-07 2024-04-26T03:39:26.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:26.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499962+0000", 2024-04-26T03:39:26.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8389999999999997e-06 2024-04-26T03:39:26.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:26.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499977+0000", 2024-04-26T03:39:26.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5126999999999999e-05 2024-04-26T03:39:26.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:26.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499986+0000", 2024-04-26T03:39:26.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9439999999999994e-06 2024-04-26T03:39:26.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519421+0000", 2024-04-26T03:39:26.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019434297999999999 2024-04-26T03:39:26.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:26.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519439+0000", 2024-04-26T03:39:26.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8110000000000001e-05 2024-04-26T03:39:26.950 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.950 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:26.950 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.950 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.cs0 35/32 ECSubReadReply(tid=5, attrs_read=0))", 2024-04-26T03:39:26.951 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.500849+0000", 2024-04-26T03:39:26.951 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.408812671, 2024-04-26T03:39:26.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019810352, 2024-04-26T03:39:26.951 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:26.951 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:26.951 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:26.952 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:26.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:26.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500849+0000", 2024-04-26T03:39:26.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:26.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500849+0000", 2024-04-26T03:39:26.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:26.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500849+0000", 2024-04-26T03:39:26.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9999999999999999e-07 2024-04-26T03:39:26.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:26.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500860+0000", 2024-04-26T03:39:26.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0811e-05 2024-04-26T03:39:26.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:26.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500867+0000", 2024-04-26T03:39:26.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9670000000000004e-06 2024-04-26T03:39:26.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:26.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500871+0000", 2024-04-26T03:39:26.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3109999999999999e-06 2024-04-26T03:39:26.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.520619+0000", 2024-04-26T03:39:26.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019747602 2024-04-26T03:39:26.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:26.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.520659+0000", 2024-04-26T03:39:26.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0361000000000002e-05 2024-04-26T03:39:26.956 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.956 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:26.956 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.957 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:604 3.13s0 3:cad73b03:::smithi09235267-37:head [copy-from ver 50] snapc d=[d,c,b,a] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-26T03:39:26.957 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:11.710211+0000", 2024-04-26T03:39:26.957 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.199450616, 2024-04-26T03:39:26.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.082214682999999997, 2024-04-26T03:39:26.957 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:26.957 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:26.957 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:26.957 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:26.957 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:26.957 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:26.957 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 604 2024-04-26T03:39:26.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.957 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:26.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:26.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.710211+0000", 2024-04-26T03:39:26.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:26.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.710211+0000", 2024-04-26T03:39:26.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:26.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.710211+0000", 2024-04-26T03:39:26.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9100000000000004e-07 2024-04-26T03:39:26.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:26.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.710218+0000", 2024-04-26T03:39:26.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6880000000000002e-06 2024-04-26T03:39:26.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:26.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.710225+0000", 2024-04-26T03:39:26.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2019999999999998e-06 2024-04-26T03:39:26.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:26.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.710232+0000", 2024-04-26T03:39:26.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8059999999999997e-06 2024-04-26T03:39:26.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.712246+0000", 2024-04-26T03:39:26.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020136189999999999 2024-04-26T03:39:26.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-26T03:39:26.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.712248+0000", 2024-04-26T03:39:26.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1179999999999999e-06 2024-04-26T03:39:26.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.743111+0000", 2024-04-26T03:39:26.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.030862865999999999 2024-04-26T03:39:26.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for missing object", 2024-04-26T03:39:26.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.743951+0000", 2024-04-26T03:39:26.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00084010900000000002 2024-04-26T03:39:26.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.759857+0000", 2024-04-26T03:39:26.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015906699 2024-04-26T03:39:26.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:26.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.760063+0000", 2024-04-26T03:39:26.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020573299999999999 2024-04-26T03:39:26.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:26.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.774271+0000", 2024-04-26T03:39:26.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014208346 2024-04-26T03:39:26.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:26.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.790920+0000", 2024-04-26T03:39:26.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016649048 2024-04-26T03:39:26.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:26.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.791986+0000", 2024-04-26T03:39:26.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010652299999999999 2024-04-26T03:39:26.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:26.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.792425+0000", 2024-04-26T03:39:26.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00043972800000000002 2024-04-26T03:39:26.965 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.965 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:26.965 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.966 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:605 3.13s0 3:cad73b03:::smithi09235267-37:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-04-26T03:39:26.966 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:11.710239+0000", 2024-04-26T03:39:26.966 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.199421961000001, 2024-04-26T03:39:26.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080074244000000003, 2024-04-26T03:39:26.966 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:26.966 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:26.967 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:26.967 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:26.967 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:26.967 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:26.967 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 605 2024-04-26T03:39:26.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.968 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:26.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:26.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.710239+0000", 2024-04-26T03:39:26.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:26.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.710239+0000", 2024-04-26T03:39:26.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:26.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.710240+0000", 2024-04-26T03:39:26.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.32e-07 2024-04-26T03:39:26.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:26.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.710243+0000", 2024-04-26T03:39:26.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5319999999999998e-06 2024-04-26T03:39:26.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:26.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.710250+0000", 2024-04-26T03:39:26.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5889999999999999e-06 2024-04-26T03:39:26.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:26.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.710256+0000", 2024-04-26T03:39:26.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5670000000000002e-06 2024-04-26T03:39:26.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.712312+0000", 2024-04-26T03:39:26.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020565190000000001 2024-04-26T03:39:26.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-26T03:39:26.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.712314+0000", 2024-04-26T03:39:26.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1880000000000001e-06 2024-04-26T03:39:26.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.746793+0000", 2024-04-26T03:39:26.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034479224000000003 2024-04-26T03:39:26.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for missing object", 2024-04-26T03:39:26.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.746907+0000", 2024-04-26T03:39:26.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113739 2024-04-26T03:39:26.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.762064+0000", 2024-04-26T03:39:26.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015157096 2024-04-26T03:39:26.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for blocked object", 2024-04-26T03:39:26.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.762162+0000", 2024-04-26T03:39:26.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8133000000000004e-05 2024-04-26T03:39:26.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.789483+0000", 2024-04-26T03:39:26.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027321055 2024-04-26T03:39:26.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:26.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.789695+0000", 2024-04-26T03:39:26.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021135500000000001 2024-04-26T03:39:26.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:26.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.790314+0000", 2024-04-26T03:39:26.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00061881499999999999 2024-04-26T03:39:26.977 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.977 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:26.977 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.977 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:619 3.13s0 3:c944ba3d:::smithi09235267-6:a [read 0~4669440,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-04-26T03:39:26.977 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:11.732770+0000", 2024-04-26T03:39:26.977 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.176891611, 2024-04-26T03:39:26.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.12874891899999999, 2024-04-26T03:39:26.977 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:26.977 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:26.977 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:26.977 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:26.977 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:26.978 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:26.978 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 619 2024-04-26T03:39:26.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.978 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:26.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:26.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732770+0000", 2024-04-26T03:39:26.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:26.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732770+0000", 2024-04-26T03:39:26.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:26.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732772+0000", 2024-04-26T03:39:26.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.266e-06 2024-04-26T03:39:26.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:26.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732781+0000", 2024-04-26T03:39:26.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0350000000000007e-06 2024-04-26T03:39:26.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:26.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732803+0000", 2024-04-26T03:39:26.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2140000000000001e-05 2024-04-26T03:39:26.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:26.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732815+0000", 2024-04-26T03:39:26.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1637000000000001e-05 2024-04-26T03:39:26.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732928+0000", 2024-04-26T03:39:26.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000112717 2024-04-26T03:39:26.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-26T03:39:26.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732936+0000", 2024-04-26T03:39:26.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5439999999999999e-06 2024-04-26T03:39:26.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.746980+0000", 2024-04-26T03:39:26.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014043959999999999 2024-04-26T03:39:26.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for missing object", 2024-04-26T03:39:26.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.748272+0000", 2024-04-26T03:39:26.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0012919139999999999 2024-04-26T03:39:26.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.852799+0000", 2024-04-26T03:39:26.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.104527289 2024-04-26T03:39:26.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:26.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.852915+0000", 2024-04-26T03:39:26.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115395 2024-04-26T03:39:26.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:26.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.861519+0000", 2024-04-26T03:39:26.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0086040219999999994 2024-04-26T03:39:26.983 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.983 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:26.983 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.983 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:620 3.13s0 3:c944ba3d:::smithi09235267-6:a [read 0~4669440,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-04-26T03:39:26.983 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:11.732848+0000", 2024-04-26T03:39:26.983 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.176813201, 2024-04-26T03:39:26.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.13189979199999999, 2024-04-26T03:39:26.983 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:26.984 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:26.984 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:26.984 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:26.984 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:26.984 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:26.984 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 620 2024-04-26T03:39:26.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.984 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:26.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:26.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732848+0000", 2024-04-26T03:39:26.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:26.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732848+0000", 2024-04-26T03:39:26.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:26.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732849+0000", 2024-04-26T03:39:26.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9800000000000003e-07 2024-04-26T03:39:26.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:26.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732860+0000", 2024-04-26T03:39:26.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0942e-05 2024-04-26T03:39:26.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:26.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732869+0000", 2024-04-26T03:39:26.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1800000000000002e-06 2024-04-26T03:39:26.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:26.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732880+0000", 2024-04-26T03:39:26.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1021e-05 2024-04-26T03:39:26.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732987+0000", 2024-04-26T03:39:26.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107095 2024-04-26T03:39:26.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-26T03:39:26.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732989+0000", 2024-04-26T03:39:26.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1919999999999999e-06 2024-04-26T03:39:26.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.748328+0000", 2024-04-26T03:39:26.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015338312 2024-04-26T03:39:26.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for missing object", 2024-04-26T03:39:26.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.748430+0000", 2024-04-26T03:39:26.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102669 2024-04-26T03:39:26.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:26.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.853182+0000", 2024-04-26T03:39:26.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10475184799999999 2024-04-26T03:39:26.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:26.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.853309+0000", 2024-04-26T03:39:26.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012699099999999999 2024-04-26T03:39:26.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:26.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.864748+0000", 2024-04-26T03:39:26.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011438844 2024-04-26T03:39:26.995 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.995 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:26.995 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:26.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.995 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:621 3.13s0 3:c944ba3d:::smithi09235267-6:a [read 0~4669440,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-04-26T03:39:26.995 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:11.732920+0000", 2024-04-26T03:39:26.996 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.176741164999999, 2024-04-26T03:39:26.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.134463005, 2024-04-26T03:39:26.996 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:26.996 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:26.996 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:26.996 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:26.996 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:26.996 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:26.997 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 621 2024-04-26T03:39:26.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.997 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:26.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:26.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732920+0000", 2024-04-26T03:39:26.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:26.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732920+0000", 2024-04-26T03:39:26.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:26.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:26.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732921+0000", 2024-04-26T03:39:26.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5400000000000001e-07 2024-04-26T03:39:26.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:26.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732925+0000", 2024-04-26T03:39:26.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0180000000000003e-06 2024-04-26T03:39:26.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:26.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:26.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:26.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732938+0000", 2024-04-26T03:39:27.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2928999999999999e-05 2024-04-26T03:39:27.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:27.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.733001+0000", 2024-04-26T03:39:27.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3621999999999995e-05 2024-04-26T03:39:27.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:27.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.733061+0000", 2024-04-26T03:39:27.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9905000000000003e-05 2024-04-26T03:39:27.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-26T03:39:27.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.733063+0000", 2024-04-26T03:39:27.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3990000000000001e-06 2024-04-26T03:39:27.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:27.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.749893+0000", 2024-04-26T03:39:27.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016830254999999999 2024-04-26T03:39:27.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for missing object", 2024-04-26T03:39:27.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.750023+0000", 2024-04-26T03:39:27.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000130264 2024-04-26T03:39:27.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:27.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.853768+0000", 2024-04-26T03:39:27.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.103745171 2024-04-26T03:39:27.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:27.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.853937+0000", 2024-04-26T03:39:27.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016834 2024-04-26T03:39:27.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:27.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.867383+0000", 2024-04-26T03:39:27.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013446448 2024-04-26T03:39:27.003 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.003 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:27.003 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.004 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDPGPushReply(3.13s0 56/55 [PushReplyOp(3:c944ba3d:::smithi09235267-6:head)])", 2024-04-26T03:39:27.004 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:11.768451+0000", 2024-04-26T03:39:27.004 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.141210824, 2024-04-26T03:39:27.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.023073083000000001, 2024-04-26T03:39:27.004 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:27.004 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:27.004 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:27.004 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:27.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:27.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.768451+0000", 2024-04-26T03:39:27.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:27.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.768451+0000", 2024-04-26T03:39:27.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:27.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.768451+0000", 2024-04-26T03:39:27.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7400000000000002e-07 2024-04-26T03:39:27.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:27.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.768460+0000", 2024-04-26T03:39:27.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0450000000000006e-06 2024-04-26T03:39:27.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:27.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.768471+0000", 2024-04-26T03:39:27.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0776e-05 2024-04-26T03:39:27.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:27.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.768482+0000", 2024-04-26T03:39:27.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1131999999999999e-05 2024-04-26T03:39:27.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:27.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.791177+0000", 2024-04-26T03:39:27.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022694269 2024-04-26T03:39:27.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:27.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.791524+0000", 2024-04-26T03:39:27.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00034708699999999999 2024-04-26T03:39:27.007 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.007 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:27.007 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.007 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDPGPushReply(3.13s0 56/55 [PushReplyOp(3:c944ba3d:::smithi09235267-6:head)])", 2024-04-26T03:39:27.007 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:11.768915+0000", 2024-04-26T03:39:27.008 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.140746729, 2024-04-26T03:39:27.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022008744, 2024-04-26T03:39:27.008 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:27.008 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:27.008 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:27.008 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:27.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:27.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.768915+0000", 2024-04-26T03:39:27.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:27.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.768915+0000", 2024-04-26T03:39:27.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:27.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.768915+0000", 2024-04-26T03:39:27.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0600000000000001e-07 2024-04-26T03:39:27.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:27.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.768920+0000", 2024-04-26T03:39:27.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9740000000000001e-06 2024-04-26T03:39:27.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:27.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.768928+0000", 2024-04-26T03:39:27.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2609999999999993e-06 2024-04-26T03:39:27.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:27.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.768941+0000", 2024-04-26T03:39:27.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2480000000000001e-05 2024-04-26T03:39:27.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:27.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.790794+0000", 2024-04-26T03:39:27.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021853289000000001 2024-04-26T03:39:27.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:27.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.790923+0000", 2024-04-26T03:39:27.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012933399999999999 2024-04-26T03:39:27.011 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.011 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:27.011 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.011 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDPGPushReply(3.13s0 56/55 [PushReplyOp(3:c944ba3d:::smithi09235267-6:head)])", 2024-04-26T03:39:27.011 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:11.769116+0000", 2024-04-26T03:39:27.011 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.140545752, 2024-04-26T03:39:27.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021803037000000001, 2024-04-26T03:39:27.011 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:27.011 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:27.011 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:27.012 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:27.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:27.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769116+0000", 2024-04-26T03:39:27.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:27.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769116+0000", 2024-04-26T03:39:27.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:27.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769116+0000", 2024-04-26T03:39:27.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9400000000000005e-07 2024-04-26T03:39:27.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:27.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769120+0000", 2024-04-26T03:39:27.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0740000000000003e-06 2024-04-26T03:39:27.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:27.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769136+0000", 2024-04-26T03:39:27.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5532e-05 2024-04-26T03:39:27.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:27.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769144+0000", 2024-04-26T03:39:27.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7549999999999994e-06 2024-04-26T03:39:27.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:27.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.790869+0000", 2024-04-26T03:39:27.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021725661 2024-04-26T03:39:27.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:27.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.790919+0000", 2024-04-26T03:39:27.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9320999999999998e-05 2024-04-26T03:39:27.015 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.015 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:27.015 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.015 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDPGPushReply(3.13s0 56/55 [PushReplyOp(3:c944ba3d:::smithi09235267-6:head)])", 2024-04-26T03:39:27.015 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:11.769260+0000", 2024-04-26T03:39:27.015 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.140401417, 2024-04-26T03:39:27.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021509337999999999, 2024-04-26T03:39:27.015 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:27.015 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:27.015 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:27.015 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:27.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:27.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769260+0000", 2024-04-26T03:39:27.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:27.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769260+0000", 2024-04-26T03:39:27.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:27.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769260+0000", 2024-04-26T03:39:27.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9200000000000002e-07 2024-04-26T03:39:27.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:27.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769266+0000", 2024-04-26T03:39:27.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2024-04-26T03:39:27.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:27.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769274+0000", 2024-04-26T03:39:27.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3289999999999996e-06 2024-04-26T03:39:27.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:27.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769279+0000", 2024-04-26T03:39:27.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3600000000000004e-06 2024-04-26T03:39:27.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:27.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.790713+0000", 2024-04-26T03:39:27.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021433612000000001 2024-04-26T03:39:27.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:27.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.790769+0000", 2024-04-26T03:39:27.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6536999999999998e-05 2024-04-26T03:39:27.018 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.018 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:27.018 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.019 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDPGPushReply(3.13s0 56/55 [PushReplyOp(3:c944ba3d:::smithi09235267-6:head)])", 2024-04-26T03:39:27.019 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:11.769267+0000", 2024-04-26T03:39:27.019 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.14039461, 2024-04-26T03:39:27.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021747525, 2024-04-26T03:39:27.019 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:27.019 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:27.019 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:27.019 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:27.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:27.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769267+0000", 2024-04-26T03:39:27.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:27.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769267+0000", 2024-04-26T03:39:27.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:27.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769268+0000", 2024-04-26T03:39:27.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9400000000000004e-07 2024-04-26T03:39:27.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:27.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769271+0000", 2024-04-26T03:39:27.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5090000000000002e-06 2024-04-26T03:39:27.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:27.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769285+0000", 2024-04-26T03:39:27.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3566e-05 2024-04-26T03:39:27.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:27.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769290+0000", 2024-04-26T03:39:27.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8420000000000001e-06 2024-04-26T03:39:27.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:27.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.790968+0000", 2024-04-26T03:39:27.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021677923000000002 2024-04-26T03:39:27.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:27.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.791014+0000", 2024-04-26T03:39:27.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5890999999999998e-05 2024-04-26T03:39:27.022 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.022 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:27.022 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.023 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:710 3.13s0 3:c8283ddb:::smithi09235267-32:head [append 0~573440] snapc f=[f,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-04-26T03:39:27.023 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:15.970859+0000", 2024-04-26T03:39:27.023 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.938802829, 2024-04-26T03:39:27.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019548756, 2024-04-26T03:39:27.023 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:27.023 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:27.023 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:27.023 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:27.023 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:27.023 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:27.023 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 710 2024-04-26T03:39:27.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.024 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:27.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:27.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.970859+0000", 2024-04-26T03:39:27.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:27.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.970859+0000", 2024-04-26T03:39:27.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:27.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.970860+0000", 2024-04-26T03:39:27.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9649999999999998e-06 2024-04-26T03:39:27.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:27.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.971054+0000", 2024-04-26T03:39:27.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000193413 2024-04-26T03:39:27.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:27.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.971071+0000", 2024-04-26T03:39:27.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.694e-05 2024-04-26T03:39:27.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:27.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.971080+0000", 2024-04-26T03:39:27.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.414e-06 2024-04-26T03:39:27.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:27.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.971155+0000", 2024-04-26T03:39:27.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4482999999999998e-05 2024-04-26T03:39:27.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:27.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.971280+0000", 2024-04-26T03:39:27.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012498899999999999 2024-04-26T03:39:27.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:27.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.971925+0000", 2024-04-26T03:39:27.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000645003 2024-04-26T03:39:27.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:27.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.982227+0000", 2024-04-26T03:39:27.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01030181 2024-04-26T03:39:27.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:27.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.990344+0000", 2024-04-26T03:39:27.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0081176319999999996 2024-04-26T03:39:27.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:27.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.990407+0000", 2024-04-26T03:39:27.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3107000000000004e-05 2024-04-26T03:39:27.028 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.028 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:27.028 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.028 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:782 3.fs0 3:f81f96ba:::smithi09235267-46:head [append 0~540672] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:27.029 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:20.821620+0000", 2024-04-26T03:39:27.029 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.0880409330000003, 2024-04-26T03:39:27.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020779720000000002, 2024-04-26T03:39:27.029 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:27.029 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:27.029 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:27.029 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:27.029 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:27.029 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:27.029 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 782 2024-04-26T03:39:27.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.029 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:27.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:27.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.821620+0000", 2024-04-26T03:39:27.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:27.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.821620+0000", 2024-04-26T03:39:27.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:27.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.821621+0000", 2024-04-26T03:39:27.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6200000000000006e-07 2024-04-26T03:39:27.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:27.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.821861+0000", 2024-04-26T03:39:27.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000240063 2024-04-26T03:39:27.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:27.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.821873+0000", 2024-04-26T03:39:27.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1525999999999999e-05 2024-04-26T03:39:27.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:27.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.821882+0000", 2024-04-26T03:39:27.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9290000000000004e-06 2024-04-26T03:39:27.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:27.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.828072+0000", 2024-04-26T03:39:27.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061898659999999996 2024-04-26T03:39:27.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:27.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.828171+0000", 2024-04-26T03:39:27.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9437000000000004e-05 2024-04-26T03:39:27.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:27.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.828799+0000", 2024-04-26T03:39:27.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00062785900000000003 2024-04-26T03:39:27.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:27.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.841267+0000", 2024-04-26T03:39:27.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012467914 2024-04-26T03:39:27.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:27.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.842350+0000", 2024-04-26T03:39:27.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010830099999999999 2024-04-26T03:39:27.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:27.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.842400+0000", 2024-04-26T03:39:27.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0154000000000001e-05 2024-04-26T03:39:27.034 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.034 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:27.034 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.034 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.034 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:27.034 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:27.035 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-04-26T03:39:27.036 INFO:tasks.ceph.osd.4.smithi028.stderr:2024-04-26T03:39:27.034+0000 7fd9511b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31963) UID: 0 2024-04-26T03:39:27.136 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:39:27.135+0000 7fba0d2ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31961) UID: 0 2024-04-26T03:39:27.139 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:27.139 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-26T03:39:27.139 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-26T03:39:27.139 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:27.147 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-04-26T03:39:27.237 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:39:27.236+0000 7fb78c544640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31956) UID: 0 2024-04-26T03:39:27.249 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:27.249 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-26T03:39:27.249 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-26T03:39:27.250 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-26T03:39:27.250 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:27.258 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-04-26T03:39:27.339 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:39:27.338+0000 7f60d3c4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31757) UID: 0 2024-04-26T03:39:27.361 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:27.361 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-26T03:39:27.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-26T03:39:27.362 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-26T03:39:27.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.362 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:30 3.5s0 3:a64abe59:::smithi09235267-10:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:27.362 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.491045+0000", 2024-04-26T03:39:27.362 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.868010188, 2024-04-26T03:39:27.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.024868352, 2024-04-26T03:39:27.362 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:27.362 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:27.362 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:27.362 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:27.362 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:27.362 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:27.362 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 30 2024-04-26T03:39:27.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.363 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:27.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:27.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.491045+0000", 2024-04-26T03:39:27.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:27.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.491045+0000", 2024-04-26T03:39:27.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:27.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.491045+0000", 2024-04-26T03:39:27.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3799999999999997e-07 2024-04-26T03:39:27.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:27.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.491049+0000", 2024-04-26T03:39:27.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3359999999999999e-06 2024-04-26T03:39:27.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:27.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.491056+0000", 2024-04-26T03:39:27.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7549999999999994e-06 2024-04-26T03:39:27.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:27.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.491062+0000", 2024-04-26T03:39:27.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3730000000000001e-06 2024-04-26T03:39:27.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:27.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.493431+0000", 2024-04-26T03:39:27.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023688490000000001 2024-04-26T03:39:27.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:27.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.493515+0000", 2024-04-26T03:39:27.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4342e-05 2024-04-26T03:39:27.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:27.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499490+0000", 2024-04-26T03:39:27.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059748809999999996 2024-04-26T03:39:27.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:27.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499562+0000", 2024-04-26T03:39:27.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1942999999999999e-05 2024-04-26T03:39:27.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:27.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515913+0000", 2024-04-26T03:39:27.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016351335000000002 2024-04-26T03:39:27.366 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.366 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:27.367 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.367 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.5s0 35/32 ECSubReadReply(tid=3, attrs_read=0))", 2024-04-26T03:39:27.367 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.500548+0000", 2024-04-26T03:39:27.367 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.858506980000001, 2024-04-26T03:39:27.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014243212999999999, 2024-04-26T03:39:27.367 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:27.367 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:27.367 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:27.367 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:27.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:27.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500548+0000", 2024-04-26T03:39:27.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500548+0000", 2024-04-26T03:39:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500549+0000", 2024-04-26T03:39:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3300000000000002e-07 2024-04-26T03:39:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500560+0000", 2024-04-26T03:39:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1316e-05 2024-04-26T03:39:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500573+0000", 2024-04-26T03:39:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2853999999999999e-05 2024-04-26T03:39:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500579+0000", 2024-04-26T03:39:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3090000000000001e-06 2024-04-26T03:39:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.514618+0000", 2024-04-26T03:39:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014038861 2024-04-26T03:39:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.514791+0000", 2024-04-26T03:39:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017323999999999999 2024-04-26T03:39:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.5s0 35/32 ECSubReadReply(tid=3, attrs_read=0))", 2024-04-26T03:39:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.500609+0000", 2024-04-26T03:39:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.858445828999997, 2024-04-26T03:39:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014706643, 2024-04-26T03:39:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500609+0000", 2024-04-26T03:39:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500609+0000", 2024-04-26T03:39:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500610+0000", 2024-04-26T03:39:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1900000000000002e-07 2024-04-26T03:39:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500621+0000", 2024-04-26T03:39:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1102e-05 2024-04-26T03:39:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500631+0000", 2024-04-26T03:39:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0197000000000001e-05 2024-04-26T03:39:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500638+0000", 2024-04-26T03:39:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3770000000000001e-06 2024-04-26T03:39:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515285+0000", 2024-04-26T03:39:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014647004 2024-04-26T03:39:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515316+0000", 2024-04-26T03:39:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1244000000000001e-05 2024-04-26T03:39:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 35/32 ECSubWriteReply(tid=2, last_complete=34'2, committed=1, applied=1))", 2024-04-26T03:39:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.501067+0000", 2024-04-26T03:39:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.857988380999998, 2024-04-26T03:39:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013961199000000001, 2024-04-26T03:39:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501067+0000", 2024-04-26T03:39:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501067+0000", 2024-04-26T03:39:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501067+0000", 2024-04-26T03:39:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7599999999999996e-07 2024-04-26T03:39:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501072+0000", 2024-04-26T03:39:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7700000000000001e-06 2024-04-26T03:39:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501083+0000", 2024-04-26T03:39:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0652000000000001e-05 2024-04-26T03:39:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:27.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501090+0000", 2024-04-26T03:39:27.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5299999999999999e-06 2024-04-26T03:39:27.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:27.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515013+0000", 2024-04-26T03:39:27.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013922436999999999 2024-04-26T03:39:27.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:27.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515028+0000", 2024-04-26T03:39:27.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5034e-05 2024-04-26T03:39:27.377 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.5s0 35/32 ECSubReadReply(tid=3, attrs_read=0))", 2024-04-26T03:39:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.501907+0000", 2024-04-26T03:39:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.857147564000002, 2024-04-26T03:39:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013462257, 2024-04-26T03:39:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501907+0000", 2024-04-26T03:39:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501907+0000", 2024-04-26T03:39:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501908+0000", 2024-04-26T03:39:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2200000000000003e-07 2024-04-26T03:39:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501918+0000", 2024-04-26T03:39:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0329999999999999e-05 2024-04-26T03:39:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501945+0000", 2024-04-26T03:39:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6047999999999999e-05 2024-04-26T03:39:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501951+0000", 2024-04-26T03:39:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7560000000000002e-06 2024-04-26T03:39:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515339+0000", 2024-04-26T03:39:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013387731 2024-04-26T03:39:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515370+0000", 2024-04-26T03:39:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0670000000000003e-05 2024-04-26T03:39:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 35/32 ECSubWriteReply(tid=2, last_complete=34'2, committed=1, applied=1))", 2024-04-26T03:39:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.501982+0000", 2024-04-26T03:39:27.382 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.857072727999999, 2024-04-26T03:39:27.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013281477999999999, 2024-04-26T03:39:27.382 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:27.382 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:27.382 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:27.382 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:27.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:27.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501982+0000", 2024-04-26T03:39:27.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:27.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501982+0000", 2024-04-26T03:39:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501983+0000", 2024-04-26T03:39:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4900000000000001e-07 2024-04-26T03:39:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501987+0000", 2024-04-26T03:39:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1030000000000004e-06 2024-04-26T03:39:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:27.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501995+0000", 2024-04-26T03:39:27.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4859999999999997e-06 2024-04-26T03:39:27.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:27.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.502002+0000", 2024-04-26T03:39:27.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3400000000000003e-06 2024-04-26T03:39:27.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:27.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515231+0000", 2024-04-26T03:39:27.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013229757 2024-04-26T03:39:27.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:27.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515264+0000", 2024-04-26T03:39:27.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2443000000000001e-05 2024-04-26T03:39:27.385 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.385 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:27.385 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.385 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:67 3.5s0 3:a79a63ac:::smithi09235267-23:head [append 0~376832,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:27.385 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.518499+0000", 2024-04-26T03:39:27.385 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.840555686999998, 2024-04-26T03:39:27.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01578007, 2024-04-26T03:39:27.385 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:27.385 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 67 2024-04-26T03:39:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518499+0000", 2024-04-26T03:39:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518499+0000", 2024-04-26T03:39:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518501+0000", 2024-04-26T03:39:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9010000000000001e-06 2024-04-26T03:39:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518725+0000", 2024-04-26T03:39:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000224183 2024-04-26T03:39:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518740+0000", 2024-04-26T03:39:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4966e-05 2024-04-26T03:39:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518748+0000", 2024-04-26T03:39:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6399999999999997e-06 2024-04-26T03:39:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518798+0000", 2024-04-26T03:39:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0263000000000003e-05 2024-04-26T03:39:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518927+0000", 2024-04-26T03:39:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012898900000000001 2024-04-26T03:39:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519605+0000", 2024-04-26T03:39:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00067774899999999995 2024-04-26T03:39:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.524447+0000", 2024-04-26T03:39:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0048424319999999998 2024-04-26T03:39:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:27.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.534195+0000", 2024-04-26T03:39:27.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097473149999999995 2024-04-26T03:39:27.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:27.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.534279+0000", 2024-04-26T03:39:27.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4631999999999999e-05 2024-04-26T03:39:27.390 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.390 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:27.390 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.391 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:68 3.5s0 3:a79a63ac:::smithi09235267-23:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:27.391 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.518766+0000", 2024-04-26T03:39:27.391 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.840288766, 2024-04-26T03:39:27.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015709772, 2024-04-26T03:39:27.391 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:27.391 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:27.391 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:27.391 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:27.391 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:27.391 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:27.391 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 68 2024-04-26T03:39:27.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.391 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:27.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518766+0000", 2024-04-26T03:39:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518766+0000", 2024-04-26T03:39:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518767+0000", 2024-04-26T03:39:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6899999999999997e-07 2024-04-26T03:39:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518771+0000", 2024-04-26T03:39:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-06 2024-04-26T03:39:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518777+0000", 2024-04-26T03:39:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3600000000000004e-06 2024-04-26T03:39:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518783+0000", 2024-04-26T03:39:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9490000000000001e-06 2024-04-26T03:39:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.522888+0000", 2024-04-26T03:39:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004105577 2024-04-26T03:39:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.522976+0000", 2024-04-26T03:39:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400999999999999e-05 2024-04-26T03:39:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.523380+0000", 2024-04-26T03:39:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00040403599999999998 2024-04-26T03:39:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.524449+0000", 2024-04-26T03:39:27.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00106897 2024-04-26T03:39:27.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:27.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.534390+0000", 2024-04-26T03:39:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099415249999999997 2024-04-26T03:39:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.534476+0000", 2024-04-26T03:39:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5884999999999995e-05 2024-04-26T03:39:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:69 3.5s0 3:a79a63ac:::smithi09235267-23:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.518790+0000", 2024-04-26T03:39:27.399 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.840264533999999, 2024-04-26T03:39:27.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018453847999999998, 2024-04-26T03:39:27.399 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:27.399 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:27.399 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:27.399 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:27.399 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:27.399 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:27.399 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 69 2024-04-26T03:39:27.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.399 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:27.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:27.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518790+0000", 2024-04-26T03:39:27.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:27.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518790+0000", 2024-04-26T03:39:27.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:27.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518791+0000", 2024-04-26T03:39:27.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9999999999999998e-07 2024-04-26T03:39:27.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:27.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518793+0000", 2024-04-26T03:39:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0949999999999998e-06 2024-04-26T03:39:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518799+0000", 2024-04-26T03:39:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.117e-06 2024-04-26T03:39:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518804+0000", 2024-04-26T03:39:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8670000000000002e-06 2024-04-26T03:39:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.523575+0000", 2024-04-26T03:39:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004771155 2024-04-26T03:39:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.523646+0000", 2024-04-26T03:39:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0992000000000001e-05 2024-04-26T03:39:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:27.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.534812+0000", 2024-04-26T03:39:27.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011166099000000001 2024-04-26T03:39:27.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:27.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.534887+0000", 2024-04-26T03:39:27.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5094999999999998e-05 2024-04-26T03:39:27.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:27.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.537244+0000", 2024-04-26T03:39:27.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023569279999999999 2024-04-26T03:39:27.405 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.405 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:27.405 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.405 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:256 3.5s0 3:a64abe59:::smithi09235267-10:head [append 0~0] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T03:39:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:56.508641+0000", 2024-04-26T03:39:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.850414062999999, 2024-04-26T03:39:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013570576000000001, 2024-04-26T03:39:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 256 2024-04-26T03:39:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508641+0000", 2024-04-26T03:39:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508641+0000", 2024-04-26T03:39:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508644+0000", 2024-04-26T03:39:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7760000000000002e-06 2024-04-26T03:39:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508655+0000", 2024-04-26T03:39:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1049e-05 2024-04-26T03:39:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508727+0000", 2024-04-26T03:39:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1907000000000001e-05 2024-04-26T03:39:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508737+0000", 2024-04-26T03:39:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0691e-05 2024-04-26T03:39:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508889+0000", 2024-04-26T03:39:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015123099999999999 2024-04-26T03:39:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-04-26T03:39:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508905+0000", 2024-04-26T03:39:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6042999999999999e-05 2024-04-26T03:39:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.511938+0000", 2024-04-26T03:39:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030333119999999998 2024-04-26T03:39:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.512054+0000", 2024-04-26T03:39:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115742 2024-04-26T03:39:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.512811+0000", 2024-04-26T03:39:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00075763100000000004 2024-04-26T03:39:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.521838+0000", 2024-04-26T03:39:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0090266489999999994 2024-04-26T03:39:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.522144+0000", 2024-04-26T03:39:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030574600000000002 2024-04-26T03:39:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.522211+0000", 2024-04-26T03:39:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7799000000000006e-05 2024-04-26T03:39:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:257 3.5s0 3:a64abe59:::smithi09235267-10:head [rollback 2] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T03:39:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:56.508783+0000", 2024-04-26T03:39:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.850272043, 2024-04-26T03:39:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014341474999999999, 2024-04-26T03:39:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 257 2024-04-26T03:39:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508783+0000", 2024-04-26T03:39:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508783+0000", 2024-04-26T03:39:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508784+0000", 2024-04-26T03:39:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9999999999999995e-07 2024-04-26T03:39:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508789+0000", 2024-04-26T03:39:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4280000000000004e-06 2024-04-26T03:39:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:27.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508800+0000", 2024-04-26T03:39:27.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.076e-05 2024-04-26T03:39:27.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:27.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508808+0000", 2024-04-26T03:39:27.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.593e-06 2024-04-26T03:39:27.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:27.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508993+0000", 2024-04-26T03:39:27.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018527999999999999 2024-04-26T03:39:27.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-04-26T03:39:27.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.509001+0000", 2024-04-26T03:39:27.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6529999999999994e-06 2024-04-26T03:39:27.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:27.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.513192+0000", 2024-04-26T03:39:27.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041913009999999997 2024-04-26T03:39:27.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:27.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.513348+0000", 2024-04-26T03:39:27.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000155654 2024-04-26T03:39:27.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.514509+0000", 2024-04-26T03:39:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011615169999999999 2024-04-26T03:39:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.521845+0000", 2024-04-26T03:39:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0073361140000000003 2024-04-26T03:39:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.523044+0000", 2024-04-26T03:39:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001198939 2024-04-26T03:39:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.523124+0000", 2024-04-26T03:39:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0235999999999995e-05 2024-04-26T03:39:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:258 3.5s0 3:a64abe59:::smithi09235267-10:head [append 0~0] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T03:39:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:56.508917+0000", 2024-04-26T03:39:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.850138199, 2024-04-26T03:39:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014562304999999999, 2024-04-26T03:39:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 258 2024-04-26T03:39:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508917+0000", 2024-04-26T03:39:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508917+0000", 2024-04-26T03:39:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508918+0000", 2024-04-26T03:39:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5600000000000004e-07 2024-04-26T03:39:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508922+0000", 2024-04-26T03:39:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4730000000000002e-06 2024-04-26T03:39:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508937+0000", 2024-04-26T03:39:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5114e-05 2024-04-26T03:39:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508948+0000", 2024-04-26T03:39:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1222e-05 2024-04-26T03:39:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.509035+0000", 2024-04-26T03:39:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7032000000000003e-05 2024-04-26T03:39:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-04-26T03:39:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.509040+0000", 2024-04-26T03:39:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6299999999999997e-06 2024-04-26T03:39:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.515348+0000", 2024-04-26T03:39:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0063082959999999997 2024-04-26T03:39:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.515613+0000", 2024-04-26T03:39:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026497100000000002 2024-04-26T03:39:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.516295+0000", 2024-04-26T03:39:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00068196499999999998 2024-04-26T03:39:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.521977+0000", 2024-04-26T03:39:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056815759999999998 2024-04-26T03:39:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.523407+0000", 2024-04-26T03:39:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014299409999999999 2024-04-26T03:39:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.523479+0000", 2024-04-26T03:39:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2229000000000003e-05 2024-04-26T03:39:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:318 3.5s0 3:a79a63ac:::smithi09235267-23:head [delete] snapc 7=[7,6,5] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e44)", 2024-04-26T03:39:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:00.528087+0000", 2024-04-26T03:39:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.830967510000001, 2024-04-26T03:39:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.020004624, 2024-04-26T03:39:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:27.425 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:27.425 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:27.425 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:27.425 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:27.425 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:27.425 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 318 2024-04-26T03:39:27.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.425 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:27.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:27.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:00.528087+0000", 2024-04-26T03:39:27.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:27.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:00.528087+0000", 2024-04-26T03:39:27.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:27.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:00.528089+0000", 2024-04-26T03:39:27.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9120000000000001e-06 2024-04-26T03:39:27.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:27.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:00.528096+0000", 2024-04-26T03:39:27.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0829999999999998e-06 2024-04-26T03:39:27.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:27.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:00.528112+0000", 2024-04-26T03:39:27.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5619000000000001e-05 2024-04-26T03:39:27.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:27.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:00.528121+0000", 2024-04-26T03:39:27.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6789999999999994e-06 2024-04-26T03:39:27.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:27.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:00.528185+0000", 2024-04-26T03:39:27.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4156e-05 2024-04-26T03:39:27.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-04-26T03:39:27.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:00.528190+0000", 2024-04-26T03:39:27.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9010000000000002e-06 2024-04-26T03:39:27.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:27.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:00.534892+0000", 2024-04-26T03:39:27.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067026239999999999 2024-04-26T03:39:27.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-26T03:39:27.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:00.534897+0000", 2024-04-26T03:39:27.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9389999999999996e-06 2024-04-26T03:39:27.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:27.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.540918+0000", 2024-04-26T03:39:27.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0060203679999999 2024-04-26T03:39:27.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:27.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.541113+0000", 2024-04-26T03:39:27.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019529200000000001 2024-04-26T03:39:27.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:27.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.541689+0000", 2024-04-26T03:39:27.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00057578299999999996 2024-04-26T03:39:27.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:27.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.544502+0000", 2024-04-26T03:39:27.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028131950000000001 2024-04-26T03:39:27.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:27.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.547870+0000", 2024-04-26T03:39:27.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0033683039999999999 2024-04-26T03:39:27.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:27.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.548092+0000", 2024-04-26T03:39:27.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000221769 2024-04-26T03:39:27.431 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.431 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:27.431 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.432 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.13s3 50/44 ECSubWrite(tid=74, reqid=client.4556.0:446, at_version=50'37, trim_to=0'0, roll_forward_to=50'36))", 2024-04-26T03:39:27.432 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:05.702348+0000", 2024-04-26T03:39:27.432 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.656706914000001, 2024-04-26T03:39:27.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013733293000000001, 2024-04-26T03:39:27.432 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:27.432 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:27.432 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:27.432 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:27.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:27.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.702348+0000", 2024-04-26T03:39:27.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:27.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.702348+0000", 2024-04-26T03:39:27.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:27.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.702349+0000", 2024-04-26T03:39:27.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5400000000000001e-07 2024-04-26T03:39:27.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:27.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.702837+0000", 2024-04-26T03:39:27.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00048857299999999996 2024-04-26T03:39:27.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:27.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.702851+0000", 2024-04-26T03:39:27.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3723999999999999e-05 2024-04-26T03:39:27.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:27.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.702862+0000", 2024-04-26T03:39:27.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1571e-05 2024-04-26T03:39:27.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:27.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.702925+0000", 2024-04-26T03:39:27.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2890000000000003e-05 2024-04-26T03:39:27.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:27.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.702942+0000", 2024-04-26T03:39:27.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6857000000000001e-05 2024-04-26T03:39:27.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:27.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716045+0000", 2024-04-26T03:39:27.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013102744 2024-04-26T03:39:27.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:27.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716081+0000", 2024-04-26T03:39:27.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6380000000000001e-05 2024-04-26T03:39:27.436 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.436 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:27.436 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.436 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:462 3.5s0 3:a64abe59:::smithi09235267-10:head [append 0~753664] snapc 8=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:27.436 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:05.749661+0000", 2024-04-26T03:39:27.436 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.609394004999999, 2024-04-26T03:39:27.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014030855, 2024-04-26T03:39:27.436 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:27.436 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:27.437 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:27.437 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:27.437 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:27.437 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:27.437 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 462 2024-04-26T03:39:27.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.437 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:27.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:27.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.749661+0000", 2024-04-26T03:39:27.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:27.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.749661+0000", 2024-04-26T03:39:27.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:27.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.749663+0000", 2024-04-26T03:39:27.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.722e-06 2024-04-26T03:39:27.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:27.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.749971+0000", 2024-04-26T03:39:27.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000308475 2024-04-26T03:39:27.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:27.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.749985+0000", 2024-04-26T03:39:27.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4191e-05 2024-04-26T03:39:27.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:27.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.749995+0000", 2024-04-26T03:39:27.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0020000000000001e-05 2024-04-26T03:39:27.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:27.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.750050+0000", 2024-04-26T03:39:27.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5000000000000002e-05 2024-04-26T03:39:27.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:27.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.750121+0000", 2024-04-26T03:39:27.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0669999999999999e-05 2024-04-26T03:39:27.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:27.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.750769+0000", 2024-04-26T03:39:27.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00064844000000000002 2024-04-26T03:39:27.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:27.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.763254+0000", 2024-04-26T03:39:27.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012484215999999999 2024-04-26T03:39:27.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:27.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.763622+0000", 2024-04-26T03:39:27.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00036791699999999999 2024-04-26T03:39:27.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:27.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.763692+0000", 2024-04-26T03:39:27.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0203999999999994e-05 2024-04-26T03:39:27.442 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.442 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:27.442 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.442 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:466 3.5s0 3:a64abe59:::smithi09235267-10:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:27.442 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:05.755663+0000", 2024-04-26T03:39:27.442 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.603391508000001, 2024-04-26T03:39:27.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021406392999999999, 2024-04-26T03:39:27.442 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:27.442 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:27.442 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:27.443 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:27.443 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:27.443 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:27.443 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 466 2024-04-26T03:39:27.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.443 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:27.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:27.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755663+0000", 2024-04-26T03:39:27.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:27.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755663+0000", 2024-04-26T03:39:27.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:27.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755664+0000", 2024-04-26T03:39:27.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6600000000000003e-07 2024-04-26T03:39:27.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:27.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755665+0000", 2024-04-26T03:39:27.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.389e-06 2024-04-26T03:39:27.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:27.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755669+0000", 2024-04-26T03:39:27.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2024-04-26T03:39:27.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:27.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755672+0000", 2024-04-26T03:39:27.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7190000000000001e-06 2024-04-26T03:39:27.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:27.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.764511+0000", 2024-04-26T03:39:27.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0088397730000000004 2024-04-26T03:39:27.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:27.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.764598+0000", 2024-04-26T03:39:27.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6438000000000001e-05 2024-04-26T03:39:27.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:27.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.766650+0000", 2024-04-26T03:39:27.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020521430000000002 2024-04-26T03:39:27.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:27.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.766698+0000", 2024-04-26T03:39:27.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8133000000000001e-05 2024-04-26T03:39:27.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:27.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.777070+0000", 2024-04-26T03:39:27.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010371672 2024-04-26T03:39:27.450 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.450 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:27.450 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.450 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:467 3.5s0 3:a79a63ac:::smithi09235267-23:head [stat,delete] snapc 8=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:27.450 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:05.755675+0000", 2024-04-26T03:39:27.450 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.603379752999999, 2024-04-26T03:39:27.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020820686000000001, 2024-04-26T03:39:27.450 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:27.450 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:27.450 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:27.451 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:27.451 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:27.451 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:27.451 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 467 2024-04-26T03:39:27.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.451 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:27.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:27.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755675+0000", 2024-04-26T03:39:27.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:27.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755675+0000", 2024-04-26T03:39:27.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:27.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755675+0000", 2024-04-26T03:39:27.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0900000000000001e-07 2024-04-26T03:39:27.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:27.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755677+0000", 2024-04-26T03:39:27.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.406e-06 2024-04-26T03:39:27.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:27.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755680+0000", 2024-04-26T03:39:27.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2329999999999998e-06 2024-04-26T03:39:27.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:27.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755684+0000", 2024-04-26T03:39:27.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.472e-06 2024-04-26T03:39:27.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:27.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.764647+0000", 2024-04-26T03:39:27.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0089629900000000005 2024-04-26T03:39:27.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:27.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.764702+0000", 2024-04-26T03:39:27.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5748000000000002e-05 2024-04-26T03:39:27.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:27.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.765004+0000", 2024-04-26T03:39:27.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030170000000000002 2024-04-26T03:39:27.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:27.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.776390+0000", 2024-04-26T03:39:27.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011386454000000001 2024-04-26T03:39:27.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:27.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.776455+0000", 2024-04-26T03:39:27.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4304e-05 2024-04-26T03:39:27.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:27.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.776496+0000", 2024-04-26T03:39:27.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1170000000000001e-05 2024-04-26T03:39:27.455 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.455 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:27.455 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.455 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:615 3.5s0 3:a79a63ac:::smithi09235267-23:head [create,setxattr Z]ca\\SkkXhsYi]s_mecUZQfPbbjaqacZrSObr_`a (734) in=40b,setxattr i (1998) in=1b,setxattr OQosgYUVOPebZdfit`Xtcc]XX_pdNqdkSof_lrck (1822) in=40b,setxattr mrQaYTllqfj^q]UYaoR`drdgY]bVfYQ\\SgOe_gca (1789) in=40b,setxattr ZaeONfchPlXaOYt[dQNtaR[NNiaSangpfQNVhVY[ (1646) in=40b,setxattr epskr[spqr^j^XqcosqPQ (1998) in=21b,setxattr WarqdipUlkSQhTbQR[NhrYRrda[QmOgj^VqWWero (313) in=40b,setxattr f\\fnrpSa`gWXj`Yo`dQY]Vdr[Z^`gkVjb^enmUig (1645) in=40b,setxattr bjWaN_RafX`the (1998) in=14b,setxattr nmdNljVWb`aR_`Oeb[l_OntR_YgidWjRabRf[]Ye (10596) in=40b,setxattr [nhisfmkVd_n_YV\\efebsW_pQaiWle_]o]rhc[bO (14456) in=40b,setxattr []hmkY]V]hkrrd]gr\\c\\hPq_smnTfakPZRNg^bY[ (1649) in=40b,setxattr ZZ\\\\jZnU`jtY\\[k]hnm`r_OYQeWTg\\rnQdX^rbhg (807) in=40b,setxattr QT_d^gX[dNZPP\\VapoetR^[gpVcZQXfhp^`l_Vh[ (66) in=40b,setxattr ee[ (1998) in=3b,setxattr RY\\sp]UifT[Zt\\[VenNWarqiiNSXsdnktf^QhT[U (487) in=40b,setxattr hUnl\\fnak]U``QN^rs`a^kfaQ`NoTVNnPXYbdmre (13680) in=40b,setxattr gha[S`afeloOcnfbeOkUV]T]ZTYWaOeaS]bbVa\\Q (1226) in=40b,setxattr NaraXbinjWQraPh`Pp]qnPWgWfZf[WaUb]oOY]^r (1611) in=40b,setxattr tpfZiddi]a^RVWpgOZormUeqYosdY_UsSpiQQRgj (1225) in=40b] snapc d=[d,c,b,a] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-26T03:39:27.455 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:11.732685+0000", 2024-04-26T03:39:27.456 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.626369645, 2024-04-26T03:39:27.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013378048999999999, 2024-04-26T03:39:27.456 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:27.456 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:27.456 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:27.456 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:27.456 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:27.456 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:27.456 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 615 2024-04-26T03:39:27.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.456 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:27.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:27.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732685+0000", 2024-04-26T03:39:27.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:27.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732685+0000", 2024-04-26T03:39:27.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:27.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732688+0000", 2024-04-26T03:39:27.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8320000000000002e-06 2024-04-26T03:39:27.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:27.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732753+0000", 2024-04-26T03:39:27.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4678999999999995e-05 2024-04-26T03:39:27.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.458 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:27.458 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732774+0000", 2024-04-26T03:39:27.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1637999999999998e-05 2024-04-26T03:39:27.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.458 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:27.458 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732784+0000", 2024-04-26T03:39:27.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5410000000000006e-06 2024-04-26T03:39:27.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:27.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732856+0000", 2024-04-26T03:39:27.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2020000000000005e-05 2024-04-26T03:39:27.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:27.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.733002+0000", 2024-04-26T03:39:27.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000145697 2024-04-26T03:39:27.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:27.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.733773+0000", 2024-04-26T03:39:27.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000770774 2024-04-26T03:39:27.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:27.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.735028+0000", 2024-04-26T03:39:27.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0012551089999999999 2024-04-26T03:39:27.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:27.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.745960+0000", 2024-04-26T03:39:27.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010932737999999999 2024-04-26T03:39:27.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:27.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.746063+0000", 2024-04-26T03:39:27.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103021 2024-04-26T03:39:27.460 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.461 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:27.461 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.461 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:818 3.5s0 3:a64abe59:::smithi09235267-10:head [read 0~2605056,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-04-26T03:39:27.461 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:22.692385+0000", 2024-04-26T03:39:27.461 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.6666703470000002, 2024-04-26T03:39:27.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018880436, 2024-04-26T03:39:27.461 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:27.461 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:27.461 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:27.461 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:27.461 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:27.462 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:27.462 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 818 2024-04-26T03:39:27.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.462 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:27.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:27.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692385+0000", 2024-04-26T03:39:27.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:27.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692385+0000", 2024-04-26T03:39:27.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:27.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692385+0000", 2024-04-26T03:39:27.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2900000000000003e-07 2024-04-26T03:39:27.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:27.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692390+0000", 2024-04-26T03:39:27.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7380000000000002e-06 2024-04-26T03:39:27.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:27.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692400+0000", 2024-04-26T03:39:27.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5880000000000004e-06 2024-04-26T03:39:27.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:27.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692406+0000", 2024-04-26T03:39:27.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5529999999999998e-06 2024-04-26T03:39:27.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:27.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692508+0000", 2024-04-26T03:39:27.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101698 2024-04-26T03:39:27.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-04-26T03:39:27.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692515+0000", 2024-04-26T03:39:27.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1629999999999999e-06 2024-04-26T03:39:27.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:27.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.697431+0000", 2024-04-26T03:39:27.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049163280000000002 2024-04-26T03:39:27.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:27.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.697602+0000", 2024-04-26T03:39:27.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017039999999999999 2024-04-26T03:39:27.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:27.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.711265+0000", 2024-04-26T03:39:27.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013663339 2024-04-26T03:39:27.466 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.466 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:27.466 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.466 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:819 3.5s0 3:a64abe59:::smithi09235267-10:head [read 0~2605056,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-04-26T03:39:27.466 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:22.692428+0000", 2024-04-26T03:39:27.466 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.6666273059999996, 2024-04-26T03:39:27.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020633177999999999, 2024-04-26T03:39:27.466 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:27.466 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:27.467 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:27.467 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:27.467 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:27.467 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:27.467 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 819 2024-04-26T03:39:27.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.467 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:27.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:27.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692428+0000", 2024-04-26T03:39:27.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:27.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692428+0000", 2024-04-26T03:39:27.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:27.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:27.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692428+0000", 2024-04-26T03:39:27.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2024-04-26T03:39:27.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:27.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692432+0000", 2024-04-26T03:39:27.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6320000000000001e-06 2024-04-26T03:39:27.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:27.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692437+0000", 2024-04-26T03:39:27.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.164e-06 2024-04-26T03:39:27.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:27.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692441+0000", 2024-04-26T03:39:27.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7459999999999998e-06 2024-04-26T03:39:27.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:27.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692537+0000", 2024-04-26T03:39:27.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6175000000000007e-05 2024-04-26T03:39:27.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-04-26T03:39:27.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692541+0000", 2024-04-26T03:39:27.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6279999999999998e-06 2024-04-26T03:39:27.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:27.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.699526+0000", 2024-04-26T03:39:27.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0069851940000000001 2024-04-26T03:39:27.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:27.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.699637+0000", 2024-04-26T03:39:27.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000111437 2024-04-26T03:39:27.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:27.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:27.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:27.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.713061+0000", 2024-04-26T03:39:27.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013423554000000001 2024-04-26T03:39:27.471 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.471 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:27.471 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.471 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:27.471 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:27.471 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:27.473 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-26T03:39:27.473 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:39:27.471+0000 7fb78c544640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31956) UID: 0 2024-04-26T03:39:27.573 INFO:tasks.ceph.osd.4.smithi028.stderr:2024-04-26T03:39:27.571+0000 7fd9511b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31963) UID: 0 2024-04-26T03:39:27.593 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:39:27.593 INFO:teuthology.orchestra.run.smithi028.stdout: "ops": [], 2024-04-26T03:39:27.593 INFO:teuthology.orchestra.run.smithi028.stdout: "num_ops": 0 2024-04-26T03:39:27.593 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:39:27.603 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-26T03:39:27.674 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-26T03:39:27.672+0000 7f610b3a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31816) UID: 0 2024-04-26T03:39:27.684 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-26T03:39:27.694 INFO:tasks.rados.rados.0.smithi092.stdout:319: done (2 left) 2024-04-26T03:39:27.694 INFO:tasks.rados.rados.0.smithi092.stdout:320: done (1 left) 2024-04-26T03:39:27.694 INFO:tasks.rados.rados.0.smithi092.stdout:322: done (0 left) 2024-04-26T03:39:27.694 INFO:tasks.rados.rados.0.smithi092.stdout:323: delete oid 26 current snap is 22 2024-04-26T03:39:27.699 INFO:tasks.rados.rados.0.smithi092.stdout:323: done (0 left) 2024-04-26T03:39:27.699 INFO:tasks.rados.rados.0.smithi092.stdout:324: read oid 25 snap -1 2024-04-26T03:39:27.699 INFO:tasks.rados.rados.0.smithi092.stdout:324: expect (ObjNum 139 snap 17 seq_num 139) 2024-04-26T03:39:27.699 INFO:tasks.rados.rados.0.smithi092.stdout:325: snap_remove snap 10 2024-04-26T03:39:27.757 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:39:27.757 INFO:teuthology.orchestra.run.smithi028.stdout: "ops": [], 2024-04-26T03:39:27.758 INFO:teuthology.orchestra.run.smithi028.stdout: "complaint_time": 30, 2024-04-26T03:39:27.758 INFO:teuthology.orchestra.run.smithi028.stdout: "num_blocked_ops": 0 2024-04-26T03:39:27.758 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:39:27.767 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-26T03:39:27.775 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-04-26T03:39:27.774+0000 7fee562fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31759) UID: 0 2024-04-26T03:39:27.815 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0: {} 2024-04-26T03:39:27.816 INFO:teuthology.orchestra.run.smithi028.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T03:39:27.835 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1: {} 2024-04-26T03:39:27.835 INFO:teuthology.orchestra.run.smithi028.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T03:39:27.853 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2: {} 2024-04-26T03:39:27.853 INFO:teuthology.orchestra.run.smithi028.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T03:39:27.870 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3: {} 2024-04-26T03:39:27.871 INFO:teuthology.orchestra.run.smithi028.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T03:39:27.876 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:27.876 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:27.876 INFO:tasks.ceph.osd.8.smithi028.stderr:2024-04-26T03:39:27.874+0000 7f5bb75d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 31968) UID: 0 2024-04-26T03:39:27.885 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-26T03:39:27.888 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4: {} 2024-04-26T03:39:27.888 INFO:teuthology.orchestra.run.smithi028.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T03:39:27.908 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5: {} 2024-04-26T03:39:27.908 INFO:teuthology.orchestra.run.smithi028.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T03:39:27.927 INFO:teuthology.orchestra.run.smithi028.stdout:osd.6: {} 2024-04-26T03:39:27.927 INFO:teuthology.orchestra.run.smithi028.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T03:39:27.945 INFO:teuthology.orchestra.run.smithi028.stdout:osd.7: {} 2024-04-26T03:39:27.945 INFO:teuthology.orchestra.run.smithi028.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T03:39:27.962 INFO:teuthology.orchestra.run.smithi028.stdout:osd.8: {} 2024-04-26T03:39:27.962 INFO:teuthology.orchestra.run.smithi028.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T03:39:27.976 INFO:tasks.ceph.osd.4.smithi028.stderr:2024-04-26T03:39:27.975+0000 7fd9511b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31963) UID: 0 2024-04-26T03:39:27.981 INFO:teuthology.orchestra.run.smithi028.stdout:osd.9: {} 2024-04-26T03:39:27.981 INFO:teuthology.orchestra.run.smithi028.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T03:39:27.998 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:27.998 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:27.999 INFO:teuthology.orchestra.run.smithi028.stdout:osd.10: {} 2024-04-26T03:39:27.999 INFO:teuthology.orchestra.run.smithi028.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T03:39:28.010 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-26T03:39:28.017 INFO:teuthology.orchestra.run.smithi028.stdout:osd.11: {} 2024-04-26T03:39:28.018 INFO:teuthology.orchestra.run.smithi028.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T03:39:28.035 INFO:teuthology.orchestra.run.smithi028.stdout:osd.12: {} 2024-04-26T03:39:28.035 INFO:teuthology.orchestra.run.smithi028.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T03:39:28.054 INFO:teuthology.orchestra.run.smithi028.stdout:osd.13: {} 2024-04-26T03:39:28.054 INFO:teuthology.orchestra.run.smithi028.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T03:39:28.072 INFO:teuthology.orchestra.run.smithi028.stdout:osd.14: {} 2024-04-26T03:39:28.072 INFO:teuthology.orchestra.run.smithi028.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T03:39:28.078 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-04-26T03:39:28.076+0000 7f745118d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 31821) UID: 0 2024-04-26T03:39:28.090 INFO:teuthology.orchestra.run.smithi028.stdout:osd.15: {} 2024-04-26T03:39:28.090 INFO:teuthology.orchestra.run.smithi028.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T03:39:28.126 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:28.126 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:28.135 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-26T03:39:28.178 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:39:28.177+0000 7f60d3c4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31757) UID: 0 2024-04-26T03:39:28.256 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-04-26T03:39:28.256 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-04-26T03:39:28.279 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:39:28.278+0000 7f60d3c4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31757) UID: 0 2024-04-26T03:39:28.283 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:28.283 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:28.292 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-04-26T03:39:28.379 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:39:28.378+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:39:28.442 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:28.442 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:28.451 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-04-26T03:39:28.481 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:39:28.479+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:39:28.557 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:28.557 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:28.566 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-04-26T03:39:28.582 INFO:tasks.ceph.osd.14.smithi066.stderr:2024-04-26T03:39:28.581+0000 7f5395f08640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 31765) UID: 0 2024-04-26T03:39:28.672 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:28.672 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:28.682 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-04-26T03:39:28.684 INFO:tasks.ceph.osd.10.smithi066.stderr:2024-04-26T03:39:28.682+0000 7f130fcc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 31763) UID: 0 2024-04-26T03:39:28.692 INFO:teuthology.orchestra.run.smithi028.stderr:reweighted osd.0 to 1 (10000) 2024-04-26T03:39:28.695 INFO:tasks.rados.rados.0.smithi092.stdout:325: done (1 left) 2024-04-26T03:39:28.695 INFO:tasks.rados.rados.0.smithi092.stdout:326: read oid 22 snap 16 2024-04-26T03:39:28.695 INFO:tasks.rados.rados.0.smithi092.stdout:326: expect (ObjNum 44 snap 0 seq_num 44) 2024-04-26T03:39:28.696 INFO:tasks.rados.rados.0.smithi092.stdout:327: delete oid 38 current snap is 22 2024-04-26T03:39:28.696 INFO:tasks.rados.rados.0.smithi092.stdout:324: expect (ObjNum 139 snap 17 seq_num 139) 2024-04-26T03:39:28.711 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-26T03:39:28.718 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-04-26T03:39:28.785 INFO:tasks.ceph.osd.10.smithi066.stderr:2024-04-26T03:39:28.784+0000 7f130fcc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 31763) UID: 0 2024-04-26T03:39:28.807 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:28.807 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:28.816 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-04-26T03:39:28.885 INFO:tasks.ceph.osd.15.smithi092.stderr:2024-04-26T03:39:28.884+0000 7f6d12dec640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 31824) UID: 0 2024-04-26T03:39:28.905 INFO:tasks.rados.rados.0.smithi092.stdout:326: expect (ObjNum 44 snap 0 seq_num 44) 2024-04-26T03:39:28.924 INFO:tasks.rados.rados.0.smithi092.stdout:324: done (2 left) 2024-04-26T03:39:28.925 INFO:tasks.rados.rados.0.smithi092.stdout:326: done (1 left) 2024-04-26T03:39:28.925 INFO:tasks.rados.rados.0.smithi092.stdout:327: done (0 left) 2024-04-26T03:39:28.925 INFO:tasks.rados.rados.0.smithi092.stdout:328: rollback oid 20 current snap is 22 2024-04-26T03:39:28.925 INFO:tasks.rados.rados.0.smithi092.stdout:rollback oid 20 to 21 2024-04-26T03:39:28.925 INFO:tasks.rados.rados.0.smithi092.stdout:329: snap_remove snap 17 2024-04-26T03:39:28.928 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:28.929 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:28.938 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-04-26T03:39:28.987 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-04-26T03:39:28.986+0000 7fee562fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31759) UID: 0 2024-04-26T03:39:29.045 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:29.046 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:29.056 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-26T03:39:29.100 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:39:29.099+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:39:29.175 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:29.176 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:29.185 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-26T03:39:29.188 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:39:29.187+0000 7f60d3c4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31757) UID: 0 2024-04-26T03:39:29.289 INFO:tasks.ceph.osd.4.smithi028.stderr:2024-04-26T03:39:29.287+0000 7fd9511b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31963) UID: 0 2024-04-26T03:39:29.334 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:29.334 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:29.342 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-26T03:39:29.390 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-04-26T03:39:29.388+0000 7fe9223c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31954) UID: 0 2024-04-26T03:39:29.448 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:29.448 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:29.457 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-26T03:39:29.490 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-04-26T03:39:29.489+0000 7fdc2d21c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31819) UID: 0 2024-04-26T03:39:29.592 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:39:29.591+0000 7f60d3c4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31757) UID: 0 2024-04-26T03:39:29.605 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:29.605 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:29.613 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-26T03:39:29.687 INFO:teuthology.orchestra.run.smithi028.stderr:noscrub is unset 2024-04-26T03:39:29.687 INFO:teuthology.orchestra.run.smithi028.stderr:reweighted osd.1 to 1 (10000) 2024-04-26T03:39:29.692 INFO:tasks.rados.rados.0.smithi092.stdout:329: done (1 left) 2024-04-26T03:39:29.692 INFO:tasks.rados.rados.0.smithi092.stdout:330: rmattr oid 6 current snap is 22 2024-04-26T03:39:29.692 INFO:tasks.rados.rados.0.smithi092.stdout:330: done (1 left) 2024-04-26T03:39:29.692 INFO:tasks.rados.rados.0.smithi092.stdout:331: copy_from oid 36 from oid 49 current snap is 22 2024-04-26T03:39:29.692 INFO:tasks.rados.rados.0.smithi092.stdout:328: finishing rollback tid 1 to smithi09235267-20 2024-04-26T03:39:29.692 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 20 v 57 (ObjNum 449 snap 0 seq_num 3583203488) dirty dne 2024-04-26T03:39:29.692 INFO:tasks.rados.rados.0.smithi092.stdout:328: done (1 left) 2024-04-26T03:39:29.692 INFO:tasks.rados.rados.0.smithi092.stdout:332: copy_from oid 39 from oid 18 current snap is 22 2024-04-26T03:39:29.692 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 41 current snap is 22 2024-04-26T03:39:29.692 INFO:tasks.rados.rados.0.smithi092.stdout:333: seq_num 150 ranges {688128=114688} 2024-04-26T03:39:29.693 INFO:tasks.ceph.osd.12.smithi028.stderr:2024-04-26T03:39:29.691+0000 7f114fe14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 31967) UID: 0 2024-04-26T03:39:29.696 INFO:tasks.rados.rados.0.smithi092.stdout:333: writing smithi09235267-41 from 688128 to 802816 tid 1 2024-04-26T03:39:29.696 INFO:tasks.rados.rados.0.smithi092.stdout:334: rollback oid 35 current snap is 22 2024-04-26T03:39:29.696 INFO:tasks.rados.rados.0.smithi092.stdout:rollback oid 35 to 21 2024-04-26T03:39:29.696 INFO:tasks.rados.rados.0.smithi092.stdout:335: snap_remove snap 12 2024-04-26T03:39:29.715 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-04-26T03:39:29.731 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:29.731 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:29.741 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-26T03:39:29.800 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:39:29.798+0000 7fba0d2ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31961) UID: 0 2024-04-26T03:39:29.847 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:29.847 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:29.856 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-04-26T03:39:29.900 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:39:29.899+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:39:29.963 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:29.963 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:29.972 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-04-26T03:39:29.995 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-04-26T03:39:29.993+0000 7fdc2d21c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31819) UID: 0 2024-04-26T03:39:30.096 INFO:tasks.ceph.osd.12.smithi028.stderr:2024-04-26T03:39:30.094+0000 7f114fe14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 31967) UID: 0 2024-04-26T03:39:30.102 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-26T03:39:30.120 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:30.120 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:30.128 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-04-26T03:39:30.196 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:39:30.194+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:39:30.226 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0: {} 2024-04-26T03:39:30.226 INFO:teuthology.orchestra.run.smithi028.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T03:39:30.235 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:30.235 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:30.244 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-04-26T03:39:30.245 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1: {} 2024-04-26T03:39:30.245 INFO:teuthology.orchestra.run.smithi028.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T03:39:30.262 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2: {} 2024-04-26T03:39:30.262 INFO:teuthology.orchestra.run.smithi028.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T03:39:30.281 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3: {} 2024-04-26T03:39:30.281 INFO:teuthology.orchestra.run.smithi028.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T03:39:30.297 INFO:tasks.ceph.osd.8.smithi028.stderr:2024-04-26T03:39:30.295+0000 7f5bb75d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 31968) UID: 0 2024-04-26T03:39:30.298 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4: {} 2024-04-26T03:39:30.298 INFO:teuthology.orchestra.run.smithi028.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T03:39:30.315 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5: {} 2024-04-26T03:39:30.316 INFO:teuthology.orchestra.run.smithi028.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T03:39:30.334 INFO:teuthology.orchestra.run.smithi028.stdout:osd.6: {} 2024-04-26T03:39:30.334 INFO:teuthology.orchestra.run.smithi028.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T03:39:30.352 INFO:teuthology.orchestra.run.smithi028.stdout:osd.7: {} 2024-04-26T03:39:30.352 INFO:teuthology.orchestra.run.smithi028.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T03:39:30.368 INFO:teuthology.orchestra.run.smithi028.stdout:osd.8: {} 2024-04-26T03:39:30.369 INFO:teuthology.orchestra.run.smithi028.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T03:39:30.386 INFO:teuthology.orchestra.run.smithi028.stdout:osd.9: {} 2024-04-26T03:39:30.386 INFO:teuthology.orchestra.run.smithi028.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T03:39:30.392 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:30.392 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:30.397 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:39:30.396+0000 7f60d3c4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31757) UID: 0 2024-04-26T03:39:30.401 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-04-26T03:39:30.405 INFO:teuthology.orchestra.run.smithi028.stdout:osd.10: {} 2024-04-26T03:39:30.405 INFO:teuthology.orchestra.run.smithi028.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T03:39:30.423 INFO:teuthology.orchestra.run.smithi028.stdout:osd.11: {} 2024-04-26T03:39:30.423 INFO:teuthology.orchestra.run.smithi028.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T03:39:30.440 INFO:teuthology.orchestra.run.smithi028.stdout:osd.12: {} 2024-04-26T03:39:30.440 INFO:teuthology.orchestra.run.smithi028.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T03:39:30.457 INFO:teuthology.orchestra.run.smithi028.stdout:osd.13: {} 2024-04-26T03:39:30.458 INFO:teuthology.orchestra.run.smithi028.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T03:39:30.476 INFO:teuthology.orchestra.run.smithi028.stdout:osd.14: {} 2024-04-26T03:39:30.476 INFO:teuthology.orchestra.run.smithi028.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T03:39:30.494 INFO:teuthology.orchestra.run.smithi028.stdout:osd.15: {} 2024-04-26T03:39:30.494 INFO:teuthology.orchestra.run.smithi028.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T03:39:30.498 INFO:tasks.ceph.osd.10.smithi066.stderr:2024-04-26T03:39:30.497+0000 7f130fcc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 31763) UID: 0 2024-04-26T03:39:30.508 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-26T03:39:30.508 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-04-26T03:39:30.508 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-04-26T03:39:30.508 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-04-26T03:39:30.508 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-26T03:39:30.516 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-04-26T03:39:30.599 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:39:30.598+0000 7fba0d2ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31961) UID: 0 2024-04-26T03:39:30.625 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-26T03:39:30.625 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-04-26T03:39:30.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-04-26T03:39:30.625 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-04-26T03:39:30.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.625 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:48 3.9s0 3:9d1726ae:::smithi09235267-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:30.625 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.504164+0000", 2024-04-26T03:39:30.625 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 39.118114007000003, 2024-04-26T03:39:30.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.014750030000000001, 2024-04-26T03:39:30.625 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:30.625 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:30.625 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-04-26T03:39:30.626 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:30.626 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:30.626 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:30.626 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 48 2024-04-26T03:39:30.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.626 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:30.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:30.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504164+0000", 2024-04-26T03:39:30.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:30.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:30.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504164+0000", 2024-04-26T03:39:30.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:30.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:30.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504165+0000", 2024-04-26T03:39:30.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8999999999999996e-07 2024-04-26T03:39:30.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:30.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504169+0000", 2024-04-26T03:39:30.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0239999999999999e-06 2024-04-26T03:39:30.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:30.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504178+0000", 2024-04-26T03:39:30.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7610000000000002e-06 2024-04-26T03:39:30.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:30.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504187+0000", 2024-04-26T03:39:30.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1799999999999996e-06 2024-04-26T03:39:30.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:30.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.509439+0000", 2024-04-26T03:39:30.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0052526150000000004 2024-04-26T03:39:30.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:30.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.509517+0000", 2024-04-26T03:39:30.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7417999999999996e-05 2024-04-26T03:39:30.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:30.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.516457+0000", 2024-04-26T03:39:30.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0069402259999999999 2024-04-26T03:39:30.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:30.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.516564+0000", 2024-04-26T03:39:30.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010746799999999999 2024-04-26T03:39:30.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:30.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.518914+0000", 2024-04-26T03:39:30.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023496480000000002 2024-04-26T03:39:30.630 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:30.630 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:30.630 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:30.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.630 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:60 3.es0 3:7db045cf:::smithi09235267-20:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:30.630 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.512353+0000", 2024-04-26T03:39:30.630 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 39.109924558000003, 2024-04-26T03:39:30.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015276305, 2024-04-26T03:39:30.630 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:30.630 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:30.630 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-04-26T03:39:30.631 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:30.631 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:30.631 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:30.631 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 60 2024-04-26T03:39:30.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.631 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:30.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:30.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.512353+0000", 2024-04-26T03:39:30.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:30.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:30.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.512353+0000", 2024-04-26T03:39:30.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:30.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:30.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.512354+0000", 2024-04-26T03:39:30.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3600000000000003e-07 2024-04-26T03:39:30.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:30.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.512357+0000", 2024-04-26T03:39:30.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3050000000000001e-06 2024-04-26T03:39:30.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:30.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.512365+0000", 2024-04-26T03:39:30.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1950000000000003e-06 2024-04-26T03:39:30.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:30.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.512372+0000", 2024-04-26T03:39:30.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1589999999999998e-06 2024-04-26T03:39:30.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:30.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.517366+0000", 2024-04-26T03:39:30.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0049946269999999997 2024-04-26T03:39:30.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:30.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.517461+0000", 2024-04-26T03:39:30.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4529000000000002e-05 2024-04-26T03:39:30.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:30.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.521520+0000", 2024-04-26T03:39:30.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0040588519999999999 2024-04-26T03:39:30.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:30.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.521609+0000", 2024-04-26T03:39:30.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8967000000000004e-05 2024-04-26T03:39:30.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:30.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.527630+0000", 2024-04-26T03:39:30.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0060210350000000001 2024-04-26T03:39:30.635 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:30.636 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:30.636 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:30.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.636 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:72 3.es0 3:75fafc97:::smithi09235267-24:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:30.636 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.521677+0000", 2024-04-26T03:39:30.636 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 39.100600915000001, 2024-04-26T03:39:30.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012551157, 2024-04-26T03:39:30.636 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:30.636 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:30.636 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-04-26T03:39:30.636 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:30.636 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:30.637 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:30.637 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 72 2024-04-26T03:39:30.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.637 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:30.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:30.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.521677+0000", 2024-04-26T03:39:30.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:30.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:30.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.521677+0000", 2024-04-26T03:39:30.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:30.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:30.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.521677+0000", 2024-04-26T03:39:30.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.15e-07 2024-04-26T03:39:30.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:30.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.521679+0000", 2024-04-26T03:39:30.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9429999999999999e-06 2024-04-26T03:39:30.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:30.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.521684+0000", 2024-04-26T03:39:30.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3610000000000003e-06 2024-04-26T03:39:30.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:30.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.521688+0000", 2024-04-26T03:39:30.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.138e-06 2024-04-26T03:39:30.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:30.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528505+0000", 2024-04-26T03:39:30.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0068170679999999999 2024-04-26T03:39:30.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:30.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528576+0000", 2024-04-26T03:39:30.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1507000000000005e-05 2024-04-26T03:39:30.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:30.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.531827+0000", 2024-04-26T03:39:30.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003250883 2024-04-26T03:39:30.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:30.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.531903+0000", 2024-04-26T03:39:30.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5957999999999993e-05 2024-04-26T03:39:30.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:30.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.534228+0000", 2024-04-26T03:39:30.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023248840000000002 2024-04-26T03:39:30.641 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:30.641 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:30.641 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:30.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.641 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:301 3.es0 3:75fafc97:::smithi09235267-24:head [append 0~442368] snapc 6=[6,5] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-26T03:39:30.641 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:59.527343+0000", 2024-04-26T03:39:30.641 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.094934475999999, 2024-04-26T03:39:30.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018417728000000001, 2024-04-26T03:39:30.641 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:30.641 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:30.641 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:30.641 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:30.641 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:30.642 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:30.642 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 301 2024-04-26T03:39:30.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.643 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:30.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:30.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.527343+0000", 2024-04-26T03:39:30.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:30.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:30.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.527343+0000", 2024-04-26T03:39:30.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:30.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:30.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.527346+0000", 2024-04-26T03:39:30.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3420000000000002e-06 2024-04-26T03:39:30.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:30.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.527805+0000", 2024-04-26T03:39:30.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00045928899999999999 2024-04-26T03:39:30.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:30.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.527825+0000", 2024-04-26T03:39:30.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9825000000000001e-05 2024-04-26T03:39:30.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:30.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.527838+0000", 2024-04-26T03:39:30.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.31e-05 2024-04-26T03:39:30.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:30.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.527930+0000", 2024-04-26T03:39:30.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2060999999999998e-05 2024-04-26T03:39:30.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-04-26T03:39:30.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.527937+0000", 2024-04-26T03:39:30.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3150000000000003e-06 2024-04-26T03:39:30.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:30.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.530067+0000", 2024-04-26T03:39:30.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0021300989999999999 2024-04-26T03:39:30.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:30.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.530147+0000", 2024-04-26T03:39:30.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9178999999999995e-05 2024-04-26T03:39:30.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:30.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.531062+0000", 2024-04-26T03:39:30.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00091551399999999998 2024-04-26T03:39:30.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:30.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.542352+0000", 2024-04-26T03:39:30.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011290117000000001 2024-04-26T03:39:30.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:39:30.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.545702+0000", 2024-04-26T03:39:30.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033499290000000002 2024-04-26T03:39:30.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:30.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.545761+0000", 2024-04-26T03:39:30.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8958e-05 2024-04-26T03:39:30.648 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:30.648 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:30.648 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:30.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.648 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:302 3.es0 3:75fafc97:::smithi09235267-24:head [append 0~720896] snapc 6=[6,5] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-26T03:39:30.648 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:59.531209+0000", 2024-04-26T03:39:30.649 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.091068443000001, 2024-04-26T03:39:30.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015580979, 2024-04-26T03:39:30.649 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:30.649 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:30.649 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:30.649 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:30.649 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:30.649 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:30.649 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 302 2024-04-26T03:39:30.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.649 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:30.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:30.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.531209+0000", 2024-04-26T03:39:30.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:30.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:30.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.531209+0000", 2024-04-26T03:39:30.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:30.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:30.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.531211+0000", 2024-04-26T03:39:30.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.37e-06 2024-04-26T03:39:30.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:30.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.531886+0000", 2024-04-26T03:39:30.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00067507799999999996 2024-04-26T03:39:30.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:30.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.531901+0000", 2024-04-26T03:39:30.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5335000000000001e-05 2024-04-26T03:39:30.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:30.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.531911+0000", 2024-04-26T03:39:30.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6569999999999992e-06 2024-04-26T03:39:30.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:30.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.535914+0000", 2024-04-26T03:39:30.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0040035410000000002 2024-04-26T03:39:30.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:30.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.536006+0000", 2024-04-26T03:39:30.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1390000000000004e-05 2024-04-26T03:39:30.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:30.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.536984+0000", 2024-04-26T03:39:30.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0009780609999999999 2024-04-26T03:39:30.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:30.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.544537+0000", 2024-04-26T03:39:30.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.007553087 2024-04-26T03:39:30.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:39:30.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.546737+0000", 2024-04-26T03:39:30.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002199613 2024-04-26T03:39:30.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:30.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.546790+0000", 2024-04-26T03:39:30.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3847000000000001e-05 2024-04-26T03:39:30.654 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:30.654 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:30.654 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:30.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.654 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:303 3.es0 3:75fafc97:::smithi09235267-24:head [append 0~49152] snapc 6=[6,5] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-26T03:39:30.654 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:59.531927+0000", 2024-04-26T03:39:30.654 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.090350901000001, 2024-04-26T03:39:30.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017418454999999999, 2024-04-26T03:39:30.654 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:30.654 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:30.655 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:30.655 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:30.655 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:30.655 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:30.655 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 303 2024-04-26T03:39:30.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.655 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:30.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:30.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.531927+0000", 2024-04-26T03:39:30.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:30.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:30.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.531927+0000", 2024-04-26T03:39:30.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:30.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:30.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.531928+0000", 2024-04-26T03:39:30.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7699999999999992e-07 2024-04-26T03:39:30.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:30.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.531984+0000", 2024-04-26T03:39:30.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5787e-05 2024-04-26T03:39:30.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.657 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:30.657 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.531993+0000", 2024-04-26T03:39:30.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5349999999999993e-06 2024-04-26T03:39:30.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.657 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:30.657 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.531998+0000", 2024-04-26T03:39:30.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8040000000000001e-06 2024-04-26T03:39:30.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.657 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:30.657 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.542497+0000", 2024-04-26T03:39:30.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010499163000000001 2024-04-26T03:39:30.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:30.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.542593+0000", 2024-04-26T03:39:30.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6026999999999994e-05 2024-04-26T03:39:30.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:30.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.543050+0000", 2024-04-26T03:39:30.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00045626999999999999 2024-04-26T03:39:30.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:30.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.544538+0000", 2024-04-26T03:39:30.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014883299999999999 2024-04-26T03:39:30.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:39:30.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.549296+0000", 2024-04-26T03:39:30.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0047584369999999999 2024-04-26T03:39:30.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:30.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.549345+0000", 2024-04-26T03:39:30.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9125000000000001e-05 2024-04-26T03:39:30.659 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:30.659 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:30.660 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:30.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.660 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:304 3.es0 3:75fafc97:::smithi09235267-24:head [setxattr _header (56) in=7b] snapc 6=[6,5] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-26T03:39:30.660 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:59.532007+0000", 2024-04-26T03:39:30.660 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.090270853, 2024-04-26T03:39:30.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017496230000000002, 2024-04-26T03:39:30.660 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:30.660 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:30.660 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:30.660 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:30.660 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:30.660 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:30.660 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 304 2024-04-26T03:39:30.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.661 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:30.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.661 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:30.661 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.532007+0000", 2024-04-26T03:39:30.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:30.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.661 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:30.661 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.532007+0000", 2024-04-26T03:39:30.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:30.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.661 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:30.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.532008+0000", 2024-04-26T03:39:30.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1600000000000001e-07 2024-04-26T03:39:30.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:30.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.532011+0000", 2024-04-26T03:39:30.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5039999999999998e-06 2024-04-26T03:39:30.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:30.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.532015+0000", 2024-04-26T03:39:30.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0520000000000003e-06 2024-04-26T03:39:30.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:30.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.532029+0000", 2024-04-26T03:39:30.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4353e-05 2024-04-26T03:39:30.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:30.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.543638+0000", 2024-04-26T03:39:30.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011608095000000001 2024-04-26T03:39:30.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:30.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.543725+0000", 2024-04-26T03:39:30.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7379999999999993e-05 2024-04-26T03:39:30.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:30.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.544159+0000", 2024-04-26T03:39:30.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000433602 2024-04-26T03:39:30.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:30.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.544744+0000", 2024-04-26T03:39:30.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00058520399999999998 2024-04-26T03:39:30.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:39:30.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.549431+0000", 2024-04-26T03:39:30.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0046875110000000001 2024-04-26T03:39:30.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.665 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:30.665 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.549503+0000", 2024-04-26T03:39:30.665 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2013000000000003e-05 2024-04-26T03:39:30.665 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:30.665 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:30.665 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:30.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.665 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:305 3.es0 3:75fafc97:::smithi09235267-24:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e43)", 2024-04-26T03:39:30.665 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:59.532037+0000", 2024-04-26T03:39:30.665 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.090240942000001, 2024-04-26T03:39:30.665 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019157461000000001, 2024-04-26T03:39:30.666 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:30.666 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:30.666 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-04-26T03:39:30.666 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:30.666 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:30.666 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:30.666 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 305 2024-04-26T03:39:30.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.666 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:30.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:30.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.532037+0000", 2024-04-26T03:39:30.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:30.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:30.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.532037+0000", 2024-04-26T03:39:30.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:30.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:30.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.532037+0000", 2024-04-26T03:39:30.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0900000000000002e-07 2024-04-26T03:39:30.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:30.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.532053+0000", 2024-04-26T03:39:30.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5586999999999998e-05 2024-04-26T03:39:30.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:30.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.532060+0000", 2024-04-26T03:39:30.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3610000000000003e-06 2024-04-26T03:39:30.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:30.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.532065+0000", 2024-04-26T03:39:30.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4989999999999996e-06 2024-04-26T03:39:30.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:30.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.544785+0000", 2024-04-26T03:39:30.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012719988 2024-04-26T03:39:30.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:30.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.544890+0000", 2024-04-26T03:39:30.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010498 2024-04-26T03:39:30.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:30.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.549851+0000", 2024-04-26T03:39:30.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0049613050000000001 2024-04-26T03:39:30.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:30.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.549944+0000", 2024-04-26T03:39:30.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2950999999999998e-05 2024-04-26T03:39:30.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:30.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.551194+0000", 2024-04-26T03:39:30.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012503810000000001 2024-04-26T03:39:30.670 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:30.670 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:30.670 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:30.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.670 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:349 3.es0 3:75fafc97:::smithi09235267-24:head [append 0~557056] snapc 7=[7,6,5] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-26T03:39:30.671 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:01.631025+0000", 2024-04-26T03:39:30.671 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.991252566, 2024-04-26T03:39:30.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012493898, 2024-04-26T03:39:30.671 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:30.671 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:30.671 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:30.671 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:30.671 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:30.671 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:30.671 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 349 2024-04-26T03:39:30.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.671 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:30.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:30.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.631025+0000", 2024-04-26T03:39:30.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:30.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:30.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.631025+0000", 2024-04-26T03:39:30.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:30.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:30.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.631027+0000", 2024-04-26T03:39:30.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2359999999999999e-06 2024-04-26T03:39:30.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.673 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:30.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.631581+0000", 2024-04-26T03:39:30.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00055440199999999995 2024-04-26T03:39:30.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.673 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:30.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.631594+0000", 2024-04-26T03:39:30.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2978e-05 2024-04-26T03:39:30.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.673 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:30.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.631602+0000", 2024-04-26T03:39:30.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9340000000000006e-06 2024-04-26T03:39:30.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.674 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:30.674 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.631664+0000", 2024-04-26T03:39:30.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2119999999999995e-05 2024-04-26T03:39:30.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.674 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:30.674 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.631781+0000", 2024-04-26T03:39:30.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011652 2024-04-26T03:39:30.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:30.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.632790+0000", 2024-04-26T03:39:30.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010097019999999999 2024-04-26T03:39:30.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:30.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.643222+0000", 2024-04-26T03:39:30.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010431753 2024-04-26T03:39:30.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.678 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.678 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:39:30.678 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.643459+0000", 2024-04-26T03:39:30.678 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00023718900000000001 2024-04-26T03:39:30.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.678 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.678 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:30.678 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.643519+0000", 2024-04-26T03:39:30.678 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0064e-05 2024-04-26T03:39:30.678 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:30.678 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:30.678 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:30.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.679 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:351 3.es0 3:75fafc97:::smithi09235267-24:head [append 0~98304] snapc 7=[7,6,5] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-26T03:39:30.679 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:01.635505+0000", 2024-04-26T03:39:30.679 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.986772602999999, 2024-04-26T03:39:30.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012808014, 2024-04-26T03:39:30.679 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:30.679 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:30.679 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:30.679 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:30.679 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:30.679 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:30.679 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 351 2024-04-26T03:39:30.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.680 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:30.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:30.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.635505+0000", 2024-04-26T03:39:30.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:30.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:30.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.635505+0000", 2024-04-26T03:39:30.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:30.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:30.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.635506+0000", 2024-04-26T03:39:30.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2099999999999995e-07 2024-04-26T03:39:30.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:30.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.635555+0000", 2024-04-26T03:39:30.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8658000000000001e-05 2024-04-26T03:39:30.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:30.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.635567+0000", 2024-04-26T03:39:30.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1809e-05 2024-04-26T03:39:30.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:30.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.635575+0000", 2024-04-26T03:39:30.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3319999999999992e-06 2024-04-26T03:39:30.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:30.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.641450+0000", 2024-04-26T03:39:30.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00587558 2024-04-26T03:39:30.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:30.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.641561+0000", 2024-04-26T03:39:30.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000110183 2024-04-26T03:39:30.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.683 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:30.683 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.642098+0000", 2024-04-26T03:39:30.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00053782199999999997 2024-04-26T03:39:30.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.683 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:30.683 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.643585+0000", 2024-04-26T03:39:30.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001486802 2024-04-26T03:39:30.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.683 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:39:30.683 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.648271+0000", 2024-04-26T03:39:30.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0046858309999999997 2024-04-26T03:39:30.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:30.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.648313+0000", 2024-04-26T03:39:30.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2175999999999997e-05 2024-04-26T03:39:30.684 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:30.684 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:30.684 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:30.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.684 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:352 3.es0 3:75fafc97:::smithi09235267-24:head [setxattr _header (56) in=7b] snapc 7=[7,6,5] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-26T03:39:30.684 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:01.635588+0000", 2024-04-26T03:39:30.684 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.98668996, 2024-04-26T03:39:30.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012886458, 2024-04-26T03:39:30.684 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:30.685 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:30.685 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:30.685 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:30.685 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:30.685 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:30.685 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 352 2024-04-26T03:39:30.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.685 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:30.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:30.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.635588+0000", 2024-04-26T03:39:30.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:30.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:30.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.635588+0000", 2024-04-26T03:39:30.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:30.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:30.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.635589+0000", 2024-04-26T03:39:30.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1699999999999998e-07 2024-04-26T03:39:30.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:30.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.635593+0000", 2024-04-26T03:39:30.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2769999999999999e-06 2024-04-26T03:39:30.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:30.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.635601+0000", 2024-04-26T03:39:30.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6249999999999996e-06 2024-04-26T03:39:30.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:30.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.635608+0000", 2024-04-26T03:39:30.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6900000000000003e-06 2024-04-26T03:39:30.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:30.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.643620+0000", 2024-04-26T03:39:30.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0080122619999999992 2024-04-26T03:39:30.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:30.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.643681+0000", 2024-04-26T03:39:30.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0593000000000001e-05 2024-04-26T03:39:30.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:30.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.643953+0000", 2024-04-26T03:39:30.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00027240400000000002 2024-04-26T03:39:30.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:30.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.644544+0000", 2024-04-26T03:39:30.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00059055000000000002 2024-04-26T03:39:30.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:39:30.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.648420+0000", 2024-04-26T03:39:30.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038758120000000002 2024-04-26T03:39:30.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:30.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.648474+0000", 2024-04-26T03:39:30.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4627999999999999e-05 2024-04-26T03:39:30.689 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:30.689 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:30.690 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:30.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.690 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:353 3.es0 3:75fafc97:::smithi09235267-24:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e45)", 2024-04-26T03:39:30.690 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:01.635616+0000", 2024-04-26T03:39:30.690 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.986661780999999, 2024-04-26T03:39:30.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013944994, 2024-04-26T03:39:30.690 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:30.690 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:30.690 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-04-26T03:39:30.690 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:30.690 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:30.690 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:30.690 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 353 2024-04-26T03:39:30.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.691 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:30.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:30.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.635616+0000", 2024-04-26T03:39:30.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:30.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:30.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.635616+0000", 2024-04-26T03:39:30.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:30.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:30.692 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.635617+0000", 2024-04-26T03:39:30.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9900000000000001e-07 2024-04-26T03:39:30.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.692 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:30.692 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.635620+0000", 2024-04-26T03:39:30.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.365e-06 2024-04-26T03:39:30.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.692 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:30.692 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.635628+0000", 2024-04-26T03:39:30.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3809999999999999e-06 2024-04-26T03:39:30.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:30.693 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.635635+0000", 2024-04-26T03:39:30.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.934e-06 2024-04-26T03:39:30.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:30.693 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.644429+0000", 2024-04-26T03:39:30.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.008794049 2024-04-26T03:39:30.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:30.693 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.644532+0000", 2024-04-26T03:39:30.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000102494 2024-04-26T03:39:30.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:30.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.648687+0000", 2024-04-26T03:39:30.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0041551770000000003 2024-04-26T03:39:30.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:30.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.648760+0000", 2024-04-26T03:39:30.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2794999999999997e-05 2024-04-26T03:39:30.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:30.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:01.649561+0000", 2024-04-26T03:39:30.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00080130000000000002 2024-04-26T03:39:30.695 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:30.695 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:30.695 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:30.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.695 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:437 3.19s0 3:9d1726ae:::smithi09235267-16:head [delete] snapc 8=[8] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e49)", 2024-04-26T03:39:30.695 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:04.697979+0000", 2024-04-26T03:39:30.695 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.924299164000001, 2024-04-26T03:39:30.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012570316999999999, 2024-04-26T03:39:30.695 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:30.695 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:30.695 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:30.695 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:30.696 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:30.696 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:30.696 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 437 2024-04-26T03:39:30.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.696 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:30.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:30.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.697979+0000", 2024-04-26T03:39:30.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:30.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:30.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.697979+0000", 2024-04-26T03:39:30.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:30.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:30.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.697981+0000", 2024-04-26T03:39:30.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.306e-06 2024-04-26T03:39:30.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:30.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.697989+0000", 2024-04-26T03:39:30.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0830000000000004e-06 2024-04-26T03:39:30.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:30.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.698011+0000", 2024-04-26T03:39:30.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1920999999999999e-05 2024-04-26T03:39:30.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:30.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.698030+0000", 2024-04-26T03:39:30.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8589000000000001e-05 2024-04-26T03:39:30.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:30.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.698117+0000", 2024-04-26T03:39:30.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7769000000000006e-05 2024-04-26T03:39:30.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:30.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.698309+0000", 2024-04-26T03:39:30.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019131700000000001 2024-04-26T03:39:30.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:30.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.699096+0000", 2024-04-26T03:39:30.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00078738300000000001 2024-04-26T03:39:30.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:30.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.700150+0000", 2024-04-26T03:39:30.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010535340000000001 2024-04-26T03:39:30.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:39:30.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.710460+0000", 2024-04-26T03:39:30.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.01031017 2024-04-26T03:39:30.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:30.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.710549+0000", 2024-04-26T03:39:30.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9245000000000004e-05 2024-04-26T03:39:30.700 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:30.700 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:30.700 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:30.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.700 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:768 3.es0 3:799cf17d:::smithi09235267-29:head [append 0~720896] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:30.701 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:20.762340+0000", 2024-04-26T03:39:30.701 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.8599383449999998, 2024-04-26T03:39:30.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.016004096999999998, 2024-04-26T03:39:30.701 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:30.701 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:30.701 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:30.701 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:30.701 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:30.701 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:30.701 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 768 2024-04-26T03:39:30.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.701 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:30.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:30.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.762340+0000", 2024-04-26T03:39:30.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:30.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:30.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.762340+0000", 2024-04-26T03:39:30.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:30.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:30.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.762341+0000", 2024-04-26T03:39:30.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.553e-06 2024-04-26T03:39:30.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:30.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.763140+0000", 2024-04-26T03:39:30.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00079871499999999997 2024-04-26T03:39:30.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:30.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.763154+0000", 2024-04-26T03:39:30.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3896e-05 2024-04-26T03:39:30.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:30.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.763160+0000", 2024-04-26T03:39:30.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7399999999999998e-06 2024-04-26T03:39:30.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:30.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.763227+0000", 2024-04-26T03:39:30.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6567000000000001e-05 2024-04-26T03:39:30.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:30.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.763354+0000", 2024-04-26T03:39:30.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012662099999999999 2024-04-26T03:39:30.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:30.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.764979+0000", 2024-04-26T03:39:30.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016252 2024-04-26T03:39:30.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:30.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.777917+0000", 2024-04-26T03:39:30.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012938219000000001 2024-04-26T03:39:30.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:39:30.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.778249+0000", 2024-04-26T03:39:30.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00033234200000000001 2024-04-26T03:39:30.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:30.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.778344+0000", 2024-04-26T03:39:30.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4244000000000001e-05 2024-04-26T03:39:30.706 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:30.706 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:30.706 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:30.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.706 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:769 3.es0 3:799cf17d:::smithi09235267-29:head [append 0~622592] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:30.706 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:20.765947+0000", 2024-04-26T03:39:30.706 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.8563313410000006, 2024-04-26T03:39:30.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020996510999999999, 2024-04-26T03:39:30.706 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:30.706 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:30.706 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:30.706 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:30.707 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:30.707 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:30.707 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 769 2024-04-26T03:39:30.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.707 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:30.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:30.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.765947+0000", 2024-04-26T03:39:30.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:30.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:30.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.765947+0000", 2024-04-26T03:39:30.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:30.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:30.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.765949+0000", 2024-04-26T03:39:30.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0760000000000001e-06 2024-04-26T03:39:30.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:30.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.766500+0000", 2024-04-26T03:39:30.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00055137199999999995 2024-04-26T03:39:30.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:30.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.766514+0000", 2024-04-26T03:39:30.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3529999999999999e-05 2024-04-26T03:39:30.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:30.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.766522+0000", 2024-04-26T03:39:30.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6300000000000004e-06 2024-04-26T03:39:30.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:30.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.772431+0000", 2024-04-26T03:39:30.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0059086709999999999 2024-04-26T03:39:30.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:30.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.772554+0000", 2024-04-26T03:39:30.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012323999999999999 2024-04-26T03:39:30.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:30.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.773579+0000", 2024-04-26T03:39:30.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001024456 2024-04-26T03:39:30.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:30.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.783946+0000", 2024-04-26T03:39:30.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010367516 2024-04-26T03:39:30.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:39:30.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.786852+0000", 2024-04-26T03:39:30.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002906414 2024-04-26T03:39:30.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:30.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.786943+0000", 2024-04-26T03:39:30.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0606000000000006e-05 2024-04-26T03:39:30.711 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:30.711 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:30.711 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:30.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.712 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:770 3.es0 3:799cf17d:::smithi09235267-29:head [append 0~688128] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:30.712 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:20.770509+0000", 2024-04-26T03:39:30.712 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.8517692189999995, 2024-04-26T03:39:30.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.016972135999999999, 2024-04-26T03:39:30.712 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:30.712 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:30.712 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:30.712 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:30.712 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:30.712 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:30.712 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 770 2024-04-26T03:39:30.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.712 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:30.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:30.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.770509+0000", 2024-04-26T03:39:30.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:30.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:30.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.770509+0000", 2024-04-26T03:39:30.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:30.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:30.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.770510+0000", 2024-04-26T03:39:30.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2869999999999999e-06 2024-04-26T03:39:30.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:30.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.771141+0000", 2024-04-26T03:39:30.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00063108999999999995 2024-04-26T03:39:30.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:30.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.771153+0000", 2024-04-26T03:39:30.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1841000000000001e-05 2024-04-26T03:39:30.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:30.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.771158+0000", 2024-04-26T03:39:30.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9779999999999996e-06 2024-04-26T03:39:30.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:30.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.778435+0000", 2024-04-26T03:39:30.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0072772890000000002 2024-04-26T03:39:30.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:30.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.778523+0000", 2024-04-26T03:39:30.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8026000000000001e-05 2024-04-26T03:39:30.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:30.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.779152+0000", 2024-04-26T03:39:30.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00062865399999999997 2024-04-26T03:39:30.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:30.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.786741+0000", 2024-04-26T03:39:30.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0075895110000000002 2024-04-26T03:39:30.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:39:30.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.787388+0000", 2024-04-26T03:39:30.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00064617099999999996 2024-04-26T03:39:30.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.717 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:30.717 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.787481+0000", 2024-04-26T03:39:30.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3288999999999994e-05 2024-04-26T03:39:30.717 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:30.717 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:30.717 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:30.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.717 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:771 3.es0 3:799cf17d:::smithi09235267-29:head [append 0~245760] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:30.717 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:20.771883+0000", 2024-04-26T03:39:30.717 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.8503951950000008, 2024-04-26T03:39:30.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017064573999999999, 2024-04-26T03:39:30.717 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:30.718 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:30.718 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:30.718 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:30.718 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:30.718 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:30.718 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 771 2024-04-26T03:39:30.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.718 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:30.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:30.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.771883+0000", 2024-04-26T03:39:30.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:30.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:30.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.771883+0000", 2024-04-26T03:39:30.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:30.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:30.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.771884+0000", 2024-04-26T03:39:30.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8800000000000001e-07 2024-04-26T03:39:30.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:30.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.772169+0000", 2024-04-26T03:39:30.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00028528599999999999 2024-04-26T03:39:30.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:30.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.772176+0000", 2024-04-26T03:39:30.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8199999999999999e-06 2024-04-26T03:39:30.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:30.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.772180+0000", 2024-04-26T03:39:30.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0300000000000004e-06 2024-04-26T03:39:30.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:30.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.784029+0000", 2024-04-26T03:39:30.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011849484 2024-04-26T03:39:30.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:30.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.784164+0000", 2024-04-26T03:39:30.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000134306 2024-04-26T03:39:30.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:30.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.784814+0000", 2024-04-26T03:39:30.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00065014100000000002 2024-04-26T03:39:30.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:30.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.787204+0000", 2024-04-26T03:39:30.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023904540000000002 2024-04-26T03:39:30.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:39:30.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.788789+0000", 2024-04-26T03:39:30.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001585155 2024-04-26T03:39:30.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:30.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.788947+0000", 2024-04-26T03:39:30.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015801000000000001 2024-04-26T03:39:30.723 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:30.723 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:30.723 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:30.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.723 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:772 3.es0 3:799cf17d:::smithi09235267-29:head [setxattr _header (57) in=7b] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:30.723 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:20.772187+0000", 2024-04-26T03:39:30.723 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.8500912770000006, 2024-04-26T03:39:30.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017604511, 2024-04-26T03:39:30.723 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:30.723 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:30.723 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:30.724 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:30.724 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:30.724 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:30.724 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 772 2024-04-26T03:39:30.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.724 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:30.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:30.724 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.772187+0000", 2024-04-26T03:39:30.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:30.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:30.724 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.772187+0000", 2024-04-26T03:39:30.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:30.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:30.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.772187+0000", 2024-04-26T03:39:30.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7400000000000003e-07 2024-04-26T03:39:30.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:30.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.772192+0000", 2024-04-26T03:39:30.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9150000000000004e-06 2024-04-26T03:39:30.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:30.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.772196+0000", 2024-04-26T03:39:30.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2300000000000002e-06 2024-04-26T03:39:30.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:30.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.772199+0000", 2024-04-26T03:39:30.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7690000000000001e-06 2024-04-26T03:39:30.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:30.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.787689+0000", 2024-04-26T03:39:30.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015489614000000001 2024-04-26T03:39:30.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:30.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.787793+0000", 2024-04-26T03:39:30.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000104355 2024-04-26T03:39:30.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:30.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.788198+0000", 2024-04-26T03:39:30.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00040535799999999998 2024-04-26T03:39:30.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:30.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.789095+0000", 2024-04-26T03:39:30.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00089674299999999998 2024-04-26T03:39:30.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:39:30.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.789713+0000", 2024-04-26T03:39:30.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00061809699999999996 2024-04-26T03:39:30.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:30.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.789791+0000", 2024-04-26T03:39:30.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7855999999999995e-05 2024-04-26T03:39:30.728 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:30.728 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:30.729 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:30.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.729 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:773 3.es0 3:799cf17d:::smithi09235267-29:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:30.729 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:20.772203+0000", 2024-04-26T03:39:30.729 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.8500747319999995, 2024-04-26T03:39:30.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019847132, 2024-04-26T03:39:30.729 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:30.729 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:30.729 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-04-26T03:39:30.730 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:30.730 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:30.730 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:30.730 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 773 2024-04-26T03:39:30.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.730 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:30.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:30.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.772203+0000", 2024-04-26T03:39:30.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:30.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:30.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.772203+0000", 2024-04-26T03:39:30.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:30.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:30.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.772203+0000", 2024-04-26T03:39:30.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5100000000000001e-07 2024-04-26T03:39:30.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:30.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.772205+0000", 2024-04-26T03:39:30.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.573e-06 2024-04-26T03:39:30.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:30.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.772209+0000", 2024-04-26T03:39:30.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6480000000000001e-06 2024-04-26T03:39:30.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:30.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.772212+0000", 2024-04-26T03:39:30.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9909999999999998e-06 2024-04-26T03:39:30.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:30.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.788498+0000", 2024-04-26T03:39:30.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.016286265000000001 2024-04-26T03:39:30.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:30.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.788590+0000", 2024-04-26T03:39:30.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2418999999999997e-05 2024-04-26T03:39:30.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:30.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.790138+0000", 2024-04-26T03:39:30.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001547449 2024-04-26T03:39:30.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:30.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.790222+0000", 2024-04-26T03:39:30.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4513999999999998e-05 2024-04-26T03:39:30.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.734 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:30.734 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.792050+0000", 2024-04-26T03:39:30.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018280219999999999 2024-04-26T03:39:30.734 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:30.734 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:30.734 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:30.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.734 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.fs1 65/62 ECSubWrite(tid=178, reqid=client.4556.0:782, at_version=65'60, trim_to=0'0, roll_forward_to=64'57))", 2024-04-26T03:39:30.734 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:20.829135+0000", 2024-04-26T03:39:30.734 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7931425450000003, 2024-04-26T03:39:30.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012301748, 2024-04-26T03:39:30.735 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:30.735 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:30.735 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:30.735 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:30.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.735 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:30.735 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.829135+0000", 2024-04-26T03:39:30.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:30.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.735 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:30.735 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.829135+0000", 2024-04-26T03:39:30.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:30.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:30.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.829136+0000", 2024-04-26T03:39:30.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7399999999999991e-07 2024-04-26T03:39:30.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:30.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.829478+0000", 2024-04-26T03:39:30.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00034213400000000001 2024-04-26T03:39:30.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:30.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.829494+0000", 2024-04-26T03:39:30.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5617999999999999e-05 2024-04-26T03:39:30.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:30.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.829503+0000", 2024-04-26T03:39:30.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5930000000000006e-06 2024-04-26T03:39:30.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:30.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.829573+0000", 2024-04-26T03:39:30.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0345999999999994e-05 2024-04-26T03:39:30.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:30.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.829584+0000", 2024-04-26T03:39:30.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0872e-05 2024-04-26T03:39:30.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:30.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.841396+0000", 2024-04-26T03:39:30.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011811663999999999 2024-04-26T03:39:30.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:30.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:30.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:30.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.841437+0000", 2024-04-26T03:39:30.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1547e-05 2024-04-26T03:39:30.739 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:30.739 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:30.740 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:30.740 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:30.740 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:30.740 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-26T03:39:30.742 INFO:tasks.rados.rados.0.smithi092.stdout:335: done (4 left) 2024-04-26T03:39:30.742 INFO:tasks.rados.rados.0.smithi092.stdout:336: snap_create 2024-04-26T03:39:30.742 INFO:tasks.rados.rados.0.smithi092.stdout:331: finishing copy_from to smithi09235267-36 2024-04-26T03:39:30.742 INFO:tasks.rados.rados.0.smithi092.stdout:331: got expected ENOENT (src dne) 2024-04-26T03:39:30.742 INFO:tasks.rados.rados.0.smithi092.stdout:331: finishing copy_from racing read to smithi09235267-36 2024-04-26T03:39:30.742 INFO:tasks.rados.rados.0.smithi092.stdout:334: finishing rollback tid 1 to smithi09235267-35 2024-04-26T03:39:30.742 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 35 v 0 (ObjNum 289 snap 0 seq_num 1) dirty dne 2024-04-26T03:39:30.742 INFO:tasks.rados.rados.0.smithi092.stdout:332: finishing copy_from racing read to smithi09235267-39 2024-04-26T03:39:30.742 INFO:tasks.rados.rados.0.smithi092.stdout:333: finishing write tid 1 to smithi09235267-41 2024-04-26T03:39:30.743 INFO:tasks.rados.rados.0.smithi092.stdout:333: finishing write tid 2 to smithi09235267-41 2024-04-26T03:39:30.743 INFO:tasks.rados.rados.0.smithi092.stdout:333: finishing write tid 3 to smithi09235267-41 2024-04-26T03:39:30.743 INFO:tasks.rados.rados.0.smithi092.stdout:333: oid 41 updating version 0 to 102 2024-04-26T03:39:30.743 INFO:tasks.rados.rados.0.smithi092.stdout:333: oid 41 updating version 102 to 103 2024-04-26T03:39:30.743 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 41 v 103 (ObjNum 150 snap 22 seq_num 150) dirty exists 2024-04-26T03:39:30.743 INFO:tasks.rados.rados.0.smithi092.stdout:333: left oid 41 (ObjNum 150 snap 22 seq_num 150) 2024-04-26T03:39:30.743 INFO:tasks.rados.rados.0.smithi092.stdout:332: finishing copy_from to smithi09235267-39 2024-04-26T03:39:30.743 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 39 v 45 (ObjNum 124 snap 14 seq_num 124) dirty exists 2024-04-26T03:39:30.743 INFO:teuthology.orchestra.run.smithi028.stderr:reweighted osd.2 to 1 (10000) 2024-04-26T03:39:30.744 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-26T03:39:30.744 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-04-26T03:39:30.745 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:39:30.740+0000 7f60d3c4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31757) UID: 0 2024-04-26T03:39:30.842 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-04-26T03:39:30.840+0000 7fdc2d21c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31819) UID: 0 2024-04-26T03:39:30.857 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:39:30.858 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-04-26T03:39:30.858 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 0 2024-04-26T03:39:30.858 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:39:30.866 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-26T03:39:30.943 INFO:tasks.ceph.osd.10.smithi066.stderr:2024-04-26T03:39:30.942+0000 7f130fcc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 31763) UID: 0 2024-04-26T03:39:30.973 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:39:30.973 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-04-26T03:39:30.973 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-04-26T03:39:30.973 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-04-26T03:39:30.973 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:39:30.981 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-26T03:39:31.044 INFO:tasks.ceph.osd.4.smithi028.stderr:2024-04-26T03:39:31.042+0000 7fd9511b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31963) UID: 0 2024-04-26T03:39:31.130 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:39:31.130 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-04-26T03:39:31.130 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-04-26T03:39:31.130 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-04-26T03:39:31.130 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.130 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.4s3 35/32 ECSubWrite(tid=13, reqid=client.4556.0:106, at_version=35'9, trim_to=0'0, roll_forward_to=35'6))", 2024-04-26T03:39:31.131 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.567072+0000", 2024-04-26T03:39:31.131 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 39.560689965999998, 2024-04-26T03:39:31.131 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0074599230000000002, 2024-04-26T03:39:31.131 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:31.131 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:31.131 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:31.131 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:31.131 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.131 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:31.131 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.567072+0000", 2024-04-26T03:39:31.131 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.131 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.131 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.132 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:31.132 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.567072+0000", 2024-04-26T03:39:31.132 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.132 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.132 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.132 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:31.132 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.567072+0000", 2024-04-26T03:39:31.132 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.8100000000000003e-07 2024-04-26T03:39:31.132 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.132 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.132 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:31.132 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.567302+0000", 2024-04-26T03:39:31.132 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000229457 2024-04-26T03:39:31.132 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.133 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.133 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:31.133 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.567315+0000", 2024-04-26T03:39:31.133 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2845e-05 2024-04-26T03:39:31.133 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.133 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.133 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:31.133 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.567322+0000", 2024-04-26T03:39:31.133 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8979999999999996e-06 2024-04-26T03:39:31.133 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.133 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.133 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:31.133 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.567396+0000", 2024-04-26T03:39:31.134 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3398000000000004e-05 2024-04-26T03:39:31.134 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.134 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.134 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:31.134 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.567413+0000", 2024-04-26T03:39:31.134 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7e-05 2024-04-26T03:39:31.134 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.134 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.134 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:31.135 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.574497+0000", 2024-04-26T03:39:31.135 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0070843640000000001 2024-04-26T03:39:31.135 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.135 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.135 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:31.135 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.574532+0000", 2024-04-26T03:39:31.135 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.4379999999999999e-05 2024-04-26T03:39:31.135 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.135 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:31.135 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.135 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.135 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.135 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.6s3 35/32 ECSubWrite(tid=0, reqid=client.4556.0:146, at_version=35'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:39:31.135 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.630308+0000", 2024-04-26T03:39:31.136 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 39.497453286000002, 2024-04-26T03:39:31.136 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0078713510000000004, 2024-04-26T03:39:31.136 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:31.136 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:31.136 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:31.136 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:31.136 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.136 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:31.136 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.630308+0000", 2024-04-26T03:39:31.136 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.136 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.136 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.136 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:31.136 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.630308+0000", 2024-04-26T03:39:31.136 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.137 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.137 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.137 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:31.137 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.630309+0000", 2024-04-26T03:39:31.137 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.9500000000000002e-07 2024-04-26T03:39:31.137 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.137 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.137 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:31.137 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.630752+0000", 2024-04-26T03:39:31.137 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00044313600000000001 2024-04-26T03:39:31.137 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.137 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.137 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:31.137 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.630769+0000", 2024-04-26T03:39:31.138 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6541999999999998e-05 2024-04-26T03:39:31.138 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.138 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.138 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:31.138 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.630777+0000", 2024-04-26T03:39:31.138 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.3410000000000003e-06 2024-04-26T03:39:31.138 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.138 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.138 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:31.138 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.630840+0000", 2024-04-26T03:39:31.138 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.3303e-05 2024-04-26T03:39:31.138 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.138 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.138 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:31.138 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.630848+0000", 2024-04-26T03:39:31.139 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2630000000000003e-06 2024-04-26T03:39:31.139 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.139 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.139 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:31.139 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.638146+0000", 2024-04-26T03:39:31.139 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0072979819999999997 2024-04-26T03:39:31.139 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.139 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.139 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:31.139 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.638180+0000", 2024-04-26T03:39:31.139 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.3188999999999997e-05 2024-04-26T03:39:31.139 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.139 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:31.139 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.140 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.140 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.140 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.4s3 40/32 ECSubWrite(tid=22, reqid=client.4556.0:262, at_version=40'14, trim_to=0'0, roll_forward_to=35'12))", 2024-04-26T03:39:31.140 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:56.520557+0000", 2024-04-26T03:39:31.140 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 34.607204240999998, 2024-04-26T03:39:31.140 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.015564889, 2024-04-26T03:39:31.140 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:31.140 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:31.140 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:31.140 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:31.140 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.140 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:31.140 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.520557+0000", 2024-04-26T03:39:31.140 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.140 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.140 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.141 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:31.141 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.520557+0000", 2024-04-26T03:39:31.141 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.141 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.141 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.141 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:31.141 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.520558+0000", 2024-04-26T03:39:31.141 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7399999999999999e-07 2024-04-26T03:39:31.141 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.141 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.141 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:31.141 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.520818+0000", 2024-04-26T03:39:31.141 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00026045899999999999 2024-04-26T03:39:31.141 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.141 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.142 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:31.142 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.520829+0000", 2024-04-26T03:39:31.142 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0701e-05 2024-04-26T03:39:31.142 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.142 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.142 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:31.142 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.520836+0000", 2024-04-26T03:39:31.142 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.9199999999999998e-06 2024-04-26T03:39:31.142 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.142 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.142 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:31.142 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.520901+0000", 2024-04-26T03:39:31.142 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5327000000000007e-05 2024-04-26T03:39:31.142 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.143 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.143 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:31.143 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.520912+0000", 2024-04-26T03:39:31.143 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0747000000000001e-05 2024-04-26T03:39:31.143 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.143 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.143 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:31.143 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.536082+0000", 2024-04-26T03:39:31.143 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.015169775 2024-04-26T03:39:31.143 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.143 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.143 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:31.143 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.536122+0000", 2024-04-26T03:39:31.143 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.0586000000000001e-05 2024-04-26T03:39:31.143 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.144 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:31.144 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.144 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.144 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.144 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.3s5 43/32 ECSubWrite(tid=30, reqid=client.4556.0:314, at_version=43'18, trim_to=0'0, roll_forward_to=43'16))", 2024-04-26T03:39:31.144 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:59.562615+0000", 2024-04-26T03:39:31.144 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 31.565146364, 2024-04-26T03:39:31.144 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.020210662000000001, 2024-04-26T03:39:31.144 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:31.144 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:31.144 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:31.144 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:31.144 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.144 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:31.145 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.562615+0000", 2024-04-26T03:39:31.145 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.145 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.145 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.145 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:31.145 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.562615+0000", 2024-04-26T03:39:31.145 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.145 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.145 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.145 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:31.145 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.562616+0000", 2024-04-26T03:39:31.145 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1699999999999997e-07 2024-04-26T03:39:31.145 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.145 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.145 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:31.146 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.562662+0000", 2024-04-26T03:39:31.146 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.5809999999999997e-05 2024-04-26T03:39:31.146 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.146 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.146 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:31.146 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.562675+0000", 2024-04-26T03:39:31.146 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.332e-05 2024-04-26T03:39:31.146 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.146 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.146 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:31.146 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.562683+0000", 2024-04-26T03:39:31.146 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3889999999999999e-06 2024-04-26T03:39:31.146 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.147 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.147 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:31.147 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.562753+0000", 2024-04-26T03:39:31.147 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.0747999999999994e-05 2024-04-26T03:39:31.147 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.147 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.147 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:31.147 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.562761+0000", 2024-04-26T03:39:31.147 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.4019999999999996e-06 2024-04-26T03:39:31.147 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.147 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.147 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:31.147 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.582798+0000", 2024-04-26T03:39:31.147 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.020036838000000001 2024-04-26T03:39:31.148 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.148 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.148 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:31.148 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.582826+0000", 2024-04-26T03:39:31.148 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8238e-05 2024-04-26T03:39:31.148 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.148 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:31.148 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.148 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.148 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.148 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.3s5 43/32 ECSubWrite(tid=31, reqid=client.4556.0:315, at_version=43'19, trim_to=0'0, roll_forward_to=43'16))", 2024-04-26T03:39:31.148 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:59.564400+0000", 2024-04-26T03:39:31.148 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 31.563361405999999, 2024-04-26T03:39:31.148 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.018851940000000001, 2024-04-26T03:39:31.148 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:31.149 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:31.149 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:31.149 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:31.149 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.149 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:31.149 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.564400+0000", 2024-04-26T03:39:31.149 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.149 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.149 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.149 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:31.149 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.564400+0000", 2024-04-26T03:39:31.149 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.149 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.149 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.150 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:31.150 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.564401+0000", 2024-04-26T03:39:31.150 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.16e-07 2024-04-26T03:39:31.150 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.150 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.150 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:31.150 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.564436+0000", 2024-04-26T03:39:31.150 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.5463000000000002e-05 2024-04-26T03:39:31.150 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.150 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.150 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:31.150 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.582521+0000", 2024-04-26T03:39:31.150 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.018084287000000001 2024-04-26T03:39:31.150 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.151 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.151 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:31.151 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.582532+0000", 2024-04-26T03:39:31.151 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.166e-05 2024-04-26T03:39:31.151 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.151 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.151 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:31.151 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.582599+0000", 2024-04-26T03:39:31.151 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.6721e-05 2024-04-26T03:39:31.151 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.151 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.151 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:31.151 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.582609+0000", 2024-04-26T03:39:31.151 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.8959999999999998e-06 2024-04-26T03:39:31.151 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.151 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.152 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:31.152 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.583226+0000", 2024-04-26T03:39:31.152 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00061683500000000004 2024-04-26T03:39:31.152 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.152 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.152 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:31.152 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.583252+0000", 2024-04-26T03:39:31.152 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6262e-05 2024-04-26T03:39:31.152 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.152 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:31.152 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.152 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.152 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.152 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.fs4 45/32 ECSubWrite(tid=49, reqid=client.4556.0:325, at_version=45'17, trim_to=0'0, roll_forward_to=42'15))", 2024-04-26T03:39:31.152 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:01.554516+0000", 2024-04-26T03:39:31.153 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 29.573245988, 2024-04-26T03:39:31.153 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.012414408, 2024-04-26T03:39:31.153 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:31.153 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:31.153 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:31.154 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:31.154 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.154 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:31.154 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.554516+0000", 2024-04-26T03:39:31.154 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.154 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.154 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.154 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:31.154 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.554516+0000", 2024-04-26T03:39:31.154 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.154 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.154 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.155 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:31.155 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.554516+0000", 2024-04-26T03:39:31.155 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.03e-07 2024-04-26T03:39:31.155 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.156 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.156 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:31.156 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.555044+0000", 2024-04-26T03:39:31.156 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00052827400000000002 2024-04-26T03:39:31.156 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.156 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.157 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:31.157 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.555067+0000", 2024-04-26T03:39:31.157 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2359999999999999e-05 2024-04-26T03:39:31.157 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.157 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.157 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:31.157 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.555077+0000", 2024-04-26T03:39:31.157 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0623e-05 2024-04-26T03:39:31.157 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.157 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.158 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:31.158 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.555156+0000", 2024-04-26T03:39:31.158 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8478000000000003e-05 2024-04-26T03:39:31.158 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.158 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.158 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:31.158 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.555167+0000", 2024-04-26T03:39:31.158 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1243000000000001e-05 2024-04-26T03:39:31.158 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.158 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.158 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:31.158 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.566750+0000", 2024-04-26T03:39:31.158 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.011583115 2024-04-26T03:39:31.159 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.159 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.159 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:31.159 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.566930+0000", 2024-04-26T03:39:31.159 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00017991200000000001 2024-04-26T03:39:31.159 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.159 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:31.159 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.159 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.159 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.159 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.fs4 45/32 ECSubWrite(tid=50, reqid=client.4556.0:326, at_version=45'18, trim_to=0'0, roll_forward_to=42'15))", 2024-04-26T03:39:31.159 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:01.561230+0000", 2024-04-26T03:39:31.159 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 29.566531188999999, 2024-04-26T03:39:31.160 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0088042480000000006, 2024-04-26T03:39:31.160 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:31.160 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:31.160 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:31.160 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:31.160 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.160 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:31.160 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.561230+0000", 2024-04-26T03:39:31.160 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.160 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.160 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.160 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:31.160 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.561230+0000", 2024-04-26T03:39:31.160 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.161 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.161 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.161 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:31.161 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.561231+0000", 2024-04-26T03:39:31.161 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.3099999999999996e-07 2024-04-26T03:39:31.161 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.161 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.161 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:31.161 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.561601+0000", 2024-04-26T03:39:31.161 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000370069 2024-04-26T03:39:31.161 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.161 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.161 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:31.161 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.561616+0000", 2024-04-26T03:39:31.162 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4250000000000001e-05 2024-04-26T03:39:31.162 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.162 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.162 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:31.162 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.561625+0000", 2024-04-26T03:39:31.162 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.9679999999999995e-06 2024-04-26T03:39:31.162 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.162 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.162 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:31.162 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.561850+0000", 2024-04-26T03:39:31.162 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00022588600000000001 2024-04-26T03:39:31.162 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.162 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.163 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:31.163 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.561870+0000", 2024-04-26T03:39:31.163 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9887000000000001e-05 2024-04-26T03:39:31.163 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.163 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.163 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:31.163 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.569991+0000", 2024-04-26T03:39:31.163 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0081205619999999996 2024-04-26T03:39:31.163 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.163 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.163 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:31.163 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.570035+0000", 2024-04-26T03:39:31.163 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.3795000000000003e-05 2024-04-26T03:39:31.163 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.164 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:31.164 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.164 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.164 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.164 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.5s3 48/44 ECSubWrite(tid=45, reqid=osd.15.0:44, at_version=0'0, trim_to=0'0, roll_forward_to=48'40))", 2024-04-26T03:39:31.164 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:04.571316+0000", 2024-04-26T03:39:31.164 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.556445737000001, 2024-04-26T03:39:31.164 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0084868870000000003, 2024-04-26T03:39:31.164 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:31.164 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:31.164 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:31.164 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:31.164 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.164 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:31.165 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.571316+0000", 2024-04-26T03:39:31.165 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.165 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.165 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.165 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:31.165 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.571316+0000", 2024-04-26T03:39:31.165 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.165 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.165 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.165 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:31.165 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.571317+0000", 2024-04-26T03:39:31.165 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.9299999999999996e-07 2024-04-26T03:39:31.165 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.165 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.166 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:31.166 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.571327+0000", 2024-04-26T03:39:31.166 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.9159999999999996e-06 2024-04-26T03:39:31.166 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.166 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.166 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:31.166 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.571339+0000", 2024-04-26T03:39:31.166 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2522e-05 2024-04-26T03:39:31.166 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.166 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.166 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:31.166 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.571348+0000", 2024-04-26T03:39:31.166 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.6400000000000003e-06 2024-04-26T03:39:31.166 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.167 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.167 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:31.167 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.571399+0000", 2024-04-26T03:39:31.167 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.1564999999999998e-05 2024-04-26T03:39:31.167 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.167 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.167 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:31.167 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.571416+0000", 2024-04-26T03:39:31.167 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6410999999999999e-05 2024-04-26T03:39:31.167 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.167 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.167 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:31.167 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.579774+0000", 2024-04-26T03:39:31.167 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0083583630000000006 2024-04-26T03:39:31.168 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.168 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.168 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:31.168 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.579803+0000", 2024-04-26T03:39:31.168 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8577000000000001e-05 2024-04-26T03:39:31.168 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.168 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:31.168 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.168 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.168 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.168 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.13s5 49/44 ECSubWrite(tid=67, reqid=osd.10.0:67, at_version=49'34, trim_to=0'0, roll_forward_to=48'30))", 2024-04-26T03:39:31.168 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:04.685012+0000", 2024-04-26T03:39:31.168 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.442749890999998, 2024-04-26T03:39:31.168 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0092065640000000004, 2024-04-26T03:39:31.168 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:31.169 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:31.169 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:31.169 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:31.169 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.169 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:31.169 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.685012+0000", 2024-04-26T03:39:31.169 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.169 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.169 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.169 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:31.169 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.685012+0000", 2024-04-26T03:39:31.169 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.169 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.169 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.170 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:31.170 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.685012+0000", 2024-04-26T03:39:31.170 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.2500000000000005e-07 2024-04-26T03:39:31.170 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.170 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.170 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:31.170 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.685038+0000", 2024-04-26T03:39:31.170 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5989000000000001e-05 2024-04-26T03:39:31.170 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.170 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.170 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:31.170 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.685050+0000", 2024-04-26T03:39:31.170 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1702000000000001e-05 2024-04-26T03:39:31.170 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.171 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.171 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:31.171 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.685067+0000", 2024-04-26T03:39:31.171 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7147e-05 2024-04-26T03:39:31.171 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.171 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.171 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:31.171 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.685191+0000", 2024-04-26T03:39:31.171 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012426000000000001 2024-04-26T03:39:31.171 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.171 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.171 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:31.171 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.685202+0000", 2024-04-26T03:39:31.171 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0908e-05 2024-04-26T03:39:31.172 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.172 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.172 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:31.172 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.694180+0000", 2024-04-26T03:39:31.172 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0089774060000000003 2024-04-26T03:39:31.172 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.172 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.172 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:31.172 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.694218+0000", 2024-04-26T03:39:31.172 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8526999999999999e-05 2024-04-26T03:39:31.172 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.172 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:31.172 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.172 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.172 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.173 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.6s3 49/44 ECSubWrite(tid=6, reqid=client.4556.0:430, at_version=49'5, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:39:31.173 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:04.689166+0000", 2024-04-26T03:39:31.173 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.438595721999999, 2024-04-26T03:39:31.173 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0091018130000000003, 2024-04-26T03:39:31.173 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:31.173 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:31.173 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:31.173 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:31.173 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.173 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:31.173 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.689166+0000", 2024-04-26T03:39:31.173 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.173 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.173 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.174 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:31.174 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.689166+0000", 2024-04-26T03:39:31.174 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.174 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.174 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.174 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:31.174 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.689166+0000", 2024-04-26T03:39:31.174 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9999999999999998e-07 2024-04-26T03:39:31.174 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.174 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.174 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:31.174 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.689185+0000", 2024-04-26T03:39:31.174 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9137e-05 2024-04-26T03:39:31.175 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.175 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.175 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:31.175 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.689196+0000", 2024-04-26T03:39:31.175 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0963e-05 2024-04-26T03:39:31.175 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.175 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.175 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:31.175 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.689205+0000", 2024-04-26T03:39:31.175 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5469999999999997e-06 2024-04-26T03:39:31.175 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.175 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.175 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:31.175 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.689261+0000", 2024-04-26T03:39:31.176 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.6100999999999997e-05 2024-04-26T03:39:31.176 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.176 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.176 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:31.176 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.689270+0000", 2024-04-26T03:39:31.176 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.2410000000000001e-06 2024-04-26T03:39:31.176 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.176 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.176 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:31.176 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.698221+0000", 2024-04-26T03:39:31.176 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0089505560000000001 2024-04-26T03:39:31.176 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.176 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.176 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:31.176 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.698268+0000", 2024-04-26T03:39:31.177 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6867999999999999e-05 2024-04-26T03:39:31.177 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.177 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:31.177 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.177 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.177 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.177 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.3s5 49/44 ECSubWrite(tid=68, reqid=osd.10.0:66, at_version=0'0, trim_to=0'0, roll_forward_to=49'30))", 2024-04-26T03:39:31.177 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:04.689742+0000", 2024-04-26T03:39:31.177 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.438019372999999, 2024-04-26T03:39:31.177 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0085702499999999997, 2024-04-26T03:39:31.177 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:31.177 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:31.177 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:31.177 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:31.178 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.178 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:31.178 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.689742+0000", 2024-04-26T03:39:31.178 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.178 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.178 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.178 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:31.178 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.689742+0000", 2024-04-26T03:39:31.178 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.178 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.178 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.178 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:31.178 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.689743+0000", 2024-04-26T03:39:31.178 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7000000000000004e-07 2024-04-26T03:39:31.179 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.179 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.179 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:31.179 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.689753+0000", 2024-04-26T03:39:31.179 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0013000000000001e-05 2024-04-26T03:39:31.179 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.179 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.179 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:31.179 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.689775+0000", 2024-04-26T03:39:31.179 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1633000000000001e-05 2024-04-26T03:39:31.179 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.179 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.179 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:31.179 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.689784+0000", 2024-04-26T03:39:31.180 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.696e-06 2024-04-26T03:39:31.180 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.180 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.180 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:31.180 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.689837+0000", 2024-04-26T03:39:31.180 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.2617000000000001e-05 2024-04-26T03:39:31.180 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.180 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.180 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:31.180 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.689848+0000", 2024-04-26T03:39:31.180 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1003e-05 2024-04-26T03:39:31.180 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.180 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.180 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:31.180 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.698260+0000", 2024-04-26T03:39:31.181 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0084119409999999992 2024-04-26T03:39:31.181 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.181 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.181 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:31.181 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.698312+0000", 2024-04-26T03:39:31.181 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.2676999999999997e-05 2024-04-26T03:39:31.181 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.181 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:31.181 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.181 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.181 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.181 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.6s3 49/44 ECSubWrite(tid=7, reqid=client.4556.0:431, at_version=49'6, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:39:31.181 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:04.690515+0000", 2024-04-26T03:39:31.181 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.437246568999999, 2024-04-26T03:39:31.182 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.010272303, 2024-04-26T03:39:31.182 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:31.182 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:31.182 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:31.182 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:31.182 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.182 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:31.182 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.690515+0000", 2024-04-26T03:39:31.182 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.182 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.182 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.182 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:31.182 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.690515+0000", 2024-04-26T03:39:31.182 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.183 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.183 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.183 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:31.183 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.690516+0000", 2024-04-26T03:39:31.183 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.1900000000000002e-07 2024-04-26T03:39:31.183 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.183 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.183 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:31.183 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.690533+0000", 2024-04-26T03:39:31.183 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7017e-05 2024-04-26T03:39:31.183 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.183 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.183 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:31.183 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.690546+0000", 2024-04-26T03:39:31.184 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2894999999999999e-05 2024-04-26T03:39:31.184 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.184 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.184 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:31.184 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.690555+0000", 2024-04-26T03:39:31.184 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.0650000000000005e-06 2024-04-26T03:39:31.184 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.184 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.184 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:31.184 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.690599+0000", 2024-04-26T03:39:31.184 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.4081e-05 2024-04-26T03:39:31.184 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.184 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.184 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:31.185 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.690605+0000", 2024-04-26T03:39:31.185 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.3820000000000001e-06 2024-04-26T03:39:31.185 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.185 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.185 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:31.185 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700747+0000", 2024-04-26T03:39:31.185 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.010142134000000001 2024-04-26T03:39:31.185 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.185 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.185 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:31.185 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700787+0000", 2024-04-26T03:39:31.185 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.0110000000000001e-05 2024-04-26T03:39:31.185 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.185 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:31.186 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.186 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.186 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.186 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.6s3 49/44 ECSubWrite(tid=8, reqid=client.4556.0:432, at_version=49'7, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:39:31.186 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:04.692710+0000", 2024-04-26T03:39:31.186 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.435051556000001, 2024-04-26T03:39:31.186 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0081017780000000005, 2024-04-26T03:39:31.186 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:31.186 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:31.186 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:31.186 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:31.186 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.186 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:31.186 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.692710+0000", 2024-04-26T03:39:31.187 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.187 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.187 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.187 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:31.187 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.692710+0000", 2024-04-26T03:39:31.187 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.187 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.187 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.187 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:31.187 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.692710+0000", 2024-04-26T03:39:31.187 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.91e-07 2024-04-26T03:39:31.187 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.187 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.187 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:31.187 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.692729+0000", 2024-04-26T03:39:31.188 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8766e-05 2024-04-26T03:39:31.188 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.188 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.188 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:31.188 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.692747+0000", 2024-04-26T03:39:31.188 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7410000000000001e-05 2024-04-26T03:39:31.188 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.188 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.188 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:31.188 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.692759+0000", 2024-04-26T03:39:31.188 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2001000000000001e-05 2024-04-26T03:39:31.188 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.188 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.188 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:31.189 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.694804+0000", 2024-04-26T03:39:31.189 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0020459010000000001 2024-04-26T03:39:31.189 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.189 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.189 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:31.189 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.694814+0000", 2024-04-26T03:39:31.189 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0081999999999999e-05 2024-04-26T03:39:31.189 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.189 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.189 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:31.189 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700792+0000", 2024-04-26T03:39:31.189 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0059770680000000003 2024-04-26T03:39:31.189 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.190 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.190 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:31.190 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700812+0000", 2024-04-26T03:39:31.190 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0259e-05 2024-04-26T03:39:31.190 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.190 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:31.190 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.190 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.190 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.190 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.14s3 49/44 ECSubWrite(tid=36, reqid=client.4556.0:439, at_version=49'25, trim_to=0'0, roll_forward_to=46'22))", 2024-04-26T03:39:31.190 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:04.734614+0000", 2024-04-26T03:39:31.190 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.393147759000001, 2024-04-26T03:39:31.190 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0076171040000000004, 2024-04-26T03:39:31.190 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:31.191 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:31.191 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:31.191 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:31.191 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.191 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:31.191 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.734614+0000", 2024-04-26T03:39:31.191 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.191 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.191 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.191 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:31.191 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.734614+0000", 2024-04-26T03:39:31.191 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.191 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.191 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.191 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:31.192 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.734615+0000", 2024-04-26T03:39:31.192 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1739999999999999e-06 2024-04-26T03:39:31.192 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.192 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.192 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:31.192 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.734797+0000", 2024-04-26T03:39:31.192 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00018156300000000001 2024-04-26T03:39:31.192 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.192 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.192 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:31.192 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.734811+0000", 2024-04-26T03:39:31.192 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4106e-05 2024-04-26T03:39:31.192 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.192 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.192 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:31.193 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.734817+0000", 2024-04-26T03:39:31.193 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.6869999999999997e-06 2024-04-26T03:39:31.193 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.193 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.193 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:31.193 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.734895+0000", 2024-04-26T03:39:31.193 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.7619000000000003e-05 2024-04-26T03:39:31.193 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.193 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.193 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:31.193 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.734906+0000", 2024-04-26T03:39:31.193 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1102e-05 2024-04-26T03:39:31.193 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.193 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.194 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:31.194 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.742201+0000", 2024-04-26T03:39:31.194 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0072949199999999999 2024-04-26T03:39:31.194 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.194 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.194 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:31.194 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.742231+0000", 2024-04-26T03:39:31.194 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.9933e-05 2024-04-26T03:39:31.194 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.194 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:31.194 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.194 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.194 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.194 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.13s5 50/44 ECSubWrite(tid=74, reqid=client.4556.0:446, at_version=50'37, trim_to=0'0, roll_forward_to=50'36))", 2024-04-26T03:39:31.195 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:05.702175+0000", 2024-04-26T03:39:31.195 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 25.425586690999999, 2024-04-26T03:39:31.195 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.012835628, 2024-04-26T03:39:31.195 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:31.195 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:31.195 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:31.195 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:31.195 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.195 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:31.195 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.702175+0000", 2024-04-26T03:39:31.195 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.195 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.195 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.195 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:31.196 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.702175+0000", 2024-04-26T03:39:31.196 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.196 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.196 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.196 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:31.196 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.702176+0000", 2024-04-26T03:39:31.196 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1429999999999999e-06 2024-04-26T03:39:31.196 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.196 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.196 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:31.196 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.702378+0000", 2024-04-26T03:39:31.196 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020168399999999999 2024-04-26T03:39:31.196 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.196 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.196 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:31.197 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.702402+0000", 2024-04-26T03:39:31.197 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4139e-05 2024-04-26T03:39:31.197 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.197 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.197 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:31.197 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.702421+0000", 2024-04-26T03:39:31.197 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9012e-05 2024-04-26T03:39:31.197 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.197 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.197 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:31.197 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.702532+0000", 2024-04-26T03:39:31.197 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00011152199999999999 2024-04-26T03:39:31.197 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.197 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.198 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:31.198 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.702544+0000", 2024-04-26T03:39:31.198 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1719000000000001e-05 2024-04-26T03:39:31.198 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.198 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.198 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:31.198 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.714976+0000", 2024-04-26T03:39:31.198 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.012432241 2024-04-26T03:39:31.198 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.198 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.198 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:31.198 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.715011+0000", 2024-04-26T03:39:31.198 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.4168000000000002e-05 2024-04-26T03:39:31.198 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.199 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:31.199 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.199 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.199 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.199 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.13s2 56/55 ECSubWrite(tid=104, reqid=client.4556.0:604, at_version=56'47, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:39:31.199 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:11.775451+0000", 2024-04-26T03:39:31.199 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 19.352310430999999, 2024-04-26T03:39:31.199 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.015718859000000002, 2024-04-26T03:39:31.199 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:31.199 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:31.199 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:31.199 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:31.199 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.199 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:31.200 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:11.775451+0000", 2024-04-26T03:39:31.200 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.200 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.200 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.200 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:31.200 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:11.775451+0000", 2024-04-26T03:39:31.200 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.200 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.200 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.200 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:31.200 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:11.775453+0000", 2024-04-26T03:39:31.200 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.407e-06 2024-04-26T03:39:31.200 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.200 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.201 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:31.201 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:11.777152+0000", 2024-04-26T03:39:31.201 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0016992789999999999 2024-04-26T03:39:31.201 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.201 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.201 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:31.201 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:11.777165+0000", 2024-04-26T03:39:31.201 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3334e-05 2024-04-26T03:39:31.201 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.201 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.201 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:31.201 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:11.777174+0000", 2024-04-26T03:39:31.201 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.8589999999999994e-06 2024-04-26T03:39:31.201 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.202 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.202 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:31.202 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:11.777228+0000", 2024-04-26T03:39:31.202 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.3536999999999999e-05 2024-04-26T03:39:31.202 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.202 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.202 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:31.202 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:11.777234+0000", 2024-04-26T03:39:31.202 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.1460000000000001e-06 2024-04-26T03:39:31.202 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.202 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.202 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:31.203 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:11.791132+0000", 2024-04-26T03:39:31.203 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.013898613000000001 2024-04-26T03:39:31.203 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.203 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.203 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:31.203 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:11.791170+0000", 2024-04-26T03:39:31.203 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7684e-05 2024-04-26T03:39:31.203 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.203 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:31.203 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.203 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.203 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.203 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDPGPush(3.13s2 56/55 [PushOp(3:c944ba3d:::smithi09235267-6:head, version: 52'44, data_included: [262144~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:c944ba3d:::smithi09235267-6:head@52'44, size: 4669440, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-04-26T03:39:31.203 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:11.796104+0000", 2024-04-26T03:39:31.204 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 19.33165799, 2024-04-26T03:39:31.204 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0093439219999999993, 2024-04-26T03:39:31.204 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:31.204 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:31.204 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:31.204 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:31.204 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.204 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:31.204 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:11.796104+0000", 2024-04-26T03:39:31.204 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.204 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.204 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.204 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:31.204 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:11.796104+0000", 2024-04-26T03:39:31.205 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.205 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.205 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.205 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:31.205 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:11.796105+0000", 2024-04-26T03:39:31.205 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5960000000000001e-06 2024-04-26T03:39:31.205 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.205 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.205 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:31.205 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:11.796382+0000", 2024-04-26T03:39:31.205 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00027643200000000001 2024-04-26T03:39:31.205 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.205 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.205 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:31.206 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:11.796402+0000", 2024-04-26T03:39:31.206 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9945999999999998e-05 2024-04-26T03:39:31.206 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.206 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.206 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:31.206 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:11.796428+0000", 2024-04-26T03:39:31.206 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6686000000000002e-05 2024-04-26T03:39:31.206 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.206 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.206 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:31.206 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:11.796544+0000", 2024-04-26T03:39:31.206 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000115377 2024-04-26T03:39:31.206 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.207 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.207 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:31.207 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:11.805448+0000", 2024-04-26T03:39:31.207 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0089038850000000003 2024-04-26T03:39:31.207 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.207 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:31.207 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.207 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.207 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.207 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDPGPush(3.13s2 56/55 [PushOp(3:c8a9e28b:::smithi09235267-30:head, version: 49'32, data_included: [0~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 4, recovery_info: ObjectRecoveryInfo(3:c8a9e28b:::smithi09235267-30:head@49'32, size: 2768896, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-04-26T03:39:31.207 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:11.812061+0000", 2024-04-26T03:39:31.207 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 19.315700708000001, 2024-04-26T03:39:31.207 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.010529920999999999, 2024-04-26T03:39:31.207 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:31.208 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:31.208 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:31.208 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:31.208 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.208 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:31.208 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:11.812061+0000", 2024-04-26T03:39:31.208 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.208 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.208 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.208 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:31.208 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:11.812061+0000", 2024-04-26T03:39:31.208 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.208 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.208 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.209 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:31.209 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:11.812061+0000", 2024-04-26T03:39:31.209 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.8599999999999998e-07 2024-04-26T03:39:31.209 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.209 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.209 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:31.209 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:11.812521+0000", 2024-04-26T03:39:31.209 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00045928899999999999 2024-04-26T03:39:31.209 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.209 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.209 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:31.209 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:11.812534+0000", 2024-04-26T03:39:31.209 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3529999999999999e-05 2024-04-26T03:39:31.209 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.209 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.210 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:31.210 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:11.812548+0000", 2024-04-26T03:39:31.210 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.34e-05 2024-04-26T03:39:31.210 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.210 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.210 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:31.210 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:11.816989+0000", 2024-04-26T03:39:31.210 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0044418030000000002 2024-04-26T03:39:31.210 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.210 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.210 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:31.210 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:11.822591+0000", 2024-04-26T03:39:31.210 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0056014130000000004 2024-04-26T03:39:31.210 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.211 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:31.211 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.211 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.211 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.211 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.14s3 65/44 ECSubWrite(tid=59, reqid=client.4556.0:754, at_version=65'37, trim_to=0'0, roll_forward_to=52'35))", 2024-04-26T03:39:31.211 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:20.731351+0000", 2024-04-26T03:39:31.211 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 10.396410801, 2024-04-26T03:39:31.211 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.013184816, 2024-04-26T03:39:31.211 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:31.211 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:31.211 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:31.211 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:31.211 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.211 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:31.212 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.731351+0000", 2024-04-26T03:39:31.212 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.212 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.212 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.212 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:31.212 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.731351+0000", 2024-04-26T03:39:31.212 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.212 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.212 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.212 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:31.212 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.731351+0000", 2024-04-26T03:39:31.212 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7700000000000001e-07 2024-04-26T03:39:31.212 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.212 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.212 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:31.213 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.731695+0000", 2024-04-26T03:39:31.213 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00034367099999999998 2024-04-26T03:39:31.213 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.213 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.213 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:31.213 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.731712+0000", 2024-04-26T03:39:31.213 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7095000000000001e-05 2024-04-26T03:39:31.213 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.213 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.213 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:31.213 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.731723+0000", 2024-04-26T03:39:31.213 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0778000000000001e-05 2024-04-26T03:39:31.213 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.213 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.213 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:31.214 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.731811+0000", 2024-04-26T03:39:31.214 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.8258999999999996e-05 2024-04-26T03:39:31.214 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.214 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.214 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:31.214 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.731822+0000", 2024-04-26T03:39:31.214 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1620000000000001e-05 2024-04-26T03:39:31.214 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.214 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.215 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:31.215 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.744487+0000", 2024-04-26T03:39:31.215 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.012664233 2024-04-26T03:39:31.215 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.215 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.215 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:31.215 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.744536+0000", 2024-04-26T03:39:31.215 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.8882999999999999e-05 2024-04-26T03:39:31.215 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.215 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:31.215 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.215 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.215 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.215 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.14s3 65/44 ECSubWrite(tid=61, reqid=client.4556.0:756, at_version=65'39, trim_to=0'0, roll_forward_to=52'35))", 2024-04-26T03:39:31.215 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:20.745074+0000", 2024-04-26T03:39:31.216 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 10.382687671999999, 2024-04-26T03:39:31.216 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0084106560000000007, 2024-04-26T03:39:31.216 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:31.216 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:31.216 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:31.216 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:31.216 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.216 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:31.216 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.745074+0000", 2024-04-26T03:39:31.216 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.216 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.216 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.216 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:31.216 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.745074+0000", 2024-04-26T03:39:31.217 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.217 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.217 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.217 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:31.217 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.745075+0000", 2024-04-26T03:39:31.217 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.9800000000000002e-07 2024-04-26T03:39:31.217 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.217 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.217 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:31.217 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.745228+0000", 2024-04-26T03:39:31.217 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000153194 2024-04-26T03:39:31.217 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.217 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.217 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:31.218 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.745244+0000", 2024-04-26T03:39:31.218 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6388e-05 2024-04-26T03:39:31.218 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.218 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.218 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:31.218 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.745254+0000", 2024-04-26T03:39:31.218 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.8069999999999994e-06 2024-04-26T03:39:31.218 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.218 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.218 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:31.218 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.745317+0000", 2024-04-26T03:39:31.218 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.2525000000000002e-05 2024-04-26T03:39:31.218 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.218 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.219 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:31.219 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.745324+0000", 2024-04-26T03:39:31.219 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.9399999999999996e-06 2024-04-26T03:39:31.219 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.219 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.219 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:31.219 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.753447+0000", 2024-04-26T03:39:31.219 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0081230109999999994 2024-04-26T03:39:31.219 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.219 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.219 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:31.219 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.753485+0000", 2024-04-26T03:39:31.219 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7793000000000002e-05 2024-04-26T03:39:31.219 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.219 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:31.220 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.220 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.220 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:31.220 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:39:31.221 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-26T03:39:31.222 INFO:tasks.ceph.osd.15.smithi092.stderr:2024-04-26T03:39:31.220+0000 7f6d12dec640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 31824) UID: 0 2024-04-26T03:39:31.322 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:39:31.321+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:39:31.329 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:39:31.329 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-04-26T03:39:31.330 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 0 2024-04-26T03:39:31.330 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:39:31.338 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-26T03:39:31.424 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:39:31.422+0000 7fb78c544640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31956) UID: 0 2024-04-26T03:39:31.444 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:39:31.444 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-04-26T03:39:31.444 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-04-26T03:39:31.444 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-04-26T03:39:31.445 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:39:31.453 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-26T03:39:31.525 INFO:tasks.ceph.osd.10.smithi066.stderr:2024-04-26T03:39:31.524+0000 7f130fcc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 31763) UID: 0 2024-04-26T03:39:31.562 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:39:31.562 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-04-26T03:39:31.562 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-04-26T03:39:31.562 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-04-26T03:39:31.562 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.562 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4213.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-04-26T03:39:31.562 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:33.315386+0000", 2024-04-26T03:39:31.562 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 58.243768318999997, 2024-04-26T03:39:31.562 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.075394833, 2024-04-26T03:39:31.562 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:31.563 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:31.563 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:31.563 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:31.563 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4213", 2024-04-26T03:39:31.563 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/331201604", 2024-04-26T03:39:31.563 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 1 2024-04-26T03:39:31.563 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.563 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:31.563 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.563 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:31.563 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:33.315386+0000", 2024-04-26T03:39:31.563 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.563 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.563 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.564 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:31.564 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:33.315386+0000", 2024-04-26T03:39:31.564 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.564 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.564 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.564 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:31.564 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:33.315388+0000", 2024-04-26T03:39:31.564 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.897e-06 2024-04-26T03:39:31.564 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.564 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.564 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:31.564 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:33.315417+0000", 2024-04-26T03:39:31.564 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8983e-05 2024-04-26T03:39:31.564 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.565 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.565 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:31.565 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:33.315447+0000", 2024-04-26T03:39:31.565 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.9920000000000002e-05 2024-04-26T03:39:31.565 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.565 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.565 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:31.566 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:33.315460+0000", 2024-04-26T03:39:31.566 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3064999999999999e-05 2024-04-26T03:39:31.566 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.566 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.566 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:31.566 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:33.315770+0000", 2024-04-26T03:39:31.566 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000309687 2024-04-26T03:39:31.566 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.566 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.566 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for peered", 2024-04-26T03:39:31.566 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:33.315784+0000", 2024-04-26T03:39:31.566 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3767e-05 2024-04-26T03:39:31.567 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.567 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.567 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:31.567 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.389505+0000", 2024-04-26T03:39:31.567 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0737211259999999 2024-04-26T03:39:31.567 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.567 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.567 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:31.567 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.389667+0000", 2024-04-26T03:39:31.567 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000162435 2024-04-26T03:39:31.567 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.567 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.567 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 5", 2024-04-26T03:39:31.567 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.389837+0000", 2024-04-26T03:39:31.568 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00016988999999999999 2024-04-26T03:39:31.568 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.568 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.568 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-04-26T03:39:31.568 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.390620+0000", 2024-04-26T03:39:31.568 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000782941 2024-04-26T03:39:31.568 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.568 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.568 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-04-26T03:39:31.568 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.390712+0000", 2024-04-26T03:39:31.568 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.2119000000000004e-05 2024-04-26T03:39:31.568 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.568 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.568 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:31.569 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.390751+0000", 2024-04-26T03:39:31.569 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9168999999999997e-05 2024-04-26T03:39:31.569 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.569 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.569 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:31.569 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.390781+0000", 2024-04-26T03:39:31.569 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.9833999999999999e-05 2024-04-26T03:39:31.569 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.569 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:31.569 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.569 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.569 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.569 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.4s4 40/32 ECSubWrite(tid=22, reqid=client.4556.0:262, at_version=40'14, trim_to=0'0, roll_forward_to=35'12))", 2024-04-26T03:39:31.569 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:56.521692+0000", 2024-04-26T03:39:31.570 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 35.037462746999999, 2024-04-26T03:39:31.570 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.014776724, 2024-04-26T03:39:31.570 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:31.570 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:31.570 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:31.570 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:31.570 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.570 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:31.570 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.521692+0000", 2024-04-26T03:39:31.570 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.570 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.570 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.570 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:31.570 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.521692+0000", 2024-04-26T03:39:31.571 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.571 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.571 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.571 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:31.571 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.521692+0000", 2024-04-26T03:39:31.571 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.4999999999999998e-07 2024-04-26T03:39:31.571 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.571 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.571 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:31.571 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.522058+0000", 2024-04-26T03:39:31.571 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00036550299999999999 2024-04-26T03:39:31.571 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.572 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.572 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:31.572 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.522068+0000", 2024-04-26T03:39:31.572 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0263e-05 2024-04-26T03:39:31.572 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.572 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.572 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:31.572 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.522074+0000", 2024-04-26T03:39:31.572 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7209999999999999e-06 2024-04-26T03:39:31.572 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.572 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.572 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:31.572 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.522144+0000", 2024-04-26T03:39:31.572 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.0240999999999994e-05 2024-04-26T03:39:31.573 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.573 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.573 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:31.573 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.522154+0000", 2024-04-26T03:39:31.573 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0457e-05 2024-04-26T03:39:31.573 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.573 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.573 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:31.573 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.536432+0000", 2024-04-26T03:39:31.573 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.014278107999999999 2024-04-26T03:39:31.573 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.573 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.573 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:31.573 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.536469+0000", 2024-04-26T03:39:31.574 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.6081000000000002e-05 2024-04-26T03:39:31.574 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.574 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:31.574 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.574 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.574 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.574 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:266 3.8s0 3:1b3fe483:::smithi09235267-19:head [append 0~737280] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T03:39:31.574 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:56.527487+0000", 2024-04-26T03:39:31.574 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 35.031667108000001, 2024-04-26T03:39:31.574 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.015863135, 2024-04-26T03:39:31.574 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:31.574 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:31.574 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:31.574 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:31.575 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:31.575 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:31.575 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 266 2024-04-26T03:39:31.575 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.575 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:31.575 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.575 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:31.575 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.527487+0000", 2024-04-26T03:39:31.575 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.575 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.575 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.575 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:31.575 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.527487+0000", 2024-04-26T03:39:31.575 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.576 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.576 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.576 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:31.576 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.527489+0000", 2024-04-26T03:39:31.576 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5239999999999999e-06 2024-04-26T03:39:31.576 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.576 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.576 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:31.576 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.528331+0000", 2024-04-26T03:39:31.576 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00084205300000000005 2024-04-26T03:39:31.576 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.576 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.576 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:31.576 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.528347+0000", 2024-04-26T03:39:31.577 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5882000000000001e-05 2024-04-26T03:39:31.577 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.577 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.577 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:31.577 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.528353+0000", 2024-04-26T03:39:31.577 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.9800000000000003e-06 2024-04-26T03:39:31.577 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.577 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.577 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:31.577 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.528421+0000", 2024-04-26T03:39:31.577 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.8608000000000004e-05 2024-04-26T03:39:31.577 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.577 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.577 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:31.578 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.528526+0000", 2024-04-26T03:39:31.578 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00010430200000000001 2024-04-26T03:39:31.578 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.578 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.578 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:31.578 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.533515+0000", 2024-04-26T03:39:31.578 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0049888149999999997 2024-04-26T03:39:31.578 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.578 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.578 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:31.578 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.542563+0000", 2024-04-26T03:39:31.578 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0090485449999999999 2024-04-26T03:39:31.578 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.578 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.579 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:31.579 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.543246+0000", 2024-04-26T03:39:31.579 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00068309800000000004 2024-04-26T03:39:31.579 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.579 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.579 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:31.579 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.543351+0000", 2024-04-26T03:39:31.579 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00010432799999999999 2024-04-26T03:39:31.579 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.579 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:31.579 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.579 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.579 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.579 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:268 3.8s0 3:1b3fe483:::smithi09235267-19:head [setxattr _header (56) in=7b] snapc 4=[2] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T03:39:31.580 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:56.531867+0000", 2024-04-26T03:39:31.580 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 35.027287682000001, 2024-04-26T03:39:31.580 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.014163701000000001, 2024-04-26T03:39:31.580 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:31.580 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:31.580 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:31.580 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:31.580 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:31.580 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:31.580 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 268 2024-04-26T03:39:31.580 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.580 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:31.580 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.580 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:31.580 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.531867+0000", 2024-04-26T03:39:31.581 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.581 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.581 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.581 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:31.581 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.531867+0000", 2024-04-26T03:39:31.581 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.581 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.581 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.581 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:31.581 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.531867+0000", 2024-04-26T03:39:31.581 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.2099999999999996e-07 2024-04-26T03:39:31.581 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.581 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.582 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:31.582 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.531872+0000", 2024-04-26T03:39:31.582 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.578e-06 2024-04-26T03:39:31.582 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.582 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.582 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:31.582 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.531881+0000", 2024-04-26T03:39:31.582 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.9369999999999993e-06 2024-04-26T03:39:31.582 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.582 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.582 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:31.582 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.531887+0000", 2024-04-26T03:39:31.582 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.9150000000000001e-06 2024-04-26T03:39:31.582 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.583 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.583 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:31.583 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.541782+0000", 2024-04-26T03:39:31.583 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0098949529999999997 2024-04-26T03:39:31.583 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.583 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.583 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:31.583 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.541877+0000", 2024-04-26T03:39:31.583 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.5051999999999997e-05 2024-04-26T03:39:31.583 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.583 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.583 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:31.583 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.542321+0000", 2024-04-26T03:39:31.583 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000443986 2024-04-26T03:39:31.583 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.584 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.584 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:31.584 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.543061+0000", 2024-04-26T03:39:31.584 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00074033599999999997 2024-04-26T03:39:31.584 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.584 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.584 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:31.584 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.545981+0000", 2024-04-26T03:39:31.584 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.002919919 2024-04-26T03:39:31.584 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.584 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.584 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:31.585 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.546031+0000", 2024-04-26T03:39:31.585 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.9404000000000003e-05 2024-04-26T03:39:31.585 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.585 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:31.585 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.585 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.585 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.585 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:269 3.8s0 3:1b3fe483:::smithi09235267-19:head [read 0~1] snapc 0=[] RETRY=1 ondisk+retry+read+rwordered+known_if_redirected+supports_pool_eio e40)", 2024-04-26T03:39:31.585 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:56.531896+0000", 2024-04-26T03:39:31.585 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 35.027258682999999, 2024-04-26T03:39:31.585 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.017279658999999999, 2024-04-26T03:39:31.585 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:31.585 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:31.585 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:31.586 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:31.586 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:31.586 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:31.586 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 269 2024-04-26T03:39:31.586 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.586 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:31.586 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.586 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:31.586 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.531896+0000", 2024-04-26T03:39:31.586 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.586 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.586 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.586 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:31.586 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.531896+0000", 2024-04-26T03:39:31.587 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.587 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.587 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.587 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:31.587 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.531896+0000", 2024-04-26T03:39:31.587 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7599999999999997e-07 2024-04-26T03:39:31.587 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.587 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.587 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:31.587 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.531899+0000", 2024-04-26T03:39:31.587 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8550000000000002e-06 2024-04-26T03:39:31.587 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.587 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.588 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:31.588 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.531907+0000", 2024-04-26T03:39:31.588 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9349999999999994e-06 2024-04-26T03:39:31.588 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.588 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.588 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:31.588 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.531913+0000", 2024-04-26T03:39:31.588 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7540000000000003e-06 2024-04-26T03:39:31.588 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.588 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.588 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:31.588 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.542916+0000", 2024-04-26T03:39:31.588 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.01100284 2024-04-26T03:39:31.589 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.589 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.589 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:31.589 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.543008+0000", 2024-04-26T03:39:31.589 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1822000000000003e-05 2024-04-26T03:39:31.589 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.589 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.589 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:31.589 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.546345+0000", 2024-04-26T03:39:31.589 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0033376539999999998 2024-04-26T03:39:31.589 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.589 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.589 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:31.589 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.546438+0000", 2024-04-26T03:39:31.589 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.2394999999999999e-05 2024-04-26T03:39:31.590 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.590 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.590 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:31.590 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.549176+0000", 2024-04-26T03:39:31.590 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0027378279999999999 2024-04-26T03:39:31.590 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.590 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:31.590 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.590 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.590 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.590 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:443 3.18s0 3:1de5c702:::smithi09235267-2:head [copy-from ver 3] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-04-26T03:39:31.590 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:04.728363+0000", 2024-04-26T03:39:31.591 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.830791904000002, 2024-04-26T03:39:31.591 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.016662854000000001, 2024-04-26T03:39:31.591 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:31.591 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:31.591 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:31.591 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:31.591 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:31.591 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:31.591 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 443 2024-04-26T03:39:31.591 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.591 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:31.591 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.591 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:31.591 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728363+0000", 2024-04-26T03:39:31.591 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.592 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.592 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.592 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:31.592 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728363+0000", 2024-04-26T03:39:31.592 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.592 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.592 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.592 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:31.592 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728364+0000", 2024-04-26T03:39:31.592 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4190000000000001e-06 2024-04-26T03:39:31.592 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.592 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.592 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:31.593 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728370+0000", 2024-04-26T03:39:31.593 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.5400000000000003e-06 2024-04-26T03:39:31.593 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.593 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.593 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:31.593 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728381+0000", 2024-04-26T03:39:31.593 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0998e-05 2024-04-26T03:39:31.593 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.593 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.593 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:31.593 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728387+0000", 2024-04-26T03:39:31.593 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.2430000000000002e-06 2024-04-26T03:39:31.593 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.593 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.593 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:31.594 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728450+0000", 2024-04-26T03:39:31.594 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.3639000000000005e-05 2024-04-26T03:39:31.594 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.594 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.594 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:31.594 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728581+0000", 2024-04-26T03:39:31.594 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013051100000000001 2024-04-26T03:39:31.594 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.594 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.594 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:31.594 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.734449+0000", 2024-04-26T03:39:31.594 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0058678250000000001 2024-04-26T03:39:31.594 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.595 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.595 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:31.595 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.744550+0000", 2024-04-26T03:39:31.595 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.010101044999999999 2024-04-26T03:39:31.595 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.595 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.595 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:31.595 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.744977+0000", 2024-04-26T03:39:31.595 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00042698699999999998 2024-04-26T03:39:31.595 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.595 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.595 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:31.595 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.745025+0000", 2024-04-26T03:39:31.595 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.8646999999999997e-05 2024-04-26T03:39:31.596 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.596 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:31.596 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.596 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.596 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.596 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:444 3.18s0 3:1de5c702:::smithi09235267-2:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e49)", 2024-04-26T03:39:31.596 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:04.728401+0000", 2024-04-26T03:39:31.596 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.830753821999998, 2024-04-26T03:39:31.596 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.01449813, 2024-04-26T03:39:31.596 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:31.596 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:31.596 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:31.596 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:31.596 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:31.596 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:31.597 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 444 2024-04-26T03:39:31.597 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.597 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:31.597 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.597 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:31.597 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728401+0000", 2024-04-26T03:39:31.597 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.597 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.597 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.597 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:31.597 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728401+0000", 2024-04-26T03:39:31.597 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.597 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.597 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.597 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:31.598 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728401+0000", 2024-04-26T03:39:31.598 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.5499999999999998e-07 2024-04-26T03:39:31.598 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.598 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.598 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:31.598 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728404+0000", 2024-04-26T03:39:31.598 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4959999999999999e-06 2024-04-26T03:39:31.598 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.598 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.598 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:31.598 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728424+0000", 2024-04-26T03:39:31.598 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0307999999999999e-05 2024-04-26T03:39:31.598 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.599 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.599 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:31.599 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728430+0000", 2024-04-26T03:39:31.599 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.5459999999999999e-06 2024-04-26T03:39:31.599 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.599 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.599 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:31.599 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728698+0000", 2024-04-26T03:39:31.599 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00026843 2024-04-26T03:39:31.599 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.599 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.599 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for blocked object", 2024-04-26T03:39:31.599 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728723+0000", 2024-04-26T03:39:31.599 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5258000000000001e-05 2024-04-26T03:39:31.600 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.600 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.600 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:31.600 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.742719+0000", 2024-04-26T03:39:31.600 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.013995254 2024-04-26T03:39:31.600 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.600 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.600 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:31.600 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.742810+0000", 2024-04-26T03:39:31.600 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1774999999999995e-05 2024-04-26T03:39:31.600 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.600 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.600 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:31.600 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.742899+0000", 2024-04-26T03:39:31.600 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.8508e-05 2024-04-26T03:39:31.601 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.601 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:31.601 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.601 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.601 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.601 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:484 3.18s0 3:1b3fe483:::smithi09235267-19:head [append 0~425984] snapc 9=[9] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e51)", 2024-04-26T03:39:31.601 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:06.700875+0000", 2024-04-26T03:39:31.601 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 24.858279402000001, 2024-04-26T03:39:31.601 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.016679452000000001, 2024-04-26T03:39:31.601 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:31.601 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:31.601 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:31.601 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:31.602 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:31.602 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:31.602 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 484 2024-04-26T03:39:31.602 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.602 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:31.602 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.602 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:31.602 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.700875+0000", 2024-04-26T03:39:31.602 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.602 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.602 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.602 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:31.602 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.700875+0000", 2024-04-26T03:39:31.602 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.603 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.603 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.603 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:31.603 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.700876+0000", 2024-04-26T03:39:31.603 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.2600000000000001e-07 2024-04-26T03:39:31.603 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.603 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.603 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:31.603 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.701387+0000", 2024-04-26T03:39:31.603 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00051046299999999995 2024-04-26T03:39:31.603 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.603 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.603 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:31.603 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.701400+0000", 2024-04-26T03:39:31.604 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3249e-05 2024-04-26T03:39:31.604 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.604 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.604 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:31.604 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.701419+0000", 2024-04-26T03:39:31.604 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.969e-05 2024-04-26T03:39:31.604 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.604 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.604 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:31.604 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.701489+0000", 2024-04-26T03:39:31.604 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.9465e-05 2024-04-26T03:39:31.604 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.604 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.605 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op must wait for map", 2024-04-26T03:39:31.605 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.701494+0000", 2024-04-26T03:39:31.605 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.215e-06 2024-04-26T03:39:31.605 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.605 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.605 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:31.605 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.703613+0000", 2024-04-26T03:39:31.605 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0021187630000000001 2024-04-26T03:39:31.605 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.605 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.605 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:31.605 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.703686+0000", 2024-04-26T03:39:31.605 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.2741e-05 2024-04-26T03:39:31.605 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.606 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.606 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:31.606 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.704604+0000", 2024-04-26T03:39:31.606 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00091841999999999998 2024-04-26T03:39:31.606 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.606 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.606 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:31.606 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.717233+0000", 2024-04-26T03:39:31.606 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.012629067000000001 2024-04-26T03:39:31.606 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.606 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.606 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:31.606 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.717494+0000", 2024-04-26T03:39:31.606 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00026084900000000001 2024-04-26T03:39:31.607 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.607 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.607 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:31.607 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.717555+0000", 2024-04-26T03:39:31.607 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.0603999999999998e-05 2024-04-26T03:39:31.607 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.607 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:31.607 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.607 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.607 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.607 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:485 3.18s0 3:1b3fe483:::smithi09235267-19:head [append 0~786432] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-26T03:39:31.607 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:06.704369+0000", 2024-04-26T03:39:31.607 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 24.854785228000001, 2024-04-26T03:39:31.607 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.013472728999999999, 2024-04-26T03:39:31.608 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:31.608 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:31.608 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:31.608 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:31.608 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:31.608 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:31.608 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 485 2024-04-26T03:39:31.608 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.608 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:31.608 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.608 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:31.608 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.704369+0000", 2024-04-26T03:39:31.608 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.608 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.608 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.609 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:31.609 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.704369+0000", 2024-04-26T03:39:31.609 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.609 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.609 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.609 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:31.609 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.704371+0000", 2024-04-26T03:39:31.609 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3e-06 2024-04-26T03:39:31.609 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.609 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.609 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:31.609 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.705189+0000", 2024-04-26T03:39:31.609 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00081839399999999996 2024-04-26T03:39:31.609 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.610 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.610 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:31.610 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.705209+0000", 2024-04-26T03:39:31.610 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0224000000000002e-05 2024-04-26T03:39:31.610 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.610 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.610 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:31.610 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.705221+0000", 2024-04-26T03:39:31.610 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1766000000000001e-05 2024-04-26T03:39:31.610 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.610 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.610 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:31.610 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.708313+0000", 2024-04-26T03:39:31.610 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0030923360000000002 2024-04-26T03:39:31.611 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.611 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.611 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:31.611 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.708466+0000", 2024-04-26T03:39:31.611 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00015266699999999999 2024-04-26T03:39:31.611 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.611 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.611 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:31.611 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.709499+0000", 2024-04-26T03:39:31.611 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001032977 2024-04-26T03:39:31.611 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.611 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.611 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:31.612 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.717345+0000", 2024-04-26T03:39:31.612 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0078458989999999999 2024-04-26T03:39:31.612 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.612 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.612 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:31.612 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.717805+0000", 2024-04-26T03:39:31.612 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00045974200000000002 2024-04-26T03:39:31.612 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.612 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.612 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:31.612 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.717842+0000", 2024-04-26T03:39:31.612 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7423999999999999e-05 2024-04-26T03:39:31.612 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.612 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:31.612 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.613 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.613 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.613 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:487 3.18s0 3:1b3fe483:::smithi09235267-19:head [setxattr _header (56) in=7b] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-26T03:39:31.613 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:06.707300+0000", 2024-04-26T03:39:31.613 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 24.851854290999999, 2024-04-26T03:39:31.613 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.013837118000000001, 2024-04-26T03:39:31.613 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:31.613 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:31.613 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:31.613 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:31.613 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:31.613 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:31.613 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 487 2024-04-26T03:39:31.613 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.614 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:31.614 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.614 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:31.614 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.707300+0000", 2024-04-26T03:39:31.614 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.614 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.614 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.614 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:31.614 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.707300+0000", 2024-04-26T03:39:31.614 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.614 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.614 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.614 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:31.614 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.707301+0000", 2024-04-26T03:39:31.615 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.0800000000000003e-07 2024-04-26T03:39:31.615 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.615 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.615 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:31.615 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.707306+0000", 2024-04-26T03:39:31.615 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.2610000000000001e-06 2024-04-26T03:39:31.615 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.615 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.615 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:31.615 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.707317+0000", 2024-04-26T03:39:31.615 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0825999999999999e-05 2024-04-26T03:39:31.615 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.615 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.615 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:31.616 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.707325+0000", 2024-04-26T03:39:31.616 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.2830000000000004e-06 2024-04-26T03:39:31.616 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.616 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.616 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:31.616 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.717877+0000", 2024-04-26T03:39:31.616 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.010552404 2024-04-26T03:39:31.616 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.616 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.616 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:31.616 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.717938+0000", 2024-04-26T03:39:31.616 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.0908e-05 2024-04-26T03:39:31.616 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.616 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.617 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:31.617 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.718210+0000", 2024-04-26T03:39:31.617 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00027242199999999997 2024-04-26T03:39:31.617 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.617 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.617 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:31.617 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.719865+0000", 2024-04-26T03:39:31.617 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001654555 2024-04-26T03:39:31.617 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.617 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.617 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:31.617 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.721044+0000", 2024-04-26T03:39:31.617 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0011796039999999999 2024-04-26T03:39:31.617 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.618 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.618 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:31.618 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.721137+0000", 2024-04-26T03:39:31.618 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.2947000000000003e-05 2024-04-26T03:39:31.618 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.618 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:31.618 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.618 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.618 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.618 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:488 3.18s0 3:1b3fe483:::smithi09235267-19:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e51)", 2024-04-26T03:39:31.618 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:06.707338+0000", 2024-04-26T03:39:31.618 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 24.851816723999999, 2024-04-26T03:39:31.619 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.015400540000000001, 2024-04-26T03:39:31.619 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:31.619 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:31.619 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:31.619 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:31.619 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:31.619 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:31.619 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 488 2024-04-26T03:39:31.619 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.619 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:31.619 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.619 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:31.619 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.707338+0000", 2024-04-26T03:39:31.619 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.620 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.620 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.620 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:31.620 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.707338+0000", 2024-04-26T03:39:31.620 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.620 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.620 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.620 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:31.620 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.707338+0000", 2024-04-26T03:39:31.620 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5099999999999999e-07 2024-04-26T03:39:31.620 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.620 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.620 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:31.620 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.707342+0000", 2024-04-26T03:39:31.621 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8879999999999999e-06 2024-04-26T03:39:31.621 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.621 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.621 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:31.621 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.707352+0000", 2024-04-26T03:39:31.621 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.4490000000000005e-06 2024-04-26T03:39:31.621 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.621 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.621 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:31.621 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.707360+0000", 2024-04-26T03:39:31.621 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.3229999999999998e-06 2024-04-26T03:39:31.621 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.621 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.621 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:31.621 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.718657+0000", 2024-04-26T03:39:31.622 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.011297019 2024-04-26T03:39:31.622 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.622 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.622 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:31.622 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.718712+0000", 2024-04-26T03:39:31.622 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.5241000000000002e-05 2024-04-26T03:39:31.622 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.622 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.622 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:31.622 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.721424+0000", 2024-04-26T03:39:31.622 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0027115529999999998 2024-04-26T03:39:31.622 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.622 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.622 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:31.623 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.721472+0000", 2024-04-26T03:39:31.623 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.8174999999999999e-05 2024-04-26T03:39:31.623 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.623 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.623 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:31.623 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.722738+0000", 2024-04-26T03:39:31.623 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001266241 2024-04-26T03:39:31.623 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.623 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:31.623 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.623 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.623 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.623 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.13s3 56/55 ECSubWrite(tid=104, reqid=client.4556.0:604, at_version=56'47, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:39:31.624 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:11.775017+0000", 2024-04-26T03:39:31.624 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 19.784137390000001, 2024-04-26T03:39:31.624 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.016407774, 2024-04-26T03:39:31.624 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:31.624 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:31.624 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:31.624 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:31.624 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.624 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:31.624 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:11.775017+0000", 2024-04-26T03:39:31.624 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.624 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.624 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.624 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:31.625 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:11.775017+0000", 2024-04-26T03:39:31.625 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.625 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.625 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.625 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:31.625 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:11.775019+0000", 2024-04-26T03:39:31.625 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5659999999999999e-06 2024-04-26T03:39:31.625 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.625 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.625 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:31.625 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:11.776937+0000", 2024-04-26T03:39:31.625 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001918105 2024-04-26T03:39:31.626 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.626 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.626 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:31.626 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:11.776949+0000", 2024-04-26T03:39:31.626 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1912999999999999e-05 2024-04-26T03:39:31.626 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.626 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.626 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:31.626 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:11.776955+0000", 2024-04-26T03:39:31.626 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.9390000000000002e-06 2024-04-26T03:39:31.626 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.626 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.626 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:31.626 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:11.777001+0000", 2024-04-26T03:39:31.627 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.5939999999999997e-05 2024-04-26T03:39:31.627 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.627 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.627 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:31.627 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:11.777012+0000", 2024-04-26T03:39:31.627 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.096e-05 2024-04-26T03:39:31.627 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.627 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.627 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:31.627 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:11.791372+0000", 2024-04-26T03:39:31.627 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.014360892 2024-04-26T03:39:31.627 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.627 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.628 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:31.628 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:11.791425+0000", 2024-04-26T03:39:31.628 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.2459e-05 2024-04-26T03:39:31.628 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.628 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:31.628 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.628 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.628 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.628 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:703 3.10s0 3:0a8403ef:::smithi09235267-5:head [append 0~557056] snapc f=[f,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-04-26T03:39:31.628 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:15.955111+0000", 2024-04-26T03:39:31.628 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 15.604044022, 2024-04-26T03:39:31.628 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.020941576, 2024-04-26T03:39:31.629 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:31.629 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:31.629 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:31.629 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:31.629 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:31.629 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:31.629 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 703 2024-04-26T03:39:31.629 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.629 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:31.629 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.629 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:31.629 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.955111+0000", 2024-04-26T03:39:31.629 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.629 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.630 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.630 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:31.630 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.955111+0000", 2024-04-26T03:39:31.630 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.630 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.630 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.630 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:31.630 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.955112+0000", 2024-04-26T03:39:31.630 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.606e-06 2024-04-26T03:39:31.630 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.630 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.630 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:31.630 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.957050+0000", 2024-04-26T03:39:31.631 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001937872 2024-04-26T03:39:31.631 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.631 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.631 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:31.631 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.957071+0000", 2024-04-26T03:39:31.631 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1271999999999999e-05 2024-04-26T03:39:31.631 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.631 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.631 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:31.631 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.957082+0000", 2024-04-26T03:39:31.631 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0708e-05 2024-04-26T03:39:31.631 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.631 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.631 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:31.631 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.957147+0000", 2024-04-26T03:39:31.632 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5135999999999994e-05 2024-04-26T03:39:31.632 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.632 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.632 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:31.632 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.957234+0000", 2024-04-26T03:39:31.632 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.6871000000000002e-05 2024-04-26T03:39:31.632 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.632 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.632 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:31.632 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.958104+0000", 2024-04-26T03:39:31.632 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00087001500000000002 2024-04-26T03:39:31.632 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.632 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.632 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:31.633 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.973753+0000", 2024-04-26T03:39:31.633 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.015648935999999999 2024-04-26T03:39:31.633 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.633 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.633 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:31.633 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.975989+0000", 2024-04-26T03:39:31.633 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.002236044 2024-04-26T03:39:31.633 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.633 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.633 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:31.633 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.976052+0000", 2024-04-26T03:39:31.633 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.3115999999999997e-05 2024-04-26T03:39:31.633 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.634 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:31.634 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.634 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.634 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.634 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:704 3.10s0 3:0a8403ef:::smithi09235267-5:head [append 0~475136] snapc f=[f,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-04-26T03:39:31.634 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:15.958967+0000", 2024-04-26T03:39:31.634 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 15.600187980999999, 2024-04-26T03:39:31.634 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.018099097000000001, 2024-04-26T03:39:31.634 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:31.634 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:31.634 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:31.634 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:31.634 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:31.635 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:31.635 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 704 2024-04-26T03:39:31.635 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.635 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:31.635 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.635 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:31.635 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.958967+0000", 2024-04-26T03:39:31.635 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.635 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.635 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.635 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:31.635 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.958967+0000", 2024-04-26T03:39:31.635 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.635 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.635 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.636 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:31.636 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.958968+0000", 2024-04-26T03:39:31.636 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.657e-06 2024-04-26T03:39:31.636 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.636 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.636 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:31.636 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.959507+0000", 2024-04-26T03:39:31.636 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00053840699999999997 2024-04-26T03:39:31.636 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.636 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.636 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:31.636 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.959523+0000", 2024-04-26T03:39:31.636 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6515999999999999e-05 2024-04-26T03:39:31.637 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.637 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.637 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:31.637 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.959531+0000", 2024-04-26T03:39:31.637 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9279999999999993e-06 2024-04-26T03:39:31.637 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.637 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.637 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:31.637 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.961991+0000", 2024-04-26T03:39:31.637 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0024603680000000001 2024-04-26T03:39:31.637 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.637 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.637 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:31.637 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.962070+0000", 2024-04-26T03:39:31.638 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8689999999999994e-05 2024-04-26T03:39:31.638 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.638 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.638 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:31.638 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.962702+0000", 2024-04-26T03:39:31.638 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00063166399999999999 2024-04-26T03:39:31.638 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.638 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.638 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:31.638 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.973754+0000", 2024-04-26T03:39:31.638 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.011052622 2024-04-26T03:39:31.638 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.638 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.638 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:31.639 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.977010+0000", 2024-04-26T03:39:31.639 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0032559260000000001 2024-04-26T03:39:31.639 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.639 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.639 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:31.639 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.977066+0000", 2024-04-26T03:39:31.639 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.5318999999999997e-05 2024-04-26T03:39:31.639 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.639 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:31.639 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.639 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.639 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.639 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:705 3.10s0 3:0a8403ef:::smithi09235267-5:head [append 0~507904] snapc f=[f,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-04-26T03:39:31.639 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:15.962584+0000", 2024-04-26T03:39:31.639 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 15.596570493, 2024-04-26T03:39:31.640 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.018831363, 2024-04-26T03:39:31.640 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:31.640 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:31.640 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:31.640 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:31.640 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:31.640 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:31.640 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 705 2024-04-26T03:39:31.640 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.640 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:31.640 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.640 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:31.640 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.962584+0000", 2024-04-26T03:39:31.640 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.641 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.641 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.641 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:31.641 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.962584+0000", 2024-04-26T03:39:31.641 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.641 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.641 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.641 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:31.641 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.962586+0000", 2024-04-26T03:39:31.641 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4810000000000001e-06 2024-04-26T03:39:31.641 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.641 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.641 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:31.641 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.963083+0000", 2024-04-26T03:39:31.641 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00049737900000000005 2024-04-26T03:39:31.642 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.642 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.642 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:31.642 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.963100+0000", 2024-04-26T03:39:31.642 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6623e-05 2024-04-26T03:39:31.642 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.642 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.642 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:31.642 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.963109+0000", 2024-04-26T03:39:31.642 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.7990000000000005e-06 2024-04-26T03:39:31.642 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.642 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.642 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:31.642 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.965845+0000", 2024-04-26T03:39:31.643 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0027354570000000002 2024-04-26T03:39:31.643 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.643 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.643 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:31.643 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.965947+0000", 2024-04-26T03:39:31.643 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000102383 2024-04-26T03:39:31.643 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.643 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.643 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:31.643 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.966825+0000", 2024-04-26T03:39:31.643 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00087811200000000001 2024-04-26T03:39:31.643 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.643 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.643 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:31.643 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.973835+0000", 2024-04-26T03:39:31.644 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0070094210000000001 2024-04-26T03:39:31.644 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.644 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.644 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:31.644 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.981333+0000", 2024-04-26T03:39:31.644 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0074981839999999998 2024-04-26T03:39:31.644 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.644 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.644 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:31.644 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.981415+0000", 2024-04-26T03:39:31.644 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2523999999999998e-05 2024-04-26T03:39:31.645 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.645 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:31.645 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.645 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.645 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.645 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:706 3.10s0 3:0a8403ef:::smithi09235267-5:head [append 0~475136] snapc f=[f,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-04-26T03:39:31.645 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:15.965849+0000", 2024-04-26T03:39:31.645 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 15.593305414, 2024-04-26T03:39:31.645 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.015731676999999999, 2024-04-26T03:39:31.645 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:31.645 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:31.645 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:31.645 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:31.645 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:31.645 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:31.646 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 706 2024-04-26T03:39:31.646 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.646 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:31.646 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.646 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:31.646 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.965849+0000", 2024-04-26T03:39:31.646 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.646 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.646 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.646 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:31.646 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.965849+0000", 2024-04-26T03:39:31.646 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.646 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.647 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.647 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:31.647 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.965850+0000", 2024-04-26T03:39:31.647 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1060000000000001e-06 2024-04-26T03:39:31.647 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.647 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.647 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:31.647 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.966348+0000", 2024-04-26T03:39:31.647 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00049790100000000003 2024-04-26T03:39:31.647 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.647 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.647 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:31.647 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.966361+0000", 2024-04-26T03:39:31.648 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3176000000000001e-05 2024-04-26T03:39:31.648 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.648 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.648 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:31.648 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.966371+0000", 2024-04-26T03:39:31.648 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.5599999999999999e-06 2024-04-26T03:39:31.648 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.648 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.648 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:31.648 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.970123+0000", 2024-04-26T03:39:31.648 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0037524720000000002 2024-04-26T03:39:31.648 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.648 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.648 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:31.649 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.970189+0000", 2024-04-26T03:39:31.649 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.6138999999999998e-05 2024-04-26T03:39:31.649 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.649 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.649 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:31.649 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.970750+0000", 2024-04-26T03:39:31.649 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00056040199999999999 2024-04-26T03:39:31.649 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.649 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.649 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:31.649 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.976064+0000", 2024-04-26T03:39:31.649 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0053144940000000003 2024-04-26T03:39:31.649 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.649 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.650 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:31.650 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.981522+0000", 2024-04-26T03:39:31.650 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0054579370000000004 2024-04-26T03:39:31.650 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.650 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.650 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:31.650 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.981581+0000", 2024-04-26T03:39:31.650 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.8489999999999997e-05 2024-04-26T03:39:31.650 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.650 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:31.650 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.650 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.650 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.650 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:707 3.10s0 3:0a8403ef:::smithi09235267-5:head [append 0~98304] snapc f=[f,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-04-26T03:39:31.651 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:15.966605+0000", 2024-04-26T03:39:31.651 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 15.592549877, 2024-04-26T03:39:31.651 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.015355298, 2024-04-26T03:39:31.651 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:31.651 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:31.651 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:31.651 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:31.651 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:31.651 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:31.651 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 707 2024-04-26T03:39:31.651 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.651 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:31.651 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.652 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:31.652 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.966605+0000", 2024-04-26T03:39:31.652 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.652 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.652 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.652 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:31.652 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.966605+0000", 2024-04-26T03:39:31.652 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.652 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.652 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.652 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:31.652 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.966606+0000", 2024-04-26T03:39:31.652 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.0599999999999999e-07 2024-04-26T03:39:31.652 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.653 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.653 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:31.653 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.966711+0000", 2024-04-26T03:39:31.653 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000105566 2024-04-26T03:39:31.653 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.653 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.653 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:31.653 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.966727+0000", 2024-04-26T03:39:31.653 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6277999999999999e-05 2024-04-26T03:39:31.653 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.653 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.653 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:31.653 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.966733+0000", 2024-04-26T03:39:31.653 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.694e-06 2024-04-26T03:39:31.654 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.654 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.654 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:31.654 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.974103+0000", 2024-04-26T03:39:31.654 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0073698349999999999 2024-04-26T03:39:31.654 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.654 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.654 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:31.654 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.974166+0000", 2024-04-26T03:39:31.654 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.2725000000000007e-05 2024-04-26T03:39:31.654 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.654 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.655 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:31.655 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.974561+0000", 2024-04-26T03:39:31.655 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000395471 2024-04-26T03:39:31.655 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.655 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.655 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:31.655 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.976345+0000", 2024-04-26T03:39:31.655 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0017836270000000001 2024-04-26T03:39:31.655 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.655 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.655 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:31.655 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.981917+0000", 2024-04-26T03:39:31.655 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0055726539999999998 2024-04-26T03:39:31.656 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.656 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.656 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:31.656 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.981960+0000", 2024-04-26T03:39:31.656 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.2542e-05 2024-04-26T03:39:31.656 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.656 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:31.656 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.656 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.656 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.656 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:708 3.10s0 3:0a8403ef:::smithi09235267-5:head [setxattr _header (56) in=7b] snapc f=[f,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-04-26T03:39:31.656 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:15.966764+0000", 2024-04-26T03:39:31.656 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 15.592390186999999, 2024-04-26T03:39:31.657 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.015696531, 2024-04-26T03:39:31.657 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:31.657 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:31.657 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:31.657 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:31.657 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:31.657 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:31.657 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 708 2024-04-26T03:39:31.657 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.657 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:31.657 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.657 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:31.657 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.966764+0000", 2024-04-26T03:39:31.657 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.657 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.658 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.658 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:31.658 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.966764+0000", 2024-04-26T03:39:31.658 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.658 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.658 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.658 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:31.658 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.966769+0000", 2024-04-26T03:39:31.658 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.1019999999999996e-06 2024-04-26T03:39:31.658 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.658 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.658 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:31.658 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.966773+0000", 2024-04-26T03:39:31.659 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.6270000000000002e-06 2024-04-26T03:39:31.659 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.659 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.659 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:31.659 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.966783+0000", 2024-04-26T03:39:31.659 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.8609999999999993e-06 2024-04-26T03:39:31.659 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.659 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.659 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:31.659 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.966787+0000", 2024-04-26T03:39:31.659 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.374e-06 2024-04-26T03:39:31.659 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.659 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.659 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:31.660 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.975352+0000", 2024-04-26T03:39:31.660 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0085645470000000005 2024-04-26T03:39:31.660 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.660 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.660 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:31.660 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.975428+0000", 2024-04-26T03:39:31.660 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.6372000000000007e-05 2024-04-26T03:39:31.660 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.660 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.660 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:31.660 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.975698+0000", 2024-04-26T03:39:31.660 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00027010100000000002 2024-04-26T03:39:31.660 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.661 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.661 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:31.661 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.976787+0000", 2024-04-26T03:39:31.661 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0010884740000000001 2024-04-26T03:39:31.661 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.661 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.661 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:31.661 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.982394+0000", 2024-04-26T03:39:31.661 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0056069789999999998 2024-04-26T03:39:31.661 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.661 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.661 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:31.661 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.982461+0000", 2024-04-26T03:39:31.661 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7094000000000005e-05 2024-04-26T03:39:31.662 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.662 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:31.662 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.662 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.662 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.662 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:709 3.10s0 3:0a8403ef:::smithi09235267-5:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e60)", 2024-04-26T03:39:31.662 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:15.966795+0000", 2024-04-26T03:39:31.662 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 15.592359715000001, 2024-04-26T03:39:31.662 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.01702424, 2024-04-26T03:39:31.662 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:31.662 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:31.662 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:31.662 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:31.662 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:31.663 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:31.663 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 709 2024-04-26T03:39:31.663 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.663 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:31.663 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.663 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:31.663 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.966795+0000", 2024-04-26T03:39:31.663 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.663 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.663 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.663 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:31.663 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.966795+0000", 2024-04-26T03:39:31.663 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.664 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.664 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.664 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:31.664 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.966795+0000", 2024-04-26T03:39:31.664 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.5600000000000001e-07 2024-04-26T03:39:31.664 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.664 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.664 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:31.664 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.966797+0000", 2024-04-26T03:39:31.664 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.894e-06 2024-04-26T03:39:31.664 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.664 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.664 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:31.664 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.966805+0000", 2024-04-26T03:39:31.665 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5789999999999998e-06 2024-04-26T03:39:31.665 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.665 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.665 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:31.665 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.966808+0000", 2024-04-26T03:39:31.665 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.3979999999999999e-06 2024-04-26T03:39:31.665 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.665 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.665 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:31.666 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.976218+0000", 2024-04-26T03:39:31.666 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0094095600000000008 2024-04-26T03:39:31.666 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.666 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.666 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:31.666 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.976269+0000", 2024-04-26T03:39:31.666 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.1554999999999997e-05 2024-04-26T03:39:31.666 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.666 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.666 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:31.666 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.982846+0000", 2024-04-26T03:39:31.666 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.006576363 2024-04-26T03:39:31.666 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.666 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.667 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:31.667 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.982912+0000", 2024-04-26T03:39:31.667 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.6713999999999999e-05 2024-04-26T03:39:31.667 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.667 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.667 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:31.667 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.983819+0000", 2024-04-26T03:39:31.667 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00090682100000000004 2024-04-26T03:39:31.667 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.667 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:31.667 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.667 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.667 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.668 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.as2 65/32 ECSubWrite(tid=120, reqid=client.4556.0:760, at_version=65'70, trim_to=0'0, roll_forward_to=64'68))", 2024-04-26T03:39:31.668 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:20.749574+0000", 2024-04-26T03:39:31.668 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 10.809580050999999, 2024-04-26T03:39:31.668 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.016094267999999998, 2024-04-26T03:39:31.668 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:31.668 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:31.668 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:31.668 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:31.668 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.668 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:31.668 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.749574+0000", 2024-04-26T03:39:31.668 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.668 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.668 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.668 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:31.669 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.749574+0000", 2024-04-26T03:39:31.669 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:31.669 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.669 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.669 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:31.669 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.749575+0000", 2024-04-26T03:39:31.669 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.0500000000000004e-07 2024-04-26T03:39:31.669 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.669 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.669 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:31.669 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.750293+0000", 2024-04-26T03:39:31.669 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00071780200000000002 2024-04-26T03:39:31.669 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.669 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.670 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:31.670 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.750301+0000", 2024-04-26T03:39:31.670 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.6090000000000001e-06 2024-04-26T03:39:31.670 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.670 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.670 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:31.670 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.750309+0000", 2024-04-26T03:39:31.670 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8499999999999994e-06 2024-04-26T03:39:31.670 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.670 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.670 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:31.670 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.750388+0000", 2024-04-26T03:39:31.670 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8306999999999994e-05 2024-04-26T03:39:31.671 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.671 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.671 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:31.671 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.750398+0000", 2024-04-26T03:39:31.671 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0421000000000001e-05 2024-04-26T03:39:31.671 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.671 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.671 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:31.671 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.765629+0000", 2024-04-26T03:39:31.671 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.015231246 2024-04-26T03:39:31.671 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:31.671 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:31.671 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:31.671 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.765669+0000", 2024-04-26T03:39:31.671 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9527999999999998e-05 2024-04-26T03:39:31.672 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.672 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:31.672 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.672 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:31.672 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:31.672 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:39:31.673 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:39:31.672+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:39:31.674 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-04-26T03:39:31.691 INFO:teuthology.orchestra.run.smithi028.stderr:reweighted osd.3 to 1 (10000) 2024-04-26T03:39:31.696 INFO:tasks.rados.rados.0.smithi092.stdout:331: done (4 left) 2024-04-26T03:39:31.696 INFO:tasks.rados.rados.0.smithi092.stdout:332: done (3 left) 2024-04-26T03:39:31.697 INFO:tasks.rados.rados.0.smithi092.stdout:333: done (2 left) 2024-04-26T03:39:31.697 INFO:tasks.rados.rados.0.smithi092.stdout:334: done (1 left) 2024-04-26T03:39:31.697 INFO:tasks.rados.rados.0.smithi092.stdout:336: done (0 left) 2024-04-26T03:39:31.697 INFO:tasks.rados.rados.0.smithi092.stdout:337: snap_create 2024-04-26T03:39:31.713 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-04-26T03:39:31.716 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-26T03:39:31.798 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:39:31.796+0000 7fba0d2ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31961) UID: 0 2024-04-26T03:39:31.822 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:39:31.822 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-04-26T03:39:31.822 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 0 2024-04-26T03:39:31.822 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:39:31.831 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-04-26T03:39:31.875 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:39:31.872+0000 7fba0d2ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31961) UID: 0 2024-04-26T03:39:31.961 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:39:31.961 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-04-26T03:39:31.961 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-04-26T03:39:31.961 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-04-26T03:39:31.961 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:39:31.971 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-04-26T03:39:32.016 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-04-26T03:39:32.015+0000 7fee562fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31759) UID: 0 2024-04-26T03:39:32.076 INFO:tasks.ceph.osd.9.smithi042.stderr:2024-04-26T03:39:32.075+0000 7fdac63b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 31960) UID: 0 2024-04-26T03:39:32.095 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:39:32.095 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-04-26T03:39:32.095 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-04-26T03:39:32.095 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-04-26T03:39:32.095 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.095 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:17 3.3s0 3:c944ba3d:::smithi09235267-6:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:32.095 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.485815+0000", 2024-04-26T03:39:32.096 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 40.606938659999997, 2024-04-26T03:39:32.096 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.013104035999999999, 2024-04-26T03:39:32.096 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:32.096 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:32.096 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:32.096 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:32.096 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:32.096 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:32.096 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 17 2024-04-26T03:39:32.096 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.096 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:32.096 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.096 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:32.096 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.485815+0000", 2024-04-26T03:39:32.097 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.097 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.097 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.097 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:32.097 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.485815+0000", 2024-04-26T03:39:32.097 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.097 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.097 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.097 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:32.097 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.485816+0000", 2024-04-26T03:39:32.097 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.0599999999999999e-07 2024-04-26T03:39:32.097 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.097 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.097 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:32.098 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.485821+0000", 2024-04-26T03:39:32.098 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.2700000000000004e-06 2024-04-26T03:39:32.098 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.098 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.098 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:32.098 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.486544+0000", 2024-04-26T03:39:32.098 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00072360100000000004 2024-04-26T03:39:32.098 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.098 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.098 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:32.099 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.486558+0000", 2024-04-26T03:39:32.099 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3127e-05 2024-04-26T03:39:32.099 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.099 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.099 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.099 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.490100+0000", 2024-04-26T03:39:32.099 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.003542084 2024-04-26T03:39:32.099 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.099 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.099 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:32.099 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.490196+0000", 2024-04-26T03:39:32.099 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.5990999999999996e-05 2024-04-26T03:39:32.100 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.100 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.100 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:32.100 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.490783+0000", 2024-04-26T03:39:32.100 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00058718699999999997 2024-04-26T03:39:32.100 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.100 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.100 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:32.100 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.494389+0000", 2024-04-26T03:39:32.101 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0036066980000000002 2024-04-26T03:39:32.101 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.101 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.101 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:32.101 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.498804+0000", 2024-04-26T03:39:32.101 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0044143150000000003 2024-04-26T03:39:32.101 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.101 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.101 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:32.101 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.498919+0000", 2024-04-26T03:39:32.101 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000114957 2024-04-26T03:39:32.101 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.101 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:32.101 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.102 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.102 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.102 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:18 3.3s0 3:c944ba3d:::smithi09235267-6:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:32.102 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.486578+0000", 2024-04-26T03:39:32.102 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 40.606175716000003, 2024-04-26T03:39:32.102 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.024780098, 2024-04-26T03:39:32.102 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:32.102 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:32.102 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:32.102 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:32.102 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:32.102 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:32.102 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 18 2024-04-26T03:39:32.103 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.103 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:32.103 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.103 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:32.103 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.486578+0000", 2024-04-26T03:39:32.103 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.103 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.103 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.103 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:32.103 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.486578+0000", 2024-04-26T03:39:32.103 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.103 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.103 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.103 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:32.104 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.486579+0000", 2024-04-26T03:39:32.104 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0529999999999999e-06 2024-04-26T03:39:32.104 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.104 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.104 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:32.104 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.486589+0000", 2024-04-26T03:39:32.104 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.9359999999999994e-06 2024-04-26T03:39:32.104 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.104 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.104 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:32.104 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.486601+0000", 2024-04-26T03:39:32.104 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2309e-05 2024-04-26T03:39:32.104 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.104 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.104 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:32.105 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.486607+0000", 2024-04-26T03:39:32.105 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5209999999999999e-06 2024-04-26T03:39:32.105 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.105 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.105 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.105 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.491290+0000", 2024-04-26T03:39:32.105 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0046821939999999998 2024-04-26T03:39:32.105 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.105 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.105 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:32.105 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.491369+0000", 2024-04-26T03:39:32.105 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9204999999999998e-05 2024-04-26T03:39:32.105 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.105 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.105 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.106 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.499348+0000", 2024-04-26T03:39:32.106 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0079792289999999991 2024-04-26T03:39:32.106 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.106 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.106 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:32.106 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.499429+0000", 2024-04-26T03:39:32.106 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.1051999999999996e-05 2024-04-26T03:39:32.106 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.106 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.106 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:32.106 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.511358+0000", 2024-04-26T03:39:32.106 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.011928599 2024-04-26T03:39:32.106 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.106 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:32.106 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.109 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.109 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.109 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:88 3.3s0 3:c8a9e28b:::smithi09235267-30:head [append 0~491520,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:32.110 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.543490+0000", 2024-04-26T03:39:32.110 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 40.549262949000003, 2024-04-26T03:39:32.110 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.01534287, 2024-04-26T03:39:32.110 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:32.110 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:32.110 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:32.110 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:32.111 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:32.111 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:32.111 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 88 2024-04-26T03:39:32.111 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.111 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:32.111 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.111 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:32.111 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.543490+0000", 2024-04-26T03:39:32.111 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.111 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.111 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.111 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:32.111 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.543490+0000", 2024-04-26T03:39:32.111 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.112 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.112 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.112 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:32.112 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.543492+0000", 2024-04-26T03:39:32.112 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5349999999999999e-06 2024-04-26T03:39:32.112 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.112 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.112 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:32.112 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.543954+0000", 2024-04-26T03:39:32.112 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00046193100000000001 2024-04-26T03:39:32.112 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.112 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.112 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:32.112 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.543967+0000", 2024-04-26T03:39:32.112 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3191e-05 2024-04-26T03:39:32.113 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.113 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.113 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:32.113 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.543972+0000", 2024-04-26T03:39:32.113 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.2950000000000001e-06 2024-04-26T03:39:32.113 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.113 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.113 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.113 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.544031+0000", 2024-04-26T03:39:32.113 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.8153999999999999e-05 2024-04-26T03:39:32.113 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.113 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.113 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:32.114 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.544176+0000", 2024-04-26T03:39:32.114 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014557900000000001 2024-04-26T03:39:32.114 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.114 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.114 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:32.114 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.545222+0000", 2024-04-26T03:39:32.114 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0010454590000000001 2024-04-26T03:39:32.114 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.114 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.114 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:32.114 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.551331+0000", 2024-04-26T03:39:32.114 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0061097180000000001 2024-04-26T03:39:32.114 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.115 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.115 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:32.115 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.558761+0000", 2024-04-26T03:39:32.115 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0074299800000000001 2024-04-26T03:39:32.115 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.115 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.115 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:32.115 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.558833+0000", 2024-04-26T03:39:32.115 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.2027999999999995e-05 2024-04-26T03:39:32.115 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.115 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:32.115 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.115 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.115 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.116 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:89 3.3s0 3:c8a9e28b:::smithi09235267-30:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:32.116 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.543986+0000", 2024-04-26T03:39:32.116 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 40.548766874000002, 2024-04-26T03:39:32.116 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.015002372, 2024-04-26T03:39:32.116 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:32.116 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:32.116 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:32.116 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:32.116 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:32.116 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:32.116 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 89 2024-04-26T03:39:32.116 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.116 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:32.116 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.116 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:32.117 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.543986+0000", 2024-04-26T03:39:32.117 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.117 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.117 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.117 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:32.117 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.543986+0000", 2024-04-26T03:39:32.117 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.117 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.117 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.117 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:32.117 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.543987+0000", 2024-04-26T03:39:32.117 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5899999999999996e-07 2024-04-26T03:39:32.117 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.117 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.118 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:32.118 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.543991+0000", 2024-04-26T03:39:32.118 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.58e-06 2024-04-26T03:39:32.118 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.118 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.118 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:32.118 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.543998+0000", 2024-04-26T03:39:32.118 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3119999999999996e-06 2024-04-26T03:39:32.118 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.118 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.118 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:32.118 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.544001+0000", 2024-04-26T03:39:32.118 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.399e-06 2024-04-26T03:39:32.118 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.119 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.119 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.119 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.550243+0000", 2024-04-26T03:39:32.119 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0062414180000000003 2024-04-26T03:39:32.119 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.119 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.119 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:32.119 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.550330+0000", 2024-04-26T03:39:32.119 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.6897999999999994e-05 2024-04-26T03:39:32.119 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.119 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.119 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:32.119 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.550794+0000", 2024-04-26T03:39:32.119 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00046398799999999998 2024-04-26T03:39:32.119 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.120 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.120 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:32.120 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.557987+0000", 2024-04-26T03:39:32.120 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0071936109999999999 2024-04-26T03:39:32.120 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.120 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.120 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:32.120 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.558928+0000", 2024-04-26T03:39:32.120 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000940153 2024-04-26T03:39:32.120 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.120 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.120 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:32.121 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.558989+0000", 2024-04-26T03:39:32.121 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.1353999999999996e-05 2024-04-26T03:39:32.121 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.121 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:32.121 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.121 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.121 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.121 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:90 3.3s0 3:c8a9e28b:::smithi09235267-30:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:32.121 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.544026+0000", 2024-04-26T03:39:32.121 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 40.548726983000002, 2024-04-26T03:39:32.122 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.017169266999999998, 2024-04-26T03:39:32.122 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:32.122 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:32.122 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:32.122 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:32.122 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:32.122 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:32.122 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 90 2024-04-26T03:39:32.122 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.122 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:32.122 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.122 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:32.122 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.544026+0000", 2024-04-26T03:39:32.122 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.123 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.123 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.123 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:32.123 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.544026+0000", 2024-04-26T03:39:32.123 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.123 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.123 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.123 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:32.123 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.544027+0000", 2024-04-26T03:39:32.123 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.0999999999999998e-07 2024-04-26T03:39:32.123 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.123 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.123 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:32.123 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.544030+0000", 2024-04-26T03:39:32.123 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6410000000000002e-06 2024-04-26T03:39:32.124 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.124 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.124 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:32.124 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.544037+0000", 2024-04-26T03:39:32.124 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.2459999999999997e-06 2024-04-26T03:39:32.124 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.124 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.124 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:32.124 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.544042+0000", 2024-04-26T03:39:32.124 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.9919999999999998e-06 2024-04-26T03:39:32.124 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.124 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.124 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.124 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.551230+0000", 2024-04-26T03:39:32.124 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0071875619999999998 2024-04-26T03:39:32.125 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.125 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.125 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:32.125 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.551311+0000", 2024-04-26T03:39:32.125 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.1182999999999998e-05 2024-04-26T03:39:32.125 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.125 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.125 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.125 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.559039+0000", 2024-04-26T03:39:32.125 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0077281479999999998 2024-04-26T03:39:32.125 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.125 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.125 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:32.125 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.559125+0000", 2024-04-26T03:39:32.125 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5678000000000002e-05 2024-04-26T03:39:32.126 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.126 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.126 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:32.126 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.561196+0000", 2024-04-26T03:39:32.126 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.002071107 2024-04-26T03:39:32.126 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.126 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:32.126 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.126 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.126 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.126 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:111 3.3s0 3:cad73b03:::smithi09235267-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:32.126 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.570167+0000", 2024-04-26T03:39:32.126 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 40.522586207000003, 2024-04-26T03:39:32.127 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.014356136, 2024-04-26T03:39:32.127 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:32.127 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:32.127 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:32.127 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:32.127 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:32.127 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:32.127 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 111 2024-04-26T03:39:32.127 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.127 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:32.127 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.127 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:32.127 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.570167+0000", 2024-04-26T03:39:32.127 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.128 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.128 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.128 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:32.128 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.570167+0000", 2024-04-26T03:39:32.128 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.128 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.128 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.128 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:32.128 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.570168+0000", 2024-04-26T03:39:32.128 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.0699999999999997e-07 2024-04-26T03:39:32.128 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.128 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.128 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:32.128 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.570170+0000", 2024-04-26T03:39:32.128 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6089999999999999e-06 2024-04-26T03:39:32.129 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.129 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.129 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:32.129 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.570178+0000", 2024-04-26T03:39:32.129 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.4449999999999999e-06 2024-04-26T03:39:32.129 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.129 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.129 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:32.129 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.570185+0000", 2024-04-26T03:39:32.129 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.9800000000000001e-06 2024-04-26T03:39:32.129 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.129 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.129 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.129 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.575666+0000", 2024-04-26T03:39:32.129 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0054812819999999996 2024-04-26T03:39:32.130 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.130 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.130 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:32.130 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.575751+0000", 2024-04-26T03:39:32.130 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5247999999999994e-05 2024-04-26T03:39:32.130 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.130 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.130 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.130 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.580678+0000", 2024-04-26T03:39:32.130 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0049265330000000003 2024-04-26T03:39:32.130 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.130 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.130 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:32.130 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.580798+0000", 2024-04-26T03:39:32.131 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000120365 2024-04-26T03:39:32.131 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.131 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.131 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:32.131 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.584523+0000", 2024-04-26T03:39:32.131 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0037251670000000001 2024-04-26T03:39:32.131 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.132 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:32.132 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.132 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.132 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.132 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.as3 35/32 ECSubWrite(tid=24, reqid=client.4556.0:150, at_version=35'13, trim_to=0'0, roll_forward_to=35'12))", 2024-04-26T03:39:32.132 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.636550+0000", 2024-04-26T03:39:32.132 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 40.456203045999999, 2024-04-26T03:39:32.132 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.013229930000000001, 2024-04-26T03:39:32.132 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:32.132 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:32.132 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:32.132 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:32.132 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.132 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:32.133 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.636550+0000", 2024-04-26T03:39:32.133 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.133 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.133 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.133 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:32.133 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.636550+0000", 2024-04-26T03:39:32.133 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.133 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.133 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.133 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:32.133 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.636551+0000", 2024-04-26T03:39:32.133 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4499999999999998e-07 2024-04-26T03:39:32.133 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.133 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.133 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:32.134 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.636656+0000", 2024-04-26T03:39:32.134 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00010504 2024-04-26T03:39:32.134 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.134 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.134 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:32.134 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.636667+0000", 2024-04-26T03:39:32.134 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1474000000000001e-05 2024-04-26T03:39:32.134 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.134 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.134 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:32.134 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.636683+0000", 2024-04-26T03:39:32.134 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6022999999999999e-05 2024-04-26T03:39:32.134 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.134 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.135 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.135 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.636857+0000", 2024-04-26T03:39:32.135 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00017374600000000001 2024-04-26T03:39:32.135 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.135 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.135 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:32.135 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.636865+0000", 2024-04-26T03:39:32.135 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.1000000000000004e-06 2024-04-26T03:39:32.135 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.135 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.135 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:32.135 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.649757+0000", 2024-04-26T03:39:32.135 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.01289242 2024-04-26T03:39:32.135 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.135 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.136 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:32.136 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.649780+0000", 2024-04-26T03:39:32.136 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2881999999999999e-05 2024-04-26T03:39:32.136 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.136 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:32.136 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.136 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.136 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.136 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.4s5 40/32 ECSubWrite(tid=22, reqid=client.4556.0:262, at_version=40'14, trim_to=0'0, roll_forward_to=35'12))", 2024-04-26T03:39:32.136 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:56.521062+0000", 2024-04-26T03:39:32.136 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 35.571691465000001, 2024-04-26T03:39:32.136 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.014879939, 2024-04-26T03:39:32.136 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:32.136 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:32.136 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:32.137 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:32.137 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.137 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:32.137 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.521062+0000", 2024-04-26T03:39:32.137 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.137 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.137 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.137 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:32.137 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.521062+0000", 2024-04-26T03:39:32.137 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.137 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.137 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.137 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:32.137 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.521062+0000", 2024-04-26T03:39:32.138 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4900000000000002e-07 2024-04-26T03:39:32.138 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.138 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.138 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:32.138 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.521346+0000", 2024-04-26T03:39:32.138 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00028391499999999998 2024-04-26T03:39:32.138 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.138 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.138 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:32.138 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.521358+0000", 2024-04-26T03:39:32.138 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1567e-05 2024-04-26T03:39:32.138 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.138 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.138 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:32.138 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.521365+0000", 2024-04-26T03:39:32.139 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.7849999999999992e-06 2024-04-26T03:39:32.139 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.139 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.139 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.139 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.521432+0000", 2024-04-26T03:39:32.139 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.6963000000000002e-05 2024-04-26T03:39:32.139 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.139 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.139 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:32.139 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.521442+0000", 2024-04-26T03:39:32.139 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0098000000000001e-05 2024-04-26T03:39:32.139 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.139 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.139 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:32.140 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.535904+0000", 2024-04-26T03:39:32.140 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.014461821999999999 2024-04-26T03:39:32.140 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.140 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.140 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:32.140 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.535942+0000", 2024-04-26T03:39:32.140 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7540000000000003e-05 2024-04-26T03:39:32.140 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.140 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:32.140 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.140 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.140 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.140 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:312 3.3s0 3:c8a9e28b:::smithi09235267-30:head [append 0~475136] snapc 6=[6,5] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-26T03:39:32.140 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:59.547999+0000", 2024-04-26T03:39:32.140 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 32.544754619000003, 2024-04-26T03:39:32.141 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.013877782, 2024-04-26T03:39:32.141 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:32.141 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:32.141 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:32.141 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:32.141 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:32.141 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:32.141 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 312 2024-04-26T03:39:32.141 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.141 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:32.142 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.142 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:32.142 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.547999+0000", 2024-04-26T03:39:32.142 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.142 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.142 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.142 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:32.142 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.547999+0000", 2024-04-26T03:39:32.142 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.142 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.143 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.143 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:32.143 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.548000+0000", 2024-04-26T03:39:32.143 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7370000000000001e-06 2024-04-26T03:39:32.143 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.143 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.143 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:32.143 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.548517+0000", 2024-04-26T03:39:32.143 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00051695699999999996 2024-04-26T03:39:32.143 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.143 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.143 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:32.143 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.548533+0000", 2024-04-26T03:39:32.143 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5985e-05 2024-04-26T03:39:32.143 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.144 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.144 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:32.144 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.548540+0000", 2024-04-26T03:39:32.144 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.0639999999999996e-06 2024-04-26T03:39:32.144 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.144 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.144 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.144 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.548614+0000", 2024-04-26T03:39:32.144 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3194000000000004e-05 2024-04-26T03:39:32.144 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.144 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.144 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:32.144 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.548715+0000", 2024-04-26T03:39:32.144 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000101309 2024-04-26T03:39:32.145 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.145 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.145 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:32.145 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.550003+0000", 2024-04-26T03:39:32.145 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001287722 2024-04-26T03:39:32.145 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.145 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.145 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:32.145 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.561581+0000", 2024-04-26T03:39:32.145 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.011578774999999999 2024-04-26T03:39:32.145 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.145 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.145 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:32.145 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.561791+0000", 2024-04-26T03:39:32.145 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020959900000000001 2024-04-26T03:39:32.146 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.146 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.146 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:32.146 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.561877+0000", 2024-04-26T03:39:32.146 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5439999999999995e-05 2024-04-26T03:39:32.146 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.146 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:32.146 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.146 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.146 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.146 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:314 3.3s0 3:c8a9e28b:::smithi09235267-30:head [append 0~147456] snapc 6=[6,5] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-26T03:39:32.146 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:59.552775+0000", 2024-04-26T03:39:32.146 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 32.539978343000001, 2024-04-26T03:39:32.146 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.030282514, 2024-04-26T03:39:32.147 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:32.147 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:32.147 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:32.147 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:32.147 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:32.147 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:32.147 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 314 2024-04-26T03:39:32.147 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.147 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:32.147 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.147 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:32.147 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552775+0000", 2024-04-26T03:39:32.147 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.147 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.148 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.148 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:32.148 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552775+0000", 2024-04-26T03:39:32.148 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.148 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.148 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.148 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:32.148 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552776+0000", 2024-04-26T03:39:32.148 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.026e-06 2024-04-26T03:39:32.148 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.148 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.148 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:32.148 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552935+0000", 2024-04-26T03:39:32.148 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00015929799999999999 2024-04-26T03:39:32.148 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.149 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.149 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:32.149 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552948+0000", 2024-04-26T03:39:32.149 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2716e-05 2024-04-26T03:39:32.149 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.149 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.149 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:32.149 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552955+0000", 2024-04-26T03:39:32.149 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.9609999999999999e-06 2024-04-26T03:39:32.149 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.149 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.149 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.149 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.562092+0000", 2024-04-26T03:39:32.149 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0091373979999999997 2024-04-26T03:39:32.150 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.150 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.150 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:32.150 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.562164+0000", 2024-04-26T03:39:32.150 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.1767999999999995e-05 2024-04-26T03:39:32.150 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.150 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.150 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:32.150 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.562591+0000", 2024-04-26T03:39:32.150 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00042723799999999998 2024-04-26T03:39:32.150 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.150 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.150 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:32.150 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.564915+0000", 2024-04-26T03:39:32.151 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0023239340000000002 2024-04-26T03:39:32.151 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.151 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.151 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:32.151 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.583009+0000", 2024-04-26T03:39:32.151 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.018093609 2024-04-26T03:39:32.151 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.151 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.151 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:32.151 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.583058+0000", 2024-04-26T03:39:32.151 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.8566000000000002e-05 2024-04-26T03:39:32.151 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.151 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:32.151 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.151 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.152 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.152 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:315 3.3s0 3:c8a9e28b:::smithi09235267-30:head [setxattr _header (56) in=7b] snapc 6=[6,5] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-26T03:39:32.152 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:59.552967+0000", 2024-04-26T03:39:32.152 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 32.539786808999999, 2024-04-26T03:39:32.152 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.030505498999999998, 2024-04-26T03:39:32.152 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:32.152 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:32.153 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:32.153 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:32.153 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:32.153 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:32.153 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 315 2024-04-26T03:39:32.153 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.153 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:32.153 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.153 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:32.153 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552967+0000", 2024-04-26T03:39:32.153 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.153 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.153 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.153 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:32.153 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552967+0000", 2024-04-26T03:39:32.154 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.154 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.154 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.154 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:32.154 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552967+0000", 2024-04-26T03:39:32.154 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.37e-07 2024-04-26T03:39:32.154 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.154 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.154 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:32.154 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552971+0000", 2024-04-26T03:39:32.154 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9580000000000001e-06 2024-04-26T03:39:32.154 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.155 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.155 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:32.155 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552980+0000", 2024-04-26T03:39:32.155 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.4549999999999995e-06 2024-04-26T03:39:32.155 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.155 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.155 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:32.155 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552985+0000", 2024-04-26T03:39:32.155 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.5119999999999999e-06 2024-04-26T03:39:32.155 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.155 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.155 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.155 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.563965+0000", 2024-04-26T03:39:32.156 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.010980177000000001 2024-04-26T03:39:32.156 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.156 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.156 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:32.156 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.564028+0000", 2024-04-26T03:39:32.156 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.2607000000000006e-05 2024-04-26T03:39:32.156 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.156 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.156 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:32.156 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.564435+0000", 2024-04-26T03:39:32.156 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00040658399999999998 2024-04-26T03:39:32.156 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.156 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.156 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:32.157 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.564918+0000", 2024-04-26T03:39:32.157 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00048333499999999999 2024-04-26T03:39:32.157 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.157 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.157 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:32.157 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.583395+0000", 2024-04-26T03:39:32.157 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.018476764999999999 2024-04-26T03:39:32.157 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.157 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.157 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:32.157 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.583472+0000", 2024-04-26T03:39:32.157 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.7368999999999997e-05 2024-04-26T03:39:32.157 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.157 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:32.157 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.158 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.158 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.158 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:316 3.3s0 3:c8a9e28b:::smithi09235267-30:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e43)", 2024-04-26T03:39:32.158 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:59.552993+0000", 2024-04-26T03:39:32.158 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 32.539760600999998, 2024-04-26T03:39:32.158 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.032226746000000001, 2024-04-26T03:39:32.158 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:32.158 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:32.158 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:32.158 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:32.158 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:32.158 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:32.158 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 316 2024-04-26T03:39:32.159 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.159 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:32.159 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.159 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:32.159 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552993+0000", 2024-04-26T03:39:32.159 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.159 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.159 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.159 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:32.159 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552993+0000", 2024-04-26T03:39:32.159 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.159 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.159 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.159 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:32.159 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552993+0000", 2024-04-26T03:39:32.160 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7299999999999996e-07 2024-04-26T03:39:32.160 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.160 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.160 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:32.160 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552996+0000", 2024-04-26T03:39:32.160 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.92e-06 2024-04-26T03:39:32.160 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.160 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.160 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:32.160 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.553005+0000", 2024-04-26T03:39:32.160 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.585e-06 2024-04-26T03:39:32.160 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.160 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.160 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:32.161 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.553010+0000", 2024-04-26T03:39:32.161 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.0719999999999999e-06 2024-04-26T03:39:32.161 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.161 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.161 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.161 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.564680+0000", 2024-04-26T03:39:32.161 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.011669736 2024-04-26T03:39:32.161 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.161 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.161 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:32.161 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.564736+0000", 2024-04-26T03:39:32.161 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.6113000000000003e-05 2024-04-26T03:39:32.161 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.161 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.161 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.162 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.583732+0000", 2024-04-26T03:39:32.162 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.018995920999999999 2024-04-26T03:39:32.162 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.162 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.162 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:32.162 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.583823+0000", 2024-04-26T03:39:32.162 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1335999999999994e-05 2024-04-26T03:39:32.162 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.163 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.163 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:32.163 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.585220+0000", 2024-04-26T03:39:32.163 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00139649 2024-04-26T03:39:32.163 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.163 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:32.163 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.163 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.163 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.163 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.7s5 49/44 ECSubWrite(tid=83, reqid=osd.4.0:83, at_version=49'10, trim_to=0'0, roll_forward_to=48'8))", 2024-04-26T03:39:32.163 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:04.696553+0000", 2024-04-26T03:39:32.163 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 27.396200733000001, 2024-04-26T03:39:32.163 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.013590896999999999, 2024-04-26T03:39:32.164 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:32.164 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:32.164 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:32.164 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:32.164 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.164 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:32.164 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.696553+0000", 2024-04-26T03:39:32.164 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.164 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.164 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.164 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:32.164 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.696553+0000", 2024-04-26T03:39:32.164 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.164 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.165 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.165 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:32.165 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.696553+0000", 2024-04-26T03:39:32.165 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.2600000000000002e-07 2024-04-26T03:39:32.165 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.165 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.165 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:32.165 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.696575+0000", 2024-04-26T03:39:32.165 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1441e-05 2024-04-26T03:39:32.165 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.166 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.166 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:32.166 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.696587+0000", 2024-04-26T03:39:32.166 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1892000000000001e-05 2024-04-26T03:39:32.166 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.166 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.166 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:32.166 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.696594+0000", 2024-04-26T03:39:32.166 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8310000000000001e-06 2024-04-26T03:39:32.166 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.166 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.166 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.166 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.696660+0000", 2024-04-26T03:39:32.166 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5550999999999996e-05 2024-04-26T03:39:32.167 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.167 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.167 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:32.167 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.696669+0000", 2024-04-26T03:39:32.167 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1940000000000004e-06 2024-04-26T03:39:32.167 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.167 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.167 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:32.167 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.710095+0000", 2024-04-26T03:39:32.167 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.013425906 2024-04-26T03:39:32.167 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.167 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.167 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:32.167 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.710144+0000", 2024-04-26T03:39:32.167 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.8455999999999998e-05 2024-04-26T03:39:32.168 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.168 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:32.168 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.168 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.168 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.168 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.2s5 49/44 ECSubWrite(tid=32, reqid=client.4556.0:433, at_version=49'20, trim_to=0'0, roll_forward_to=48'19))", 2024-04-26T03:39:32.168 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:04.700213+0000", 2024-04-26T03:39:32.168 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 27.392539984999999, 2024-04-26T03:39:32.168 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.018834018000000001, 2024-04-26T03:39:32.168 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:32.168 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:32.168 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:32.168 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:32.168 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.168 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:32.169 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700213+0000", 2024-04-26T03:39:32.169 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.169 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.169 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.169 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:32.169 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700213+0000", 2024-04-26T03:39:32.169 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.169 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.169 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.169 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:32.169 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700214+0000", 2024-04-26T03:39:32.169 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.48e-07 2024-04-26T03:39:32.169 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.169 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.169 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:32.170 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700972+0000", 2024-04-26T03:39:32.170 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00075845299999999997 2024-04-26T03:39:32.170 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.170 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.170 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:32.170 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700983+0000", 2024-04-26T03:39:32.170 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1105e-05 2024-04-26T03:39:32.170 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.170 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.170 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:32.170 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700991+0000", 2024-04-26T03:39:32.170 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9140000000000008e-06 2024-04-26T03:39:32.170 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.171 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.171 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.171 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.701100+0000", 2024-04-26T03:39:32.171 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000108915 2024-04-26T03:39:32.171 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.171 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.171 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:32.171 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.701120+0000", 2024-04-26T03:39:32.171 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9811000000000001e-05 2024-04-26T03:39:32.171 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.171 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.171 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:32.171 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.718957+0000", 2024-04-26T03:39:32.171 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.017837604 2024-04-26T03:39:32.171 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.172 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.172 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:32.172 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.719047+0000", 2024-04-26T03:39:32.172 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.9968000000000003e-05 2024-04-26T03:39:32.172 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.172 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:32.172 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.172 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.172 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.172 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.bs2 50/32 ECSubWrite(tid=88, reqid=osd.4.0:88, at_version=50'48, trim_to=0'0, roll_forward_to=49'46))", 2024-04-26T03:39:32.172 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:05.698893+0000", 2024-04-26T03:39:32.172 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.393860045, 2024-04-26T03:39:32.172 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.015901432, 2024-04-26T03:39:32.172 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:32.173 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:32.173 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:32.173 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:32.173 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.173 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:32.173 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.698893+0000", 2024-04-26T03:39:32.173 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.173 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.174 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.174 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:32.174 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.698893+0000", 2024-04-26T03:39:32.174 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.174 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.174 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.174 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:32.174 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.698894+0000", 2024-04-26T03:39:32.174 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.1e-07 2024-04-26T03:39:32.174 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.174 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.174 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:32.174 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.698914+0000", 2024-04-26T03:39:32.174 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0613000000000001e-05 2024-04-26T03:39:32.175 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.175 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.175 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:32.175 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.698924+0000", 2024-04-26T03:39:32.175 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.3859999999999996e-06 2024-04-26T03:39:32.175 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.175 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.175 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:32.175 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.698931+0000", 2024-04-26T03:39:32.175 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.0829999999999998e-06 2024-04-26T03:39:32.175 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.175 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.175 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.175 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.699137+0000", 2024-04-26T03:39:32.176 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020594 2024-04-26T03:39:32.176 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.176 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.176 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:32.176 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.699153+0000", 2024-04-26T03:39:32.176 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6218999999999998e-05 2024-04-26T03:39:32.176 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.176 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.176 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:32.176 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.714755+0000", 2024-04-26T03:39:32.176 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.015602517 2024-04-26T03:39:32.176 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.176 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.176 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:32.176 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.714795+0000", 2024-04-26T03:39:32.177 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9363999999999998e-05 2024-04-26T03:39:32.177 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.177 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:32.177 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.177 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.177 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.177 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:446 3.13s0 3:c944ba3d:::smithi09235267-6:head [append 0~720896] snapc 8=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:32.177 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:05.700096+0000", 2024-04-26T03:39:32.177 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.392657463999999, 2024-04-26T03:39:32.177 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.019772888999999998, 2024-04-26T03:39:32.177 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:32.177 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:32.177 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:32.177 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:32.177 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:32.178 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:32.178 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 446 2024-04-26T03:39:32.178 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.178 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:32.178 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.178 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:32.178 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.700096+0000", 2024-04-26T03:39:32.178 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.178 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.178 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.178 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:32.178 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.700096+0000", 2024-04-26T03:39:32.178 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.178 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.179 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.179 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:32.179 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.700098+0000", 2024-04-26T03:39:32.179 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.464e-06 2024-04-26T03:39:32.179 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.179 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.179 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:32.179 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.700859+0000", 2024-04-26T03:39:32.179 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00076090299999999995 2024-04-26T03:39:32.179 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.179 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.179 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:32.179 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.700875+0000", 2024-04-26T03:39:32.179 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5299e-05 2024-04-26T03:39:32.180 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.180 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.180 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:32.180 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.700881+0000", 2024-04-26T03:39:32.180 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.528e-06 2024-04-26T03:39:32.180 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.180 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.180 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.180 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.700945+0000", 2024-04-26T03:39:32.180 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.3744999999999994e-05 2024-04-26T03:39:32.180 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.180 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.180 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:32.180 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.701071+0000", 2024-04-26T03:39:32.180 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012569600000000001 2024-04-26T03:39:32.181 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.181 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.181 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:32.181 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.702209+0000", 2024-04-26T03:39:32.181 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001138266 2024-04-26T03:39:32.181 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.181 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.181 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:32.181 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.714847+0000", 2024-04-26T03:39:32.181 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.012638508 2024-04-26T03:39:32.181 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.181 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.181 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:32.181 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.719810+0000", 2024-04-26T03:39:32.181 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0049625989999999998 2024-04-26T03:39:32.182 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.182 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.182 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:32.182 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.719869+0000", 2024-04-26T03:39:32.182 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.8881000000000001e-05 2024-04-26T03:39:32.182 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.182 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:32.182 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.182 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.182 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.182 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:447 3.13s0 3:c944ba3d:::smithi09235267-6:head [append 0~442368] snapc 8=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:32.182 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:05.702913+0000", 2024-04-26T03:39:32.182 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.389840762999999, 2024-04-26T03:39:32.182 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.01837453, 2024-04-26T03:39:32.183 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:32.183 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:32.183 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:32.183 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:32.183 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:32.183 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:32.183 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 447 2024-04-26T03:39:32.183 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.183 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:32.183 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.183 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:32.183 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.702913+0000", 2024-04-26T03:39:32.183 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.183 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.183 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.184 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:32.184 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.702913+0000", 2024-04-26T03:39:32.184 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.184 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.184 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.184 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:32.184 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.702915+0000", 2024-04-26T03:39:32.184 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0360000000000001e-06 2024-04-26T03:39:32.184 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.184 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.184 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:32.184 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.703366+0000", 2024-04-26T03:39:32.184 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00045175800000000002 2024-04-26T03:39:32.184 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.185 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.185 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:32.185 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.703382+0000", 2024-04-26T03:39:32.185 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5707999999999999e-05 2024-04-26T03:39:32.185 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.185 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.185 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:32.185 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.703391+0000", 2024-04-26T03:39:32.185 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.3889999999999994e-06 2024-04-26T03:39:32.185 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.185 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.185 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.185 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.710509+0000", 2024-04-26T03:39:32.185 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0071172650000000002 2024-04-26T03:39:32.185 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.186 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.186 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:32.186 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.710619+0000", 2024-04-26T03:39:32.186 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00011061 2024-04-26T03:39:32.186 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.186 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.186 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:32.186 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.711319+0000", 2024-04-26T03:39:32.186 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00070012799999999995 2024-04-26T03:39:32.186 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.186 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.186 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:32.186 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.719699+0000", 2024-04-26T03:39:32.186 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0083798540000000008 2024-04-26T03:39:32.187 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.187 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.187 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:32.187 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.721236+0000", 2024-04-26T03:39:32.187 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0015364129999999999 2024-04-26T03:39:32.187 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.187 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.187 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:32.187 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.721287+0000", 2024-04-26T03:39:32.187 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.1369000000000002e-05 2024-04-26T03:39:32.187 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.187 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:32.187 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.187 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.187 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.188 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:448 3.13s0 3:c944ba3d:::smithi09235267-6:head [append 0~327680] snapc 8=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:32.188 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:05.704961+0000", 2024-04-26T03:39:32.188 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.387792424000001, 2024-04-26T03:39:32.188 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.019901015000000001, 2024-04-26T03:39:32.188 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:32.188 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:32.188 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:32.188 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:32.188 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:32.188 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:32.188 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 448 2024-04-26T03:39:32.188 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.188 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:32.188 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.189 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:32.189 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.704961+0000", 2024-04-26T03:39:32.189 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.189 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.189 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.189 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:32.189 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.704961+0000", 2024-04-26T03:39:32.189 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.189 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.189 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.189 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:32.189 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.704963+0000", 2024-04-26T03:39:32.189 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.81e-06 2024-04-26T03:39:32.189 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.189 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.190 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:32.190 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.705283+0000", 2024-04-26T03:39:32.190 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00032049100000000001 2024-04-26T03:39:32.190 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.190 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.190 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:32.190 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.705309+0000", 2024-04-26T03:39:32.190 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5789999999999999e-05 2024-04-26T03:39:32.190 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.190 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.190 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:32.190 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.705352+0000", 2024-04-26T03:39:32.190 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.2865999999999999e-05 2024-04-26T03:39:32.190 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.190 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.191 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.191 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.714894+0000", 2024-04-26T03:39:32.191 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.009542109 2024-04-26T03:39:32.191 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.191 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.191 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:32.191 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.715004+0000", 2024-04-26T03:39:32.191 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00010983400000000001 2024-04-26T03:39:32.191 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.191 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.191 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:32.191 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.715894+0000", 2024-04-26T03:39:32.191 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00089006399999999998 2024-04-26T03:39:32.191 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.191 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.192 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:32.192 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.720882+0000", 2024-04-26T03:39:32.192 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0049879499999999997 2024-04-26T03:39:32.192 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.192 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.192 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:32.192 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.724816+0000", 2024-04-26T03:39:32.192 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0039342830000000002 2024-04-26T03:39:32.192 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.192 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.192 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:32.192 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.724862+0000", 2024-04-26T03:39:32.192 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.5818000000000001e-05 2024-04-26T03:39:32.192 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.193 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:32.193 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.193 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.193 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.193 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:449 3.13s0 3:c944ba3d:::smithi09235267-6:head [setxattr _header (55) in=7b] snapc 8=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:32.193 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:05.705415+0000", 2024-04-26T03:39:32.193 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.387338203999999, 2024-04-26T03:39:32.193 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.021062219, 2024-04-26T03:39:32.193 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:32.193 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:32.193 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:32.193 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:32.193 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:32.193 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:32.193 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 449 2024-04-26T03:39:32.194 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.194 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:32.194 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.194 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:32.194 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.705415+0000", 2024-04-26T03:39:32.194 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.194 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.194 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.194 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:32.194 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.705415+0000", 2024-04-26T03:39:32.194 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.194 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.194 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.194 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:32.194 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.705416+0000", 2024-04-26T03:39:32.195 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5099999999999998e-07 2024-04-26T03:39:32.195 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.195 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.195 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:32.195 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.705422+0000", 2024-04-26T03:39:32.195 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.5079999999999996e-06 2024-04-26T03:39:32.195 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.195 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.195 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:32.195 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.705507+0000", 2024-04-26T03:39:32.195 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5282000000000001e-05 2024-04-26T03:39:32.195 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.195 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.195 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:32.195 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.705525+0000", 2024-04-26T03:39:32.196 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8173999999999999e-05 2024-04-26T03:39:32.196 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.196 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.196 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.196 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.720102+0000", 2024-04-26T03:39:32.196 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.014577345 2024-04-26T03:39:32.196 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.196 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.196 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:32.196 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.720161+0000", 2024-04-26T03:39:32.196 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.9070000000000002e-05 2024-04-26T03:39:32.196 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.196 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.196 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:32.197 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.720462+0000", 2024-04-26T03:39:32.197 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00030043300000000002 2024-04-26T03:39:32.197 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.197 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.197 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:32.197 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.722295+0000", 2024-04-26T03:39:32.197 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001833339 2024-04-26T03:39:32.197 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.197 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.197 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:32.197 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.726405+0000", 2024-04-26T03:39:32.197 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0041094959999999998 2024-04-26T03:39:32.197 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.197 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.197 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:32.198 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.726477+0000", 2024-04-26T03:39:32.198 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.2720999999999997e-05 2024-04-26T03:39:32.198 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.198 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:32.198 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.198 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.198 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.198 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:450 3.13s0 3:c944ba3d:::smithi09235267-6:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:32.198 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:05.705540+0000", 2024-04-26T03:39:32.198 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.387213436, 2024-04-26T03:39:32.198 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.022180959, 2024-04-26T03:39:32.198 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:32.198 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:32.199 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:32.199 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:32.199 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:32.199 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:32.199 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 450 2024-04-26T03:39:32.199 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.199 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:32.199 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.199 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:32.199 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.705540+0000", 2024-04-26T03:39:32.199 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.200 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.200 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.200 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:32.200 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.705540+0000", 2024-04-26T03:39:32.200 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.200 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.200 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.200 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:32.200 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.705541+0000", 2024-04-26T03:39:32.200 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.6000000000000003e-07 2024-04-26T03:39:32.200 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.200 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.200 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:32.200 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.705655+0000", 2024-04-26T03:39:32.200 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000114528 2024-04-26T03:39:32.201 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.201 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.201 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:32.201 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.705668+0000", 2024-04-26T03:39:32.201 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2712e-05 2024-04-26T03:39:32.201 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.201 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.201 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:32.201 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.705676+0000", 2024-04-26T03:39:32.201 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9179999999999994e-06 2024-04-26T03:39:32.201 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.201 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.201 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.201 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.720674+0000", 2024-04-26T03:39:32.201 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.014998177 2024-04-26T03:39:32.202 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.202 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.202 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:32.202 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.720807+0000", 2024-04-26T03:39:32.202 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000133053 2024-04-26T03:39:32.202 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.202 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.202 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.202 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.726766+0000", 2024-04-26T03:39:32.202 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0059592270000000001 2024-04-26T03:39:32.202 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.202 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.202 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:32.202 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.726868+0000", 2024-04-26T03:39:32.203 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000101931 2024-04-26T03:39:32.203 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.203 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.203 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:32.203 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.727721+0000", 2024-04-26T03:39:32.203 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00085265299999999998 2024-04-26T03:39:32.203 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.203 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:32.203 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.203 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.203 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:32.203 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:39:32.205 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-04-26T03:39:32.205 INFO:tasks.ceph.osd.10.smithi066.stderr:2024-04-26T03:39:32.204+0000 7f130fcc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 31763) UID: 0 2024-04-26T03:39:32.305 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:39:32.304+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:39:32.353 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:39:32.353 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-04-26T03:39:32.353 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 0 2024-04-26T03:39:32.353 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:39:32.362 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-04-26T03:39:32.407 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-04-26T03:39:32.405+0000 7f745118d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 31821) UID: 0 2024-04-26T03:39:32.469 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:39:32.469 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-04-26T03:39:32.469 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-04-26T03:39:32.469 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-04-26T03:39:32.469 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:39:32.478 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-04-26T03:39:32.505 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-26T03:39:32.522 INFO:tasks.ceph.osd.14.smithi066.stderr:2024-04-26T03:39:32.521+0000 7f5395f08640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 31765) UID: 0 2024-04-26T03:39:32.586 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:39:32.586 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-04-26T03:39:32.587 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-04-26T03:39:32.587 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-04-26T03:39:32.587 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.587 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:3 3.2s0 3:42fcabaa:::smithi09235267-1:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:32.587 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.480913+0000", 2024-04-26T03:39:32.587 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 41.103090209999998, 2024-04-26T03:39:32.587 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.012266038999999999, 2024-04-26T03:39:32.587 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:32.587 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:32.587 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:32.587 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:32.587 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:32.587 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:32.587 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 3 2024-04-26T03:39:32.588 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.588 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:32.588 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.588 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:32.588 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.480913+0000", 2024-04-26T03:39:32.588 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.588 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.588 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.588 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:32.588 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.480913+0000", 2024-04-26T03:39:32.588 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.588 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.588 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.588 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:32.589 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.480914+0000", 2024-04-26T03:39:32.589 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.7999999999999996e-07 2024-04-26T03:39:32.589 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.589 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.589 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:32.589 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.480917+0000", 2024-04-26T03:39:32.589 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.366e-06 2024-04-26T03:39:32.589 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.589 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.589 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:32.589 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.480925+0000", 2024-04-26T03:39:32.589 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9759999999999995e-06 2024-04-26T03:39:32.589 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.589 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.589 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:32.590 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.480932+0000", 2024-04-26T03:39:32.590 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.0339999999999999e-06 2024-04-26T03:39:32.590 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.590 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.590 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.590 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.483324+0000", 2024-04-26T03:39:32.590 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.002392323 2024-04-26T03:39:32.590 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.590 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.590 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:32.590 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.483402+0000", 2024-04-26T03:39:32.590 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.7745999999999997e-05 2024-04-26T03:39:32.590 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.591 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.591 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.591 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.487467+0000", 2024-04-26T03:39:32.591 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0040650310000000002 2024-04-26T03:39:32.591 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.591 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.591 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:32.591 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.487555+0000", 2024-04-26T03:39:32.591 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.8271999999999998e-05 2024-04-26T03:39:32.591 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.591 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.591 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:32.591 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.493179+0000", 2024-04-26T03:39:32.591 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0056238110000000003 2024-04-26T03:39:32.591 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.592 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:32.592 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.592 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.592 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.592 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:79 3.2s0 3:41b9d80c:::smithi09235267-27:head [append 0~442368,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:32.592 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.531542+0000", 2024-04-26T03:39:32.592 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 41.052461157000003, 2024-04-26T03:39:32.592 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.013543507999999999, 2024-04-26T03:39:32.592 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:32.592 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:32.592 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:32.592 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:32.592 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:32.592 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:32.592 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 79 2024-04-26T03:39:32.593 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.593 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:32.593 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.593 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:32.593 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.531542+0000", 2024-04-26T03:39:32.593 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.593 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.593 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.593 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:32.593 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.531542+0000", 2024-04-26T03:39:32.593 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.593 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.593 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.593 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:32.594 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.531544+0000", 2024-04-26T03:39:32.594 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9719999999999999e-06 2024-04-26T03:39:32.594 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.594 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.594 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:32.594 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.531978+0000", 2024-04-26T03:39:32.594 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00043396899999999998 2024-04-26T03:39:32.594 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.594 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.594 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:32.594 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.531990+0000", 2024-04-26T03:39:32.594 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2150999999999999e-05 2024-04-26T03:39:32.594 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.594 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.595 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:32.595 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.531997+0000", 2024-04-26T03:39:32.595 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.6320000000000002e-06 2024-04-26T03:39:32.595 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.595 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.595 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.595 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532044+0000", 2024-04-26T03:39:32.595 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.7095999999999998e-05 2024-04-26T03:39:32.595 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.595 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.595 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:32.595 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532156+0000", 2024-04-26T03:39:32.595 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000112131 2024-04-26T03:39:32.595 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.595 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.596 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:32.596 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532990+0000", 2024-04-26T03:39:32.596 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00083437099999999998 2024-04-26T03:39:32.596 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.596 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.596 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:32.596 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.537471+0000", 2024-04-26T03:39:32.596 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0044802160000000004 2024-04-26T03:39:32.596 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.596 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.596 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:32.596 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.545008+0000", 2024-04-26T03:39:32.596 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0075372089999999996 2024-04-26T03:39:32.596 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.597 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.597 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:32.597 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.545086+0000", 2024-04-26T03:39:32.597 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.7761000000000003e-05 2024-04-26T03:39:32.597 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.597 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:32.597 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.597 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.597 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.597 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:80 3.2s0 3:41b9d80c:::smithi09235267-27:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:32.597 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.532011+0000", 2024-04-26T03:39:32.597 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 41.051991829000002, 2024-04-26T03:39:32.597 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.013370922, 2024-04-26T03:39:32.597 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:32.598 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:32.598 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:32.598 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:32.598 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:32.598 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:32.598 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 80 2024-04-26T03:39:32.598 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.598 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:32.598 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.598 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:32.598 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532011+0000", 2024-04-26T03:39:32.598 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.598 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.598 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.599 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:32.599 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532011+0000", 2024-04-26T03:39:32.599 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.599 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.599 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.599 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:32.599 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532012+0000", 2024-04-26T03:39:32.599 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.1900000000000003e-07 2024-04-26T03:39:32.599 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.599 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.599 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:32.599 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532015+0000", 2024-04-26T03:39:32.599 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5919999999999999e-06 2024-04-26T03:39:32.599 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.600 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.600 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:32.600 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532025+0000", 2024-04-26T03:39:32.600 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0132999999999999e-05 2024-04-26T03:39:32.600 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.600 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.600 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:32.600 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532029+0000", 2024-04-26T03:39:32.600 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9820000000000002e-06 2024-04-26T03:39:32.600 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.600 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.600 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.600 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.536416+0000", 2024-04-26T03:39:32.600 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0043876480000000001 2024-04-26T03:39:32.601 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.601 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.601 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:32.601 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.536558+0000", 2024-04-26T03:39:32.601 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000141994 2024-04-26T03:39:32.601 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.601 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.601 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:32.601 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.537081+0000", 2024-04-26T03:39:32.601 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00052260200000000005 2024-04-26T03:39:32.601 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.601 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.601 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:32.601 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.537579+0000", 2024-04-26T03:39:32.602 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00049790100000000003 2024-04-26T03:39:32.602 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.602 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.602 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:32.602 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.545304+0000", 2024-04-26T03:39:32.602 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.007724936 2024-04-26T03:39:32.602 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.602 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.602 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:32.602 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.545382+0000", 2024-04-26T03:39:32.602 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8615000000000005e-05 2024-04-26T03:39:32.602 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.602 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:32.602 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.603 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.603 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.603 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:81 3.2s0 3:41b9d80c:::smithi09235267-27:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:32.603 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.532034+0000", 2024-04-26T03:39:32.603 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 41.051968915000003, 2024-04-26T03:39:32.603 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.015088236, 2024-04-26T03:39:32.603 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:32.603 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:32.603 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:32.603 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:32.603 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:32.603 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:32.603 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 81 2024-04-26T03:39:32.603 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.603 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:32.604 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.604 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:32.604 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532034+0000", 2024-04-26T03:39:32.604 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.604 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.604 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.604 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:32.604 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532034+0000", 2024-04-26T03:39:32.604 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.604 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.604 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.604 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:32.604 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532035+0000", 2024-04-26T03:39:32.604 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.22e-07 2024-04-26T03:39:32.605 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.605 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.605 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:32.605 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532036+0000", 2024-04-26T03:39:32.605 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6279999999999999e-06 2024-04-26T03:39:32.605 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.605 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.605 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:32.605 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532041+0000", 2024-04-26T03:39:32.605 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.391e-06 2024-04-26T03:39:32.605 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.605 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.605 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:32.605 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532046+0000", 2024-04-26T03:39:32.606 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.4090000000000002e-06 2024-04-26T03:39:32.606 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.606 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.606 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.606 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.537306+0000", 2024-04-26T03:39:32.606 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0052594319999999996 2024-04-26T03:39:32.606 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.606 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.606 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:32.606 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.537400+0000", 2024-04-26T03:39:32.606 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.4753999999999994e-05 2024-04-26T03:39:32.606 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.606 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.606 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.607 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.545720+0000", 2024-04-26T03:39:32.607 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.008319557 2024-04-26T03:39:32.607 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.607 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.607 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:32.607 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.545802+0000", 2024-04-26T03:39:32.607 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.1898999999999996e-05 2024-04-26T03:39:32.607 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.607 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.607 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:32.607 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.547123+0000", 2024-04-26T03:39:32.607 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0013208440000000001 2024-04-26T03:39:32.607 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.607 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:32.608 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.608 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.608 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.608 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:139 3.2s0 3:450a1cc5:::smithi09235267-47:head [append 0~786432,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:32.608 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.619094+0000", 2024-04-26T03:39:32.608 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 40.964908868999999, 2024-04-26T03:39:32.608 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.015918081000000001, 2024-04-26T03:39:32.608 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:32.608 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:32.608 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:32.608 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:32.608 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:32.608 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:32.608 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 139 2024-04-26T03:39:32.608 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.609 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:32.609 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.609 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:32.609 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619094+0000", 2024-04-26T03:39:32.609 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.609 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.609 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.609 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:32.609 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619094+0000", 2024-04-26T03:39:32.609 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.610 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.610 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.610 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:32.610 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619096+0000", 2024-04-26T03:39:32.610 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6559999999999999e-06 2024-04-26T03:39:32.610 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.610 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.610 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:32.610 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619809+0000", 2024-04-26T03:39:32.610 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00071288600000000003 2024-04-26T03:39:32.610 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.610 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.610 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:32.610 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619821+0000", 2024-04-26T03:39:32.611 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1586000000000001e-05 2024-04-26T03:39:32.611 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.611 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.611 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:32.611 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619826+0000", 2024-04-26T03:39:32.611 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.3870000000000003e-06 2024-04-26T03:39:32.611 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.611 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.611 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.611 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619879+0000", 2024-04-26T03:39:32.612 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.2660000000000001e-05 2024-04-26T03:39:32.612 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.612 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.612 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:32.612 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619966+0000", 2024-04-26T03:39:32.612 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7777999999999999e-05 2024-04-26T03:39:32.612 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.612 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.612 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:32.612 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.620812+0000", 2024-04-26T03:39:32.612 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00084571199999999998 2024-04-26T03:39:32.612 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.612 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.612 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:32.613 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.627482+0000", 2024-04-26T03:39:32.613 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0066701590000000002 2024-04-26T03:39:32.613 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.613 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.613 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:32.613 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.634919+0000", 2024-04-26T03:39:32.613 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.007437056 2024-04-26T03:39:32.613 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.613 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.613 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:32.613 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.635013+0000", 2024-04-26T03:39:32.613 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.3201000000000004e-05 2024-04-26T03:39:32.613 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.613 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:32.614 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.614 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.614 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.614 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:140 3.2s0 3:450a1cc5:::smithi09235267-47:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:32.614 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.619840+0000", 2024-04-26T03:39:32.614 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 40.964163808000002, 2024-04-26T03:39:32.614 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.015356137000000001, 2024-04-26T03:39:32.614 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:32.614 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:32.614 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:32.614 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:32.614 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:32.614 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:32.615 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 140 2024-04-26T03:39:32.615 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.615 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:32.615 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.615 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:32.615 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619840+0000", 2024-04-26T03:39:32.615 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.615 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.615 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.615 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:32.615 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619840+0000", 2024-04-26T03:39:32.615 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.615 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.615 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.616 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:32.616 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619840+0000", 2024-04-26T03:39:32.616 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.8100000000000002e-07 2024-04-26T03:39:32.616 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.616 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.616 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:32.616 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619843+0000", 2024-04-26T03:39:32.616 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.785e-06 2024-04-26T03:39:32.616 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.616 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.616 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:32.616 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619848+0000", 2024-04-26T03:39:32.616 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.2490000000000001e-06 2024-04-26T03:39:32.616 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.616 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.617 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:32.617 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619852+0000", 2024-04-26T03:39:32.617 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.1869999999999999e-06 2024-04-26T03:39:32.617 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.617 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.617 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.617 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.626570+0000", 2024-04-26T03:39:32.617 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0067177679999999998 2024-04-26T03:39:32.617 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.617 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.617 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:32.617 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.626669+0000", 2024-04-26T03:39:32.617 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.9177000000000003e-05 2024-04-26T03:39:32.617 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.617 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.618 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:32.618 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.627020+0000", 2024-04-26T03:39:32.618 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00035101199999999999 2024-04-26T03:39:32.618 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.618 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.618 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:32.618 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.627486+0000", 2024-04-26T03:39:32.618 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00046560299999999998 2024-04-26T03:39:32.618 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.618 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.618 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:32.618 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.635101+0000", 2024-04-26T03:39:32.618 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0076154680000000002 2024-04-26T03:39:32.619 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.619 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.619 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:32.619 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.635196+0000", 2024-04-26T03:39:32.619 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.4207000000000001e-05 2024-04-26T03:39:32.619 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.619 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:32.619 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.619 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.619 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.619 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:141 3.2s0 3:450a1cc5:::smithi09235267-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:32.619 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.619858+0000", 2024-04-26T03:39:32.619 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 40.964145475999999, 2024-04-26T03:39:32.619 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.017889577, 2024-04-26T03:39:32.619 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:32.620 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:32.620 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:32.620 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:32.620 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:32.620 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:32.620 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 141 2024-04-26T03:39:32.620 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.620 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:32.620 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.620 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:32.620 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619858+0000", 2024-04-26T03:39:32.620 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.620 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.620 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.621 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:32.621 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619858+0000", 2024-04-26T03:39:32.621 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.621 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.621 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.621 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:32.621 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619858+0000", 2024-04-26T03:39:32.621 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7000000000000001e-07 2024-04-26T03:39:32.621 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.621 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.621 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:32.621 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619860+0000", 2024-04-26T03:39:32.621 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1090000000000001e-06 2024-04-26T03:39:32.621 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.621 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.622 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:32.622 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619864+0000", 2024-04-26T03:39:32.622 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.1330000000000001e-06 2024-04-26T03:39:32.622 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.622 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.622 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:32.622 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619867+0000", 2024-04-26T03:39:32.622 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.9100000000000001e-06 2024-04-26T03:39:32.622 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.622 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.622 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.622 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.627233+0000", 2024-04-26T03:39:32.622 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0073655659999999996 2024-04-26T03:39:32.622 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.623 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.623 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:32.623 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.627322+0000", 2024-04-26T03:39:32.623 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.9456999999999994e-05 2024-04-26T03:39:32.623 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.623 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.623 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.623 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.635530+0000", 2024-04-26T03:39:32.623 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0082076349999999996 2024-04-26T03:39:32.623 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.623 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.623 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:32.623 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.635609+0000", 2024-04-26T03:39:32.623 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8724e-05 2024-04-26T03:39:32.623 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.624 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.624 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:32.624 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.637747+0000", 2024-04-26T03:39:32.624 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0021387730000000001 2024-04-26T03:39:32.624 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.624 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:32.624 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.624 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.624 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.624 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:223 3.2s0 3:42fcabaa:::smithi09235267-1:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:32.624 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.838898+0000", 2024-04-26T03:39:32.624 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 40.745104951999998, 2024-04-26T03:39:32.624 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.012225915, 2024-04-26T03:39:32.624 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:32.625 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:32.625 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:32.625 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:32.625 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:32.625 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:32.625 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 223 2024-04-26T03:39:32.625 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.625 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:32.625 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.625 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:32.625 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.838898+0000", 2024-04-26T03:39:32.625 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.625 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.625 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.625 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:32.626 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.838898+0000", 2024-04-26T03:39:32.626 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.626 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.626 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.626 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:32.626 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.838899+0000", 2024-04-26T03:39:32.626 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.7e-07 2024-04-26T03:39:32.626 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.626 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.626 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:32.626 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.838901+0000", 2024-04-26T03:39:32.627 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5799999999999999e-06 2024-04-26T03:39:32.627 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.627 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.627 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:32.627 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.838910+0000", 2024-04-26T03:39:32.627 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.1189999999999997e-06 2024-04-26T03:39:32.627 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.627 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.627 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:32.627 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.838916+0000", 2024-04-26T03:39:32.627 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.8630000000000002e-06 2024-04-26T03:39:32.627 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.627 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.627 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.627 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.845205+0000", 2024-04-26T03:39:32.628 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0062883449999999999 2024-04-26T03:39:32.628 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.628 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.628 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:32.628 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.845254+0000", 2024-04-26T03:39:32.628 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.9101000000000003e-05 2024-04-26T03:39:32.628 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.628 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.628 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.628 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.850264+0000", 2024-04-26T03:39:32.628 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0050106079999999997 2024-04-26T03:39:32.628 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.628 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.628 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:32.629 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.850341+0000", 2024-04-26T03:39:32.629 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.6654000000000002e-05 2024-04-26T03:39:32.629 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.629 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.629 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:32.629 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.851124+0000", 2024-04-26T03:39:32.629 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000783175 2024-04-26T03:39:32.629 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.629 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:32.629 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.629 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.629 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.629 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.es1 43/32 ECSubWrite(tid=48, reqid=client.4556.0:301, at_version=43'16, trim_to=0'0, roll_forward_to=35'14))", 2024-04-26T03:39:32.629 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:59.531315+0000", 2024-04-26T03:39:32.630 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 33.052688001, 2024-04-26T03:39:32.630 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.014140454, 2024-04-26T03:39:32.630 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:32.630 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:32.630 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:32.630 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:32.630 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.630 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:32.630 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.531315+0000", 2024-04-26T03:39:32.630 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.630 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.630 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.630 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:32.630 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.531315+0000", 2024-04-26T03:39:32.631 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.631 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.631 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.631 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:32.631 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.531316+0000", 2024-04-26T03:39:32.631 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1799999999999999e-07 2024-04-26T03:39:32.631 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.631 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.631 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:32.631 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.531691+0000", 2024-04-26T03:39:32.631 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00037520700000000001 2024-04-26T03:39:32.631 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.631 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.631 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:32.632 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.531705+0000", 2024-04-26T03:39:32.632 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.395e-05 2024-04-26T03:39:32.632 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.632 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.632 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:32.632 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.531713+0000", 2024-04-26T03:39:32.632 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.0150000000000003e-06 2024-04-26T03:39:32.632 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.632 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.632 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.632 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.531798+0000", 2024-04-26T03:39:32.632 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5714e-05 2024-04-26T03:39:32.632 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.632 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.632 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:32.633 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.531811+0000", 2024-04-26T03:39:32.633 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2782000000000001e-05 2024-04-26T03:39:32.633 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.633 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.633 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:32.633 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.545405+0000", 2024-04-26T03:39:32.633 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.013593888 2024-04-26T03:39:32.633 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.633 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.633 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:32.633 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.545456+0000", 2024-04-26T03:39:32.633 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.0680000000000003e-05 2024-04-26T03:39:32.633 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.633 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:32.634 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.634 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.634 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.634 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.fs3 45/32 ECSubWrite(tid=49, reqid=client.4556.0:325, at_version=45'17, trim_to=0'0, roll_forward_to=42'15))", 2024-04-26T03:39:32.634 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:01.554312+0000", 2024-04-26T03:39:32.634 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 31.029690905999999, 2024-04-26T03:39:32.634 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.012529861, 2024-04-26T03:39:32.634 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:32.634 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:32.634 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:32.634 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:32.634 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.634 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:32.634 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.554312+0000", 2024-04-26T03:39:32.635 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.635 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.635 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.635 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:32.635 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.554312+0000", 2024-04-26T03:39:32.635 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.635 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.635 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.635 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:32.635 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.554313+0000", 2024-04-26T03:39:32.635 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.9999999999999996e-07 2024-04-26T03:39:32.635 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.635 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.635 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:32.636 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.554865+0000", 2024-04-26T03:39:32.636 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00055152399999999996 2024-04-26T03:39:32.636 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.636 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.636 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:32.636 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.554886+0000", 2024-04-26T03:39:32.636 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1418999999999999e-05 2024-04-26T03:39:32.636 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.636 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.636 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:32.636 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.554897+0000", 2024-04-26T03:39:32.636 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0994e-05 2024-04-26T03:39:32.636 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.636 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.636 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.637 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.554971+0000", 2024-04-26T03:39:32.637 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3516000000000006e-05 2024-04-26T03:39:32.637 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.637 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.637 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:32.637 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.554983+0000", 2024-04-26T03:39:32.637 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2133999999999999e-05 2024-04-26T03:39:32.637 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.637 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.637 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:32.637 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.566763+0000", 2024-04-26T03:39:32.637 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.011780067 2024-04-26T03:39:32.637 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.637 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.638 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:32.638 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.566842+0000", 2024-04-26T03:39:32.638 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9307000000000005e-05 2024-04-26T03:39:32.638 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.638 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:32.638 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.638 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.638 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.638 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:333 3.2s0 3:450a1cc5:::smithi09235267-47:head [copy-from ver 6] snapc 7=[7,6,5] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-26T03:39:32.639 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:01.558145+0000", 2024-04-26T03:39:32.639 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 31.025858567, 2024-04-26T03:39:32.639 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.017282072999999998, 2024-04-26T03:39:32.639 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:32.639 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:32.639 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:32.639 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:32.639 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:32.639 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:32.639 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 333 2024-04-26T03:39:32.639 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.639 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:32.639 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.639 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:32.639 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558145+0000", 2024-04-26T03:39:32.640 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.640 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.640 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.640 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:32.640 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558145+0000", 2024-04-26T03:39:32.640 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.640 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.640 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.640 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:32.640 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558147+0000", 2024-04-26T03:39:32.640 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4360000000000001e-06 2024-04-26T03:39:32.640 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.640 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.640 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:32.641 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558166+0000", 2024-04-26T03:39:32.641 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8411999999999999e-05 2024-04-26T03:39:32.641 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.641 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.641 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:32.641 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558181+0000", 2024-04-26T03:39:32.641 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4956e-05 2024-04-26T03:39:32.641 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.641 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.641 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:32.641 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558190+0000", 2024-04-26T03:39:32.641 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.7599999999999997e-06 2024-04-26T03:39:32.641 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.641 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.641 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.642 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558309+0000", 2024-04-26T03:39:32.642 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000118893 2024-04-26T03:39:32.642 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.642 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.642 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:32.642 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558491+0000", 2024-04-26T03:39:32.642 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00018157000000000001 2024-04-26T03:39:32.642 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.642 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.642 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:32.642 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.566556+0000", 2024-04-26T03:39:32.642 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0080647549999999998 2024-04-26T03:39:32.642 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.643 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.643 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:32.643 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.572889+0000", 2024-04-26T03:39:32.643 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0063337100000000002 2024-04-26T03:39:32.643 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.643 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.643 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:32.643 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.575342+0000", 2024-04-26T03:39:32.643 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0024523269999999998 2024-04-26T03:39:32.643 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.643 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.643 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:32.643 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.575427+0000", 2024-04-26T03:39:32.643 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5253999999999994e-05 2024-04-26T03:39:32.643 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.644 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:32.644 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.644 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.644 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.644 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:334 3.2s0 3:450a1cc5:::smithi09235267-47:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e45)", 2024-04-26T03:39:32.644 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:01.558215+0000", 2024-04-26T03:39:32.644 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 31.025788416000001, 2024-04-26T03:39:32.644 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.013358238999999999, 2024-04-26T03:39:32.644 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:32.644 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:32.644 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:32.644 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:32.644 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:32.644 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:32.645 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 334 2024-04-26T03:39:32.645 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.645 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:32.645 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.645 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:32.645 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558215+0000", 2024-04-26T03:39:32.645 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.645 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.645 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.645 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:32.645 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558215+0000", 2024-04-26T03:39:32.645 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.645 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.645 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.646 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:32.646 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558216+0000", 2024-04-26T03:39:32.646 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.0800000000000004e-07 2024-04-26T03:39:32.646 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.646 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.646 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:32.646 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558220+0000", 2024-04-26T03:39:32.646 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8410000000000002e-06 2024-04-26T03:39:32.646 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.646 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.646 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:32.647 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558233+0000", 2024-04-26T03:39:32.647 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3556e-05 2024-04-26T03:39:32.647 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.647 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.647 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:32.647 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558241+0000", 2024-04-26T03:39:32.647 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.4080000000000001e-06 2024-04-26T03:39:32.647 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.647 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.647 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.647 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558731+0000", 2024-04-26T03:39:32.647 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00049039899999999998 2024-04-26T03:39:32.647 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.647 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.647 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for blocked object", 2024-04-26T03:39:32.648 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558777+0000", 2024-04-26T03:39:32.648 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6301000000000003e-05 2024-04-26T03:39:32.648 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.648 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.648 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.648 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.571433+0000", 2024-04-26T03:39:32.648 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.012656137 2024-04-26T03:39:32.648 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.648 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.648 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:32.648 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.571501+0000", 2024-04-26T03:39:32.648 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7319999999999999e-05 2024-04-26T03:39:32.649 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.649 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.649 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:32.649 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.571573+0000", 2024-04-26T03:39:32.649 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.2469e-05 2024-04-26T03:39:32.649 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.649 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:32.649 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.649 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.649 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.649 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:433 3.2s0 3:42fcabaa:::smithi09235267-1:head [append 0~671744] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-04-26T03:39:32.649 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:04.692691+0000", 2024-04-26T03:39:32.649 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 27.891311925, 2024-04-26T03:39:32.649 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.026947377000000002, 2024-04-26T03:39:32.650 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:32.650 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:32.650 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:32.650 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:32.650 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:32.650 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:32.650 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 433 2024-04-26T03:39:32.650 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.650 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:32.650 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.650 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:32.650 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.692691+0000", 2024-04-26T03:39:32.650 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.650 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.651 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.651 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:32.651 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.692691+0000", 2024-04-26T03:39:32.651 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.651 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.651 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.651 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:32.651 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.692694+0000", 2024-04-26T03:39:32.651 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.216e-06 2024-04-26T03:39:32.651 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.651 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.651 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:32.651 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.693610+0000", 2024-04-26T03:39:32.651 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00091655899999999995 2024-04-26T03:39:32.651 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.652 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.652 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:32.652 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.693622+0000", 2024-04-26T03:39:32.652 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1976e-05 2024-04-26T03:39:32.652 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.652 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.652 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:32.652 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.693629+0000", 2024-04-26T03:39:32.652 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5749999999999997e-06 2024-04-26T03:39:32.652 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.652 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.652 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.652 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.693698+0000", 2024-04-26T03:39:32.652 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.9529000000000005e-05 2024-04-26T03:39:32.652 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.653 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.653 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op must wait for map", 2024-04-26T03:39:32.653 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.693708+0000", 2024-04-26T03:39:32.653 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.6649999999999998e-06 2024-04-26T03:39:32.653 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.653 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.653 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.653 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.698905+0000", 2024-04-26T03:39:32.653 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.005197279 2024-04-26T03:39:32.653 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.653 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.653 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:32.653 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.698993+0000", 2024-04-26T03:39:32.654 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.8029999999999996e-05 2024-04-26T03:39:32.654 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.654 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.654 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:32.654 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.699957+0000", 2024-04-26T03:39:32.654 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00096368199999999999 2024-04-26T03:39:32.654 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.654 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.654 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:32.654 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.718295+0000", 2024-04-26T03:39:32.654 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.018338255000000001 2024-04-26T03:39:32.654 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.654 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.654 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:32.655 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.719556+0000", 2024-04-26T03:39:32.655 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001261086 2024-04-26T03:39:32.655 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.655 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.655 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:32.655 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.719639+0000", 2024-04-26T03:39:32.655 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2525e-05 2024-04-26T03:39:32.655 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.655 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:32.655 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.655 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.655 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.655 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:434 3.2s0 3:42fcabaa:::smithi09235267-1:head [append 0~770048] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-04-26T03:39:32.655 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:04.699291+0000", 2024-04-26T03:39:32.655 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 27.884712228000001, 2024-04-26T03:39:32.656 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.029702725999999999, 2024-04-26T03:39:32.656 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:32.656 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:32.656 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:32.656 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:32.656 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:32.656 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:32.656 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 434 2024-04-26T03:39:32.656 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.656 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:32.656 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.656 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:32.656 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.699291+0000", 2024-04-26T03:39:32.656 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.657 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.657 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.657 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:32.657 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.699291+0000", 2024-04-26T03:39:32.657 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.657 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.657 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.657 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:32.657 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.699292+0000", 2024-04-26T03:39:32.657 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.192e-06 2024-04-26T03:39:32.657 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.657 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.657 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:32.657 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700019+0000", 2024-04-26T03:39:32.658 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00072667100000000002 2024-04-26T03:39:32.658 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.658 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.658 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:32.658 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700032+0000", 2024-04-26T03:39:32.658 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3064999999999999e-05 2024-04-26T03:39:32.658 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.658 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.658 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:32.658 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700040+0000", 2024-04-26T03:39:32.658 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5299999999999999e-06 2024-04-26T03:39:32.658 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.658 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.658 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.659 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.711078+0000", 2024-04-26T03:39:32.659 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.011038536 2024-04-26T03:39:32.659 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.659 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.659 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:32.659 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.711181+0000", 2024-04-26T03:39:32.659 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000102833 2024-04-26T03:39:32.659 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.659 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.659 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:32.659 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.712518+0000", 2024-04-26T03:39:32.659 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0013371769999999999 2024-04-26T03:39:32.659 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.659 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.659 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:32.660 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.719306+0000", 2024-04-26T03:39:32.660 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0067879009999999998 2024-04-26T03:39:32.660 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.660 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.660 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:32.660 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728932+0000", 2024-04-26T03:39:32.660 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0096258230000000004 2024-04-26T03:39:32.660 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.660 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.660 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:32.660 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728994+0000", 2024-04-26T03:39:32.660 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.1997999999999998e-05 2024-04-26T03:39:32.660 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.660 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:32.661 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.661 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.661 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.661 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:435 3.2s0 3:42fcabaa:::smithi09235267-1:head [setxattr _header (55) in=7b] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-04-26T03:39:32.661 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:04.700054+0000", 2024-04-26T03:39:32.661 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 27.883948970999999, 2024-04-26T03:39:32.661 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.029111279, 2024-04-26T03:39:32.661 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:32.661 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:32.661 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:32.661 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:32.661 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:32.661 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:32.661 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 435 2024-04-26T03:39:32.661 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.662 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:32.662 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.662 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:32.662 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700054+0000", 2024-04-26T03:39:32.662 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.662 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.662 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.662 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:32.662 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700054+0000", 2024-04-26T03:39:32.662 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.662 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.662 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.662 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:32.662 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700055+0000", 2024-04-26T03:39:32.663 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0470000000000001e-06 2024-04-26T03:39:32.663 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.663 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.663 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:32.663 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700059+0000", 2024-04-26T03:39:32.663 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8909999999999997e-06 2024-04-26T03:39:32.663 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.663 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.663 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:32.663 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700067+0000", 2024-04-26T03:39:32.663 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.447e-06 2024-04-26T03:39:32.663 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.663 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.663 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:32.664 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700072+0000", 2024-04-26T03:39:32.664 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7529999999999998e-06 2024-04-26T03:39:32.664 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.664 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.664 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.664 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.718645+0000", 2024-04-26T03:39:32.664 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.018572564 2024-04-26T03:39:32.664 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.664 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.664 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:32.664 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.718711+0000", 2024-04-26T03:39:32.664 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5743999999999999e-05 2024-04-26T03:39:32.664 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.664 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.664 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:32.665 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.718985+0000", 2024-04-26T03:39:32.665 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00027395400000000003 2024-04-26T03:39:32.665 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.665 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.665 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:32.665 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.719895+0000", 2024-04-26T03:39:32.665 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00091011499999999997 2024-04-26T03:39:32.665 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.665 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.665 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:32.665 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.729090+0000", 2024-04-26T03:39:32.665 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0091950920000000002 2024-04-26T03:39:32.665 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.665 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.665 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:32.666 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.729166+0000", 2024-04-26T03:39:32.666 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5672000000000003e-05 2024-04-26T03:39:32.666 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.666 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:32.666 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.666 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.666 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.666 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:436 3.2s0 3:42fcabaa:::smithi09235267-1:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e49)", 2024-04-26T03:39:32.666 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:04.700082+0000", 2024-04-26T03:39:32.666 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 27.883921325999999, 2024-04-26T03:39:32.666 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.032317973, 2024-04-26T03:39:32.666 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:32.666 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:32.666 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:32.667 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:32.667 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:32.667 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:32.667 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 436 2024-04-26T03:39:32.667 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.667 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:32.667 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.667 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:32.667 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700082+0000", 2024-04-26T03:39:32.667 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.667 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.667 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.667 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:32.667 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700082+0000", 2024-04-26T03:39:32.667 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.668 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.668 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.668 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:32.668 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700082+0000", 2024-04-26T03:39:32.668 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.3000000000000001e-07 2024-04-26T03:39:32.668 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.668 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.668 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:32.668 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700085+0000", 2024-04-26T03:39:32.668 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4739999999999999e-06 2024-04-26T03:39:32.668 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.668 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.668 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:32.668 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700092+0000", 2024-04-26T03:39:32.669 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3000000000000004e-06 2024-04-26T03:39:32.669 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.669 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.669 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:32.669 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700097+0000", 2024-04-26T03:39:32.669 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.2290000000000002e-06 2024-04-26T03:39:32.669 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.669 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.669 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.669 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.719230+0000", 2024-04-26T03:39:32.669 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.019132564000000001 2024-04-26T03:39:32.669 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.669 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.669 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:32.670 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.719296+0000", 2024-04-26T03:39:32.670 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.6370000000000003e-05 2024-04-26T03:39:32.670 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.670 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.670 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.670 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.729620+0000", 2024-04-26T03:39:32.670 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.010323183999999999 2024-04-26T03:39:32.670 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.670 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.670 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:32.670 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.729695+0000", 2024-04-26T03:39:32.670 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5500000000000006e-05 2024-04-26T03:39:32.670 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.670 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.671 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:32.671 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.732400+0000", 2024-04-26T03:39:32.671 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0027049219999999998 2024-04-26T03:39:32.671 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.671 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:32.671 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.671 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.671 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.671 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:478 3.2s0 3:450a1cc5:::smithi09235267-47:head [append 0~802816] snapc 8=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:32.671 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:05.910200+0000", 2024-04-26T03:39:32.671 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.673803642999999, 2024-04-26T03:39:32.671 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.013459199999999999, 2024-04-26T03:39:32.671 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:32.671 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:32.672 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:32.672 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:32.672 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:32.672 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:32.672 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 478 2024-04-26T03:39:32.672 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.672 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:32.672 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.672 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:32.672 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.910200+0000", 2024-04-26T03:39:32.672 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.672 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.672 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.672 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:32.672 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.910200+0000", 2024-04-26T03:39:32.673 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.673 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.673 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.673 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:32.673 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.910202+0000", 2024-04-26T03:39:32.673 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.683e-06 2024-04-26T03:39:32.673 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.673 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.673 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:32.673 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.910944+0000", 2024-04-26T03:39:32.673 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00074142299999999995 2024-04-26T03:39:32.673 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.673 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.673 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:32.674 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.910959+0000", 2024-04-26T03:39:32.674 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4909000000000001e-05 2024-04-26T03:39:32.674 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.674 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.674 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:32.674 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.910966+0000", 2024-04-26T03:39:32.674 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.827e-06 2024-04-26T03:39:32.674 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.674 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.674 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.674 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.911019+0000", 2024-04-26T03:39:32.674 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.3817999999999999e-05 2024-04-26T03:39:32.674 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.674 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.675 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:32.675 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.911094+0000", 2024-04-26T03:39:32.675 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5059000000000001e-05 2024-04-26T03:39:32.675 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.675 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.675 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:32.675 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.911822+0000", 2024-04-26T03:39:32.675 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00072724499999999995 2024-04-26T03:39:32.675 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.675 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.675 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:32.675 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.923328+0000", 2024-04-26T03:39:32.675 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.01150674 2024-04-26T03:39:32.675 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.675 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.676 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:32.676 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.923574+0000", 2024-04-26T03:39:32.676 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00024563200000000002 2024-04-26T03:39:32.676 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.676 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.676 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:32.676 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.923659+0000", 2024-04-26T03:39:32.676 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.4864000000000006e-05 2024-04-26T03:39:32.676 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.676 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:32.676 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.676 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.676 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.676 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:481 3.2s0 3:450a1cc5:::smithi09235267-47:head [setxattr _header (56) in=7b] snapc 8=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:32.676 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:05.914415+0000", 2024-04-26T03:39:32.677 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.669588654000002, 2024-04-26T03:39:32.677 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.011349236, 2024-04-26T03:39:32.677 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:32.677 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:32.677 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:32.677 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:32.677 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:32.677 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:32.677 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 481 2024-04-26T03:39:32.677 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.677 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:32.677 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.677 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:32.677 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.914415+0000", 2024-04-26T03:39:32.677 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.678 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.678 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.678 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:32.678 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.914415+0000", 2024-04-26T03:39:32.678 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.678 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.678 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.678 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:32.678 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.914415+0000", 2024-04-26T03:39:32.678 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.8800000000000003e-07 2024-04-26T03:39:32.678 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.678 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.678 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:32.678 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.914422+0000", 2024-04-26T03:39:32.679 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.9199999999999998e-06 2024-04-26T03:39:32.679 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.679 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.679 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:32.679 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.914431+0000", 2024-04-26T03:39:32.679 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.8559999999999996e-06 2024-04-26T03:39:32.679 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.679 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.679 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:32.679 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.914438+0000", 2024-04-26T03:39:32.679 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.9199999999999998e-06 2024-04-26T03:39:32.679 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.679 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.679 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.680 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.924215+0000", 2024-04-26T03:39:32.680 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0097767129999999994 2024-04-26T03:39:32.680 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.680 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.680 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:32.680 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.924272+0000", 2024-04-26T03:39:32.680 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7003000000000003e-05 2024-04-26T03:39:32.680 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.680 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.680 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:32.680 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.924552+0000", 2024-04-26T03:39:32.680 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00028005100000000002 2024-04-26T03:39:32.680 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.680 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.680 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:32.681 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.925009+0000", 2024-04-26T03:39:32.681 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00045688899999999998 2024-04-26T03:39:32.681 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.681 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.681 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:32.681 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.925717+0000", 2024-04-26T03:39:32.681 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000708339 2024-04-26T03:39:32.681 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.681 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.682 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:32.682 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.925764+0000", 2024-04-26T03:39:32.682 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.7057e-05 2024-04-26T03:39:32.682 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.682 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:32.682 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.682 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.682 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.682 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:482 3.2s0 3:450a1cc5:::smithi09235267-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:32.682 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:05.914446+0000", 2024-04-26T03:39:32.682 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.669557076, 2024-04-26T03:39:32.682 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.01384406, 2024-04-26T03:39:32.682 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:32.682 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:32.683 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:32.683 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:32.683 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:32.683 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:32.683 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 482 2024-04-26T03:39:32.683 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.683 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:32.683 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.683 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:32.683 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.914446+0000", 2024-04-26T03:39:32.683 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.683 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.683 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.683 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:32.684 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.914446+0000", 2024-04-26T03:39:32.684 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.684 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.684 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.684 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:32.684 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.914447+0000", 2024-04-26T03:39:32.684 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.1600000000000001e-07 2024-04-26T03:39:32.684 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.684 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.684 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:32.684 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.914449+0000", 2024-04-26T03:39:32.684 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4229999999999999e-06 2024-04-26T03:39:32.684 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.684 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.685 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:32.685 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.914458+0000", 2024-04-26T03:39:32.685 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.9299999999999992e-06 2024-04-26T03:39:32.685 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.685 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.685 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:32.685 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.914465+0000", 2024-04-26T03:39:32.685 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.6599999999999998e-06 2024-04-26T03:39:32.685 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.685 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.685 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.685 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.924755+0000", 2024-04-26T03:39:32.685 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.010289777 2024-04-26T03:39:32.685 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.685 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.686 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:32.686 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.924801+0000", 2024-04-26T03:39:32.686 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6109000000000002e-05 2024-04-26T03:39:32.686 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.686 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.686 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.686 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.926051+0000", 2024-04-26T03:39:32.686 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001250393 2024-04-26T03:39:32.686 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.686 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.686 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:32.686 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.926140+0000", 2024-04-26T03:39:32.686 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.9227999999999993e-05 2024-04-26T03:39:32.686 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.687 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.687 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:32.687 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.928290+0000", 2024-04-26T03:39:32.687 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0021500239999999999 2024-04-26T03:39:32.687 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.687 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:32.687 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.687 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.687 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.687 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.fs3 65/62 ECSubWrite(tid=178, reqid=client.4556.0:782, at_version=65'60, trim_to=0'0, roll_forward_to=64'57))", 2024-04-26T03:39:32.687 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:20.829001+0000", 2024-04-26T03:39:32.687 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 11.755002108999999, 2024-04-26T03:39:32.687 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.012433502000000001, 2024-04-26T03:39:32.687 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:32.688 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:32.688 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:32.688 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:32.688 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.688 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:32.688 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.829001+0000", 2024-04-26T03:39:32.688 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.688 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.688 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.688 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:32.688 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.829001+0000", 2024-04-26T03:39:32.688 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:32.688 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.688 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.689 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:32.689 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.829002+0000", 2024-04-26T03:39:32.689 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.6000000000000003e-07 2024-04-26T03:39:32.689 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.689 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.689 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:32.689 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.829342+0000", 2024-04-26T03:39:32.689 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00033970400000000003 2024-04-26T03:39:32.689 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.689 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.689 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:32.689 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.829356+0000", 2024-04-26T03:39:32.689 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.434e-05 2024-04-26T03:39:32.689 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.690 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.690 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:32.690 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.829365+0000", 2024-04-26T03:39:32.690 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.0149999999999992e-06 2024-04-26T03:39:32.690 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.690 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.690 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:32.690 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.829431+0000", 2024-04-26T03:39:32.690 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.6080000000000004e-05 2024-04-26T03:39:32.690 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.690 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.690 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:32.690 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.829442+0000", 2024-04-26T03:39:32.690 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1439e-05 2024-04-26T03:39:32.691 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.691 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.691 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:32.691 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.841390+0000", 2024-04-26T03:39:32.691 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.01194714 2024-04-26T03:39:32.691 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:32.691 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:32.691 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:32.691 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.841435+0000", 2024-04-26T03:39:32.691 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.5123999999999997e-05 2024-04-26T03:39:32.691 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.691 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:32.691 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.691 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:32.691 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:32.692 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:39:32.693 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0: {} 2024-04-26T03:39:32.693 INFO:teuthology.orchestra.run.smithi028.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T03:39:32.694 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1: {} 2024-04-26T03:39:32.694 INFO:teuthology.orchestra.run.smithi028.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T03:39:32.694 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-26T03:39:32.695 INFO:tasks.ceph.osd.10.smithi066.stderr:2024-04-26T03:39:32.692+0000 7f130fcc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 31763) UID: 0 2024-04-26T03:39:32.695 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2: {} 2024-04-26T03:39:32.695 INFO:teuthology.orchestra.run.smithi028.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T03:39:32.696 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3: {} 2024-04-26T03:39:32.696 INFO:teuthology.orchestra.run.smithi028.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T03:39:32.696 INFO:teuthology.orchestra.run.smithi028.stderr:nodeep-scrub is unset 2024-04-26T03:39:32.696 INFO:teuthology.orchestra.run.smithi028.stderr:reweighted osd.4 to 1 (10000) 2024-04-26T03:39:32.697 INFO:tasks.rados.rados.0.smithi092.stdout:337: done (0 left) 2024-04-26T03:39:32.697 INFO:tasks.rados.rados.0.smithi092.stdout:338: read oid 37 snap 23 2024-04-26T03:39:32.698 INFO:tasks.rados.rados.0.smithi092.stdout:338: expect (ObjNum 40 snap 0 seq_num 40) 2024-04-26T03:39:32.698 INFO:tasks.rados.rados.0.smithi092.stdout:339: read oid 11 snap -1 2024-04-26T03:39:32.698 INFO:tasks.rados.rados.0.smithi092.stdout:339: expect deleted 2024-04-26T03:39:32.698 INFO:tasks.rados.rados.0.smithi092.stdout:340: snap_remove snap 14 2024-04-26T03:39:32.709 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4: {} 2024-04-26T03:39:32.710 INFO:teuthology.orchestra.run.smithi028.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T03:39:32.714 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-04-26T03:39:32.727 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5: {} 2024-04-26T03:39:32.728 INFO:teuthology.orchestra.run.smithi028.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T03:39:32.745 INFO:teuthology.orchestra.run.smithi028.stdout:osd.6: {} 2024-04-26T03:39:32.745 INFO:teuthology.orchestra.run.smithi028.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T03:39:32.766 INFO:teuthology.orchestra.run.smithi028.stdout:osd.7: {} 2024-04-26T03:39:32.766 INFO:teuthology.orchestra.run.smithi028.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T03:39:32.780 INFO:teuthology.orchestra.run.smithi028.stdout:osd.8: {} 2024-04-26T03:39:32.780 INFO:teuthology.orchestra.run.smithi028.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T03:39:32.794 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-26T03:39:32.792+0000 7f610b3a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31816) UID: 0 2024-04-26T03:39:32.799 INFO:teuthology.orchestra.run.smithi028.stdout:osd.9: {} 2024-04-26T03:39:32.800 INFO:teuthology.orchestra.run.smithi028.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T03:39:32.809 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:32.809 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:32.817 INFO:teuthology.orchestra.run.smithi028.stdout:osd.10: {} 2024-04-26T03:39:32.818 INFO:teuthology.orchestra.run.smithi028.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T03:39:32.819 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-26T03:39:32.835 INFO:teuthology.orchestra.run.smithi028.stdout:osd.11: {} 2024-04-26T03:39:32.835 INFO:teuthology.orchestra.run.smithi028.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T03:39:32.854 INFO:teuthology.orchestra.run.smithi028.stdout:osd.12: {} 2024-04-26T03:39:32.854 INFO:teuthology.orchestra.run.smithi028.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T03:39:32.872 INFO:teuthology.orchestra.run.smithi028.stdout:osd.13: {} 2024-04-26T03:39:32.872 INFO:teuthology.orchestra.run.smithi028.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T03:39:32.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:39:32.871+0000 7f0422ffd640 0 -- 172.21.15.28:0/2282137682 >> [v2:172.21.15.66:6824/1534905602,v1:172.21.15.66:6825/1534905602] conn(0x7f042421b3d0 msgr2=0x7f042421d7c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-26T03:39:32.890 INFO:teuthology.orchestra.run.smithi028.stdout:osd.14: {} 2024-04-26T03:39:32.890 INFO:teuthology.orchestra.run.smithi028.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T03:39:32.894 INFO:tasks.ceph.osd.4.smithi028.stderr:2024-04-26T03:39:32.893+0000 7fd9511b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31963) UID: 0 2024-04-26T03:39:32.908 INFO:teuthology.orchestra.run.smithi028.stdout:osd.15: {} 2024-04-26T03:39:32.908 INFO:teuthology.orchestra.run.smithi028.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T03:39:32.962 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:32.962 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:32.971 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-26T03:39:33.015 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-04-26T03:39:33.014+0000 7fdc2d21c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31819) UID: 0 2024-04-26T03:39:33.073 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:33.074 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:33.082 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-26T03:39:33.123 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-04-26T03:39:33.122+0000 7f745118d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 31821) UID: 0 2024-04-26T03:39:33.197 INFO:tasks.ceph.osd.14.smithi066.stderr:2024-04-26T03:39:33.196+0000 7f5395f08640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 31765) UID: 0 2024-04-26T03:39:33.225 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:33.225 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:33.233 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-26T03:39:33.298 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:39:33.296+0000 7fba0d2ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31961) UID: 0 2024-04-26T03:39:33.336 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:33.336 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:33.344 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-26T03:39:33.399 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-04-26T03:39:33.398+0000 7fee562fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31759) UID: 0 2024-04-26T03:39:33.447 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:33.447 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:33.456 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-04-26T03:39:33.500 INFO:tasks.ceph.osd.9.smithi042.stderr:2024-04-26T03:39:33.499+0000 7fdac63b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 31960) UID: 0 2024-04-26T03:39:33.559 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:33.559 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:33.568 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-04-26T03:39:33.601 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-04-26T03:39:33.600+0000 7fe9223c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31954) UID: 0 2024-04-26T03:39:33.670 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:33.671 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:33.679 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-04-26T03:39:33.694 INFO:teuthology.orchestra.run.smithi028.stderr:reweighted osd.5 to 1 (10000) 2024-04-26T03:39:33.702 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:39:33.700+0000 7fb78c544640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31956) UID: 0 2024-04-26T03:39:33.702 INFO:tasks.rados.rados.0.smithi092.stdout:340: done (2 left) 2024-04-26T03:39:33.702 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 50 current snap is 24 2024-04-26T03:39:33.702 INFO:tasks.rados.rados.0.smithi092.stdout:341: seq_num 151 ranges {1507328=131072} 2024-04-26T03:39:33.712 INFO:tasks.rados.rados.0.smithi092.stdout:341: writing smithi09235267-50 from 1507328 to 1638400 tid 1 2024-04-26T03:39:33.712 INFO:tasks.rados.rados.0.smithi092.stdout:342: delete oid 33 current snap is 24 2024-04-26T03:39:33.712 INFO:tasks.rados.rados.0.smithi092.stdout:338: expect (ObjNum 40 snap 0 seq_num 40) 2024-04-26T03:39:33.720 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-04-26T03:39:33.737 INFO:tasks.rados.rados.0.smithi092.stdout:338: done (3 left) 2024-04-26T03:39:33.737 INFO:tasks.rados.rados.0.smithi092.stdout:339: done (2 left) 2024-04-26T03:39:33.738 INFO:tasks.rados.rados.0.smithi092.stdout:342: done (1 left) 2024-04-26T03:39:33.738 INFO:tasks.rados.rados.0.smithi092.stdout:343: snap_create 2024-04-26T03:39:33.738 INFO:tasks.rados.rados.0.smithi092.stdout:341: finishing write tid 1 to smithi09235267-50 2024-04-26T03:39:33.738 INFO:tasks.rados.rados.0.smithi092.stdout:341: finishing write tid 2 to smithi09235267-50 2024-04-26T03:39:33.738 INFO:tasks.rados.rados.0.smithi092.stdout:341: finishing write tid 3 to smithi09235267-50 2024-04-26T03:39:33.738 INFO:tasks.rados.rados.0.smithi092.stdout:341: oid 50 updating version 0 to 121 2024-04-26T03:39:33.738 INFO:tasks.rados.rados.0.smithi092.stdout:341: oid 50 updating version 121 to 122 2024-04-26T03:39:33.738 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 50 v 122 (ObjNum 151 snap 24 seq_num 151) dirty exists 2024-04-26T03:39:33.738 INFO:tasks.rados.rados.0.smithi092.stdout:341: left oid 50 (ObjNum 151 snap 24 seq_num 151) 2024-04-26T03:39:33.793 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:33.793 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:33.802 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-04-26T03:39:33.803 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-26T03:39:33.800+0000 7f610b3a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31816) UID: 0 2024-04-26T03:39:33.903 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-26T03:39:33.902+0000 7f610b3a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31816) UID: 0 2024-04-26T03:39:33.946 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:33.947 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:33.955 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-04-26T03:39:34.004 INFO:tasks.ceph.osd.4.smithi028.stderr:2024-04-26T03:39:34.003+0000 7fd9511b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31963) UID: 0 2024-04-26T03:39:34.098 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:34.098 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:34.105 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-26T03:39:34.104+0000 7f610b3a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31816) UID: 0 2024-04-26T03:39:34.106 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-04-26T03:39:34.207 INFO:tasks.ceph.osd.10.smithi066.stderr:2024-04-26T03:39:34.206+0000 7f130fcc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 31763) UID: 0 2024-04-26T03:39:34.252 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:34.252 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:34.261 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-26T03:39:34.307 INFO:tasks.ceph.osd.10.smithi066.stderr:2024-04-26T03:39:34.306+0000 7f130fcc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 31763) UID: 0 2024-04-26T03:39:34.368 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:34.369 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:34.378 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-26T03:39:34.409 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-04-26T03:39:34.407+0000 7fdc2d21c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31819) UID: 0 2024-04-26T03:39:34.485 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:34.485 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:34.493 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-26T03:39:34.537 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:39:34.535+0000 7fba0d2ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31961) UID: 0 2024-04-26T03:39:34.601 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:34.601 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:34.609 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-26T03:39:34.610 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-26T03:39:34.609+0000 7f610b3a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31816) UID: 0 2024-04-26T03:39:34.689 INFO:teuthology.orchestra.run.smithi028.stderr:reweighted osd.6 to 1 (10000) 2024-04-26T03:39:34.692 INFO:tasks.rados.rados.0.smithi092.stdout:341: done (1 left) 2024-04-26T03:39:34.692 INFO:tasks.rados.rados.0.smithi092.stdout:343: done (0 left) 2024-04-26T03:39:34.692 INFO:tasks.rados.rados.0.smithi092.stdout:344: setattr oid 1 current snap is 25 2024-04-26T03:39:34.693 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 7 current snap is 25 2024-04-26T03:39:34.693 INFO:tasks.rados.rados.0.smithi092.stdout:345: seq_num 153 ranges {1114112=163840} 2024-04-26T03:39:34.701 INFO:tasks.rados.rados.0.smithi092.stdout:345: writing smithi09235267-7 from 1114112 to 1277952 tid 1 2024-04-26T03:39:34.701 INFO:tasks.rados.rados.0.smithi092.stdout:346: delete oid 35 current snap is 25 2024-04-26T03:39:34.705 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-04-26T03:39:34.707 INFO:tasks.rados.rados.0.smithi092.stdout:345: finishing write tid 1 to smithi09235267-7 2024-04-26T03:39:34.707 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 1 v 63 (ObjNum 449 snap 0 seq_num 3583475296) dirty exists 2024-04-26T03:39:34.708 INFO:tasks.rados.rados.0.smithi092.stdout:345: finishing write tid 2 to smithi09235267-7 2024-04-26T03:39:34.710 INFO:tasks.rados.rados.0.smithi092.stdout:345: finishing write tid 3 to smithi09235267-7 2024-04-26T03:39:34.710 INFO:tasks.rados.rados.0.smithi092.stdout:345: oid 7 updating version 0 to 128 2024-04-26T03:39:34.710 INFO:tasks.rados.rados.0.smithi092.stdout:345: oid 7 updating version 128 to 129 2024-04-26T03:39:34.710 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 7 v 129 (ObjNum 153 snap 25 seq_num 153) dirty exists 2024-04-26T03:39:34.710 INFO:tasks.rados.rados.0.smithi092.stdout:345: left oid 7 (ObjNum 153 snap 25 seq_num 153) 2024-04-26T03:39:34.711 INFO:tasks.ceph.osd.15.smithi092.stderr:2024-04-26T03:39:34.709+0000 7f6d12dec640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 31824) UID: 0 2024-04-26T03:39:34.712 INFO:tasks.rados.rados.0.smithi092.stdout:344: done (2 left) 2024-04-26T03:39:34.712 INFO:tasks.rados.rados.0.smithi092.stdout:345: done (1 left) 2024-04-26T03:39:34.712 INFO:tasks.rados.rados.0.smithi092.stdout:346: done (0 left) 2024-04-26T03:39:34.712 INFO:tasks.rados.rados.0.smithi092.stdout:347: read oid 10 snap -1 2024-04-26T03:39:34.712 INFO:tasks.rados.rados.0.smithi092.stdout:347: expect (ObjNum 90 snap 7 seq_num 90) 2024-04-26T03:39:34.712 INFO:tasks.rados.rados.0.smithi092.stdout:348: snap_create 2024-04-26T03:39:34.715 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-26T03:39:34.722 INFO:tasks.rados.rados.0.smithi092.stdout:347: expect (ObjNum 90 snap 7 seq_num 90) 2024-04-26T03:39:34.763 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:34.763 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:34.773 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-26T03:39:34.814 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:39:34.812+0000 7fba0d2ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31961) UID: 0 2024-04-26T03:39:34.913 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:39:34.912+0000 7fb78c544640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31956) UID: 0 2024-04-26T03:39:34.920 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-26T03:39:34.925 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:34.925 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:34.935 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-26T03:39:35.014 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:39:35.013+0000 7f60d3c4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31757) UID: 0 2024-04-26T03:39:35.067 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0: {} 2024-04-26T03:39:35.068 INFO:teuthology.orchestra.run.smithi028.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T03:39:35.086 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:35.086 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:35.095 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-04-26T03:39:35.105 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1: {} 2024-04-26T03:39:35.105 INFO:teuthology.orchestra.run.smithi028.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T03:39:35.115 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-04-26T03:39:35.114+0000 7fee562fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31759) UID: 0 2024-04-26T03:39:35.124 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2: {} 2024-04-26T03:39:35.124 INFO:teuthology.orchestra.run.smithi028.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T03:39:35.141 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3: {} 2024-04-26T03:39:35.142 INFO:teuthology.orchestra.run.smithi028.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T03:39:35.159 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4: {} 2024-04-26T03:39:35.159 INFO:teuthology.orchestra.run.smithi028.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T03:39:35.177 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5: {} 2024-04-26T03:39:35.177 INFO:teuthology.orchestra.run.smithi028.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T03:39:35.196 INFO:teuthology.orchestra.run.smithi028.stdout:osd.6: {} 2024-04-26T03:39:35.196 INFO:teuthology.orchestra.run.smithi028.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T03:39:35.204 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:35.204 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:35.213 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-04-26T03:39:35.214 INFO:teuthology.orchestra.run.smithi028.stdout:osd.7: {} 2024-04-26T03:39:35.214 INFO:teuthology.orchestra.run.smithi028.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T03:39:35.215 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-26T03:39:35.214+0000 7f610b3a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31816) UID: 0 2024-04-26T03:39:35.231 INFO:teuthology.orchestra.run.smithi028.stdout:osd.8: {} 2024-04-26T03:39:35.231 INFO:teuthology.orchestra.run.smithi028.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T03:39:35.249 INFO:teuthology.orchestra.run.smithi028.stdout:osd.9: {} 2024-04-26T03:39:35.250 INFO:teuthology.orchestra.run.smithi028.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T03:39:35.268 INFO:teuthology.orchestra.run.smithi028.stdout:osd.10: {} 2024-04-26T03:39:35.268 INFO:teuthology.orchestra.run.smithi028.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T03:39:35.285 INFO:teuthology.orchestra.run.smithi028.stdout:osd.11: {} 2024-04-26T03:39:35.285 INFO:teuthology.orchestra.run.smithi028.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T03:39:35.303 INFO:teuthology.orchestra.run.smithi028.stdout:osd.12: {} 2024-04-26T03:39:35.303 INFO:teuthology.orchestra.run.smithi028.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T03:39:35.316 INFO:tasks.ceph.osd.8.smithi028.stderr:2024-04-26T03:39:35.315+0000 7f5bb75d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 31968) UID: 0 2024-04-26T03:39:35.321 INFO:teuthology.orchestra.run.smithi028.stdout:osd.13: {} 2024-04-26T03:39:35.322 INFO:teuthology.orchestra.run.smithi028.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T03:39:35.322 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:39:35.322 INFO:teuthology.orchestra.run.smithi028.stdout: "ops": [], 2024-04-26T03:39:35.322 INFO:teuthology.orchestra.run.smithi028.stdout: "complaint_time": 30, 2024-04-26T03:39:35.322 INFO:teuthology.orchestra.run.smithi028.stdout: "num_blocked_ops": 0 2024-04-26T03:39:35.322 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:39:35.331 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-04-26T03:39:35.340 INFO:teuthology.orchestra.run.smithi028.stdout:osd.14: {} 2024-04-26T03:39:35.340 INFO:teuthology.orchestra.run.smithi028.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T03:39:35.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:39:35.339+0000 7f505d418640 0 -- 172.21.15.28:0/244565862 >> [v2:172.21.15.92:6826/3982634323,v1:172.21.15.92:6827/3982634323] conn(0x7f5058222bf0 msgr2=0x7f5058225020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-26T03:39:35.358 INFO:teuthology.orchestra.run.smithi028.stdout:osd.15: {} 2024-04-26T03:39:35.358 INFO:teuthology.orchestra.run.smithi028.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T03:39:35.417 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:39:35.415+0000 7fba0d2ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31961) UID: 0 2024-04-26T03:39:35.450 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:39:35.450 INFO:teuthology.orchestra.run.smithi028.stdout: "size": 20, 2024-04-26T03:39:35.450 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 600, 2024-04-26T03:39:35.450 INFO:teuthology.orchestra.run.smithi028.stdout: "ops": [ 2024-04-26T03:39:35.450 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.450 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:34 3.4s0 3:2ee2abb1:::smithi09235267-12:head [append 0~196608,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:35.450 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.494587+0000", 2024-04-26T03:39:35.450 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 43.952694026000003, 2024-04-26T03:39:35.450 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.014012205, 2024-04-26T03:39:35.450 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:35.450 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:35.451 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:35.451 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:35.451 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:35.451 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:35.451 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 34 2024-04-26T03:39:35.451 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.451 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:35.451 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.451 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:35.451 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.494587+0000", 2024-04-26T03:39:35.451 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:35.451 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.451 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.451 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:35.452 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.494587+0000", 2024-04-26T03:39:35.452 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:35.452 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.452 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.452 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:35.452 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.494589+0000", 2024-04-26T03:39:35.452 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.3530000000000002e-06 2024-04-26T03:39:35.452 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.452 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.452 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:35.452 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.494944+0000", 2024-04-26T03:39:35.452 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00035535200000000002 2024-04-26T03:39:35.452 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.452 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.453 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:35.453 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.494963+0000", 2024-04-26T03:39:35.453 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.8745000000000001e-05 2024-04-26T03:39:35.453 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.453 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.453 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:35.453 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.494972+0000", 2024-04-26T03:39:35.453 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.0629999999999995e-06 2024-04-26T03:39:35.453 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.453 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.453 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:35.453 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495054+0000", 2024-04-26T03:39:35.453 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.1810000000000004e-05 2024-04-26T03:39:35.453 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.453 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.454 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:35.454 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495184+0000", 2024-04-26T03:39:35.454 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00012965399999999999 2024-04-26T03:39:35.454 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.454 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.454 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:35.454 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495943+0000", 2024-04-26T03:39:35.454 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00075938199999999996 2024-04-26T03:39:35.454 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.454 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.454 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:35.454 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.500314+0000", 2024-04-26T03:39:35.454 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0043711669999999996 2024-04-26T03:39:35.455 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.455 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.455 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:35.455 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.508524+0000", 2024-04-26T03:39:35.455 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0082094520000000008 2024-04-26T03:39:35.455 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.455 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.455 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:35.455 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.508599+0000", 2024-04-26T03:39:35.455 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.5227000000000003e-05 2024-04-26T03:39:35.455 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:35.455 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:35.455 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:35.455 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.456 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.456 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:36 3.4s0 3:2ee2abb1:::smithi09235267-12:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:35.456 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.495035+0000", 2024-04-26T03:39:35.456 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 43.952245202, 2024-04-26T03:39:35.456 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.018906236, 2024-04-26T03:39:35.456 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:35.456 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:35.456 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "started", 2024-04-26T03:39:35.456 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:35.456 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:35.456 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:35.456 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 36 2024-04-26T03:39:35.456 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.456 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:35.456 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.457 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:35.457 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495035+0000", 2024-04-26T03:39:35.457 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:35.457 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.457 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.457 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:35.457 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495035+0000", 2024-04-26T03:39:35.457 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:35.457 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.457 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.457 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:35.457 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495036+0000", 2024-04-26T03:39:35.457 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.5199999999999997e-07 2024-04-26T03:39:35.457 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.458 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.458 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:35.458 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495039+0000", 2024-04-26T03:39:35.458 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.5770000000000001e-06 2024-04-26T03:39:35.458 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.458 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.458 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:35.458 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495045+0000", 2024-04-26T03:39:35.458 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.8199999999999999e-06 2024-04-26T03:39:35.458 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.458 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.458 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:35.458 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495050+0000", 2024-04-26T03:39:35.458 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.8520000000000003e-06 2024-04-26T03:39:35.459 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.459 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.459 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:35.459 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.500349+0000", 2024-04-26T03:39:35.459 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0052990069999999997 2024-04-26T03:39:35.459 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.459 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.459 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:35.459 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.500436+0000", 2024-04-26T03:39:35.459 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.6327000000000002e-05 2024-04-26T03:39:35.459 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.459 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.459 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:35.459 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.508818+0000", 2024-04-26T03:39:35.459 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0083819900000000006 2024-04-26T03:39:35.460 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.460 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.460 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:35.460 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.508897+0000", 2024-04-26T03:39:35.460 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.9580999999999996e-05 2024-04-26T03:39:35.460 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.460 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.460 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:35.460 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.513942+0000", 2024-04-26T03:39:35.460 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0050445300000000002 2024-04-26T03:39:35.461 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:35.461 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:35.461 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:35.461 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.461 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.461 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:101 3.4s0 3:2869fefd:::smithi09235267-34:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:35.461 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.559280+0000", 2024-04-26T03:39:35.461 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 43.888000546999997, 2024-04-26T03:39:35.461 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.015441082, 2024-04-26T03:39:35.461 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:35.461 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:35.461 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:35.461 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:35.461 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:35.461 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:35.462 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 101 2024-04-26T03:39:35.462 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.462 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:35.462 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.462 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:35.462 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.559280+0000", 2024-04-26T03:39:35.462 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:35.462 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.462 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.462 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:35.462 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.559280+0000", 2024-04-26T03:39:35.462 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:35.462 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.463 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.463 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:35.463 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.559281+0000", 2024-04-26T03:39:35.463 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.4600000000000004e-07 2024-04-26T03:39:35.463 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.463 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.463 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:35.463 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.559285+0000", 2024-04-26T03:39:35.463 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.5129999999999998e-06 2024-04-26T03:39:35.463 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.463 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.463 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:35.463 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.559294+0000", 2024-04-26T03:39:35.463 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.0059999999999998e-06 2024-04-26T03:39:35.463 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.464 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.464 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:35.464 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.559303+0000", 2024-04-26T03:39:35.464 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.1999999999999994e-06 2024-04-26T03:39:35.464 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.464 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.464 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:35.464 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.564299+0000", 2024-04-26T03:39:35.464 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0049963150000000003 2024-04-26T03:39:35.464 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.464 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.464 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:35.464 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.564369+0000", 2024-04-26T03:39:35.464 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.0238000000000001e-05 2024-04-26T03:39:35.465 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.465 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.465 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:35.465 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.564677+0000", 2024-04-26T03:39:35.465 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00030746099999999998 2024-04-26T03:39:35.465 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.465 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.465 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:35.465 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.565235+0000", 2024-04-26T03:39:35.465 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00055854500000000003 2024-04-26T03:39:35.465 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.465 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.465 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:35.465 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.574673+0000", 2024-04-26T03:39:35.466 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0094382830000000004 2024-04-26T03:39:35.466 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.466 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.466 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:35.466 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.574721+0000", 2024-04-26T03:39:35.466 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.7775000000000003e-05 2024-04-26T03:39:35.466 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:35.466 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:35.466 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:35.466 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.466 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.466 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:102 3.4s0 3:2869fefd:::smithi09235267-34:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:35.466 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.559325+0000", 2024-04-26T03:39:35.466 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 43.887955867999999, 2024-04-26T03:39:35.467 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.017964909000000001, 2024-04-26T03:39:35.467 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:35.467 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:35.467 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "started", 2024-04-26T03:39:35.467 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:35.467 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:35.467 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:35.467 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 102 2024-04-26T03:39:35.467 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.467 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:35.467 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.467 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:35.467 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.559325+0000", 2024-04-26T03:39:35.468 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:35.468 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.468 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.468 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:35.468 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.559325+0000", 2024-04-26T03:39:35.468 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:35.468 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.468 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.468 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:35.468 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.559325+0000", 2024-04-26T03:39:35.468 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.2900000000000003e-07 2024-04-26T03:39:35.468 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.468 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.468 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:35.468 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.559330+0000", 2024-04-26T03:39:35.469 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.7029999999999997e-06 2024-04-26T03:39:35.469 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.469 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.469 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:35.469 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.559339+0000", 2024-04-26T03:39:35.469 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.8969999999999997e-06 2024-04-26T03:39:35.469 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.469 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.469 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:35.469 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.559346+0000", 2024-04-26T03:39:35.469 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.8560000000000001e-06 2024-04-26T03:39:35.469 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.469 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.469 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:35.470 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.564881+0000", 2024-04-26T03:39:35.470 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0055355830000000002 2024-04-26T03:39:35.470 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.470 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.470 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:35.470 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.564965+0000", 2024-04-26T03:39:35.470 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.3248e-05 2024-04-26T03:39:35.470 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.470 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.470 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:35.470 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.574764+0000", 2024-04-26T03:39:35.470 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0097988769999999992 2024-04-26T03:39:35.470 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.470 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.471 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:35.471 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.574811+0000", 2024-04-26T03:39:35.471 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.7466999999999998e-05 2024-04-26T03:39:35.471 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.471 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.471 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:35.471 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.577290+0000", 2024-04-26T03:39:35.471 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0024786489999999999 2024-04-26T03:39:35.471 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:35.471 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:35.471 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:35.471 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.471 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.471 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:262 3.4s0 3:2d740be2:::smithi09235267-28:head [append 0~720896] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T03:39:35.471 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:56.518030+0000", 2024-04-26T03:39:35.472 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 38.929250443000001, 2024-04-26T03:39:35.472 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.018690736999999999, 2024-04-26T03:39:35.472 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:35.472 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:35.472 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:35.472 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:35.472 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:35.472 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:35.472 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 262 2024-04-26T03:39:35.472 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.472 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:35.473 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.473 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:35.473 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.518030+0000", 2024-04-26T03:39:35.473 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:35.473 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.473 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.473 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:35.473 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.518030+0000", 2024-04-26T03:39:35.473 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:35.473 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.473 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.473 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:35.473 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.518039+0000", 2024-04-26T03:39:35.473 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.3999999999999992e-06 2024-04-26T03:39:35.473 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.474 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.474 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:35.474 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.519077+0000", 2024-04-26T03:39:35.474 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.001038593 2024-04-26T03:39:35.474 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.474 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.474 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:35.474 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.519089+0000", 2024-04-26T03:39:35.474 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.2092e-05 2024-04-26T03:39:35.474 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.474 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.474 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:35.474 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.519097+0000", 2024-04-26T03:39:35.474 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.7680000000000008e-06 2024-04-26T03:39:35.475 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.475 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.475 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:35.475 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.519162+0000", 2024-04-26T03:39:35.475 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.4517999999999995e-05 2024-04-26T03:39:35.475 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.475 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.475 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:35.475 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.519303+0000", 2024-04-26T03:39:35.475 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000141602 2024-04-26T03:39:35.476 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.476 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.476 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:35.476 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.520994+0000", 2024-04-26T03:39:35.476 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0016909780000000001 2024-04-26T03:39:35.476 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.476 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.476 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:35.476 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.531345+0000", 2024-04-26T03:39:35.476 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.010350471999999999 2024-04-26T03:39:35.476 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.476 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.477 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:35.477 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.536659+0000", 2024-04-26T03:39:35.477 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0053146649999999997 2024-04-26T03:39:35.477 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.477 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.477 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:35.477 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.536721+0000", 2024-04-26T03:39:35.477 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.1649000000000006e-05 2024-04-26T03:39:35.477 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:35.477 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:35.477 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:35.477 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.477 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.478 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:263 3.4s0 3:2d740be2:::smithi09235267-28:head [append 0~376832] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T03:39:35.478 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:56.520937+0000", 2024-04-26T03:39:35.478 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 38.926343926000001, 2024-04-26T03:39:35.478 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.016100099999999999, 2024-04-26T03:39:35.478 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:35.478 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:35.478 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:35.478 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:35.478 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:35.478 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:35.478 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 263 2024-04-26T03:39:35.478 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.478 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:35.478 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.479 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:35.479 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.520937+0000", 2024-04-26T03:39:35.479 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:35.479 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.479 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.479 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:35.479 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.520937+0000", 2024-04-26T03:39:35.479 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:35.479 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.479 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.480 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:35.480 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.520939+0000", 2024-04-26T03:39:35.480 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.0669999999999999e-06 2024-04-26T03:39:35.480 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.480 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.480 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:35.480 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.521258+0000", 2024-04-26T03:39:35.480 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00031960300000000001 2024-04-26T03:39:35.480 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.480 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.480 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:35.480 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.521281+0000", 2024-04-26T03:39:35.480 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.2600999999999999e-05 2024-04-26T03:39:35.480 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.480 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.481 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:35.481 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.521290+0000", 2024-04-26T03:39:35.481 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.1169999999999993e-06 2024-04-26T03:39:35.481 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.481 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.481 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:35.481 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.528401+0000", 2024-04-26T03:39:35.481 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0071106499999999996 2024-04-26T03:39:35.481 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.481 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.481 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:35.481 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.528472+0000", 2024-04-26T03:39:35.481 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.1081999999999995e-05 2024-04-26T03:39:35.481 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.482 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.482 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:35.482 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.529015+0000", 2024-04-26T03:39:35.482 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00054347300000000005 2024-04-26T03:39:35.482 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.482 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.482 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:35.482 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.531961+0000", 2024-04-26T03:39:35.482 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.002945526 2024-04-26T03:39:35.482 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.482 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.482 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:35.482 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.536998+0000", 2024-04-26T03:39:35.482 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0050376960000000004 2024-04-26T03:39:35.483 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.483 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.483 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:35.483 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.537037+0000", 2024-04-26T03:39:35.483 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.8284999999999997e-05 2024-04-26T03:39:35.483 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:35.483 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:35.483 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:35.483 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.483 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.483 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:264 3.4s0 3:2d740be2:::smithi09235267-28:head [setxattr _header (56) in=7b] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T03:39:35.483 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:56.521315+0000", 2024-04-26T03:39:35.483 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 38.925965325999996, 2024-04-26T03:39:35.483 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.015842726000000001, 2024-04-26T03:39:35.483 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:35.484 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:35.484 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:35.484 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:35.484 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:35.484 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:35.484 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 264 2024-04-26T03:39:35.484 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.484 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:35.484 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.484 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:35.484 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.521315+0000", 2024-04-26T03:39:35.484 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:35.484 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.485 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.485 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:35.485 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.521315+0000", 2024-04-26T03:39:35.485 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:35.485 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.485 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.485 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:35.485 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.521316+0000", 2024-04-26T03:39:35.485 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.2799999999999995e-07 2024-04-26T03:39:35.485 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.485 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.485 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:35.485 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.521321+0000", 2024-04-26T03:39:35.485 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.9230000000000001e-06 2024-04-26T03:39:35.485 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.486 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.486 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:35.486 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.521330+0000", 2024-04-26T03:39:35.486 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.1859999999999998e-06 2024-04-26T03:39:35.486 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.486 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.486 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:35.486 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.521336+0000", 2024-04-26T03:39:35.486 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.8499999999999999e-06 2024-04-26T03:39:35.486 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.486 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.486 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:35.486 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.531370+0000", 2024-04-26T03:39:35.486 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.010033767000000001 2024-04-26T03:39:35.487 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.487 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.487 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:35.487 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.531431+0000", 2024-04-26T03:39:35.487 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.1359000000000007e-05 2024-04-26T03:39:35.487 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.487 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.487 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:35.487 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.531718+0000", 2024-04-26T03:39:35.487 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00028652000000000001 2024-04-26T03:39:35.487 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.487 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.487 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:35.487 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.532128+0000", 2024-04-26T03:39:35.488 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00041016700000000002 2024-04-26T03:39:35.488 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.488 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.488 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:35.488 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.537102+0000", 2024-04-26T03:39:35.488 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0049742520000000002 2024-04-26T03:39:35.488 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.488 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.488 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:35.488 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.537158+0000", 2024-04-26T03:39:35.488 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.5874000000000001e-05 2024-04-26T03:39:35.488 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:35.488 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:35.488 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:35.488 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.489 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.489 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:265 3.4s0 3:2d740be2:::smithi09235267-28:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e40)", 2024-04-26T03:39:35.489 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:56.521345+0000", 2024-04-26T03:39:35.489 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 38.925936016999998, 2024-04-26T03:39:35.489 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.017065469999999999, 2024-04-26T03:39:35.489 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:35.489 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:35.489 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "started", 2024-04-26T03:39:35.489 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:35.489 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:35.489 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:35.489 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 265 2024-04-26T03:39:35.489 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.489 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:35.490 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.490 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:35.490 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.521345+0000", 2024-04-26T03:39:35.490 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:35.490 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.490 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.490 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:35.490 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.521345+0000", 2024-04-26T03:39:35.490 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:35.490 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.490 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.490 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:35.490 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.521345+0000", 2024-04-26T03:39:35.490 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.1799999999999995e-07 2024-04-26T03:39:35.491 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.491 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.491 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:35.491 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.521347+0000", 2024-04-26T03:39:35.491 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.2199999999999999e-06 2024-04-26T03:39:35.491 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.491 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.491 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:35.491 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.521355+0000", 2024-04-26T03:39:35.491 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.9140000000000008e-06 2024-04-26T03:39:35.491 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.491 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.491 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:35.491 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.521360+0000", 2024-04-26T03:39:35.491 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.9989999999999999e-06 2024-04-26T03:39:35.492 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.492 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.492 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:35.492 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.531908+0000", 2024-04-26T03:39:35.492 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.010547707 2024-04-26T03:39:35.492 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.492 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.492 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:35.492 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.531953+0000", 2024-04-26T03:39:35.492 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.5543000000000001e-05 2024-04-26T03:39:35.492 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.492 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.492 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:35.492 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.537458+0000", 2024-04-26T03:39:35.493 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0055043699999999997 2024-04-26T03:39:35.493 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.493 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.493 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:35.493 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.537538+0000", 2024-04-26T03:39:35.493 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.9798999999999999e-05 2024-04-26T03:39:35.493 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.493 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.493 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:35.493 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:56.538410+0000", 2024-04-26T03:39:35.493 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00087239999999999996 2024-04-26T03:39:35.493 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:35.493 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:35.493 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:35.494 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.494 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.494 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:438 3.14s0 3:2d740be2:::smithi09235267-28:head [append 0~704512] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-04-26T03:39:35.494 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:04.723796+0000", 2024-04-26T03:39:35.494 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 30.723484204000002, 2024-04-26T03:39:35.494 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.016243738000000001, 2024-04-26T03:39:35.494 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:35.494 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:35.494 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:35.494 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:35.494 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:35.494 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:35.494 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 438 2024-04-26T03:39:35.494 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.495 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:35.495 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.495 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:35.495 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.723796+0000", 2024-04-26T03:39:35.495 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:35.495 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.495 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.495 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:35.495 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.723796+0000", 2024-04-26T03:39:35.495 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:35.495 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.495 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.495 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:35.495 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.723798+0000", 2024-04-26T03:39:35.496 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.4580000000000001e-06 2024-04-26T03:39:35.496 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.496 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.496 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:35.496 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.724544+0000", 2024-04-26T03:39:35.496 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00074663500000000005 2024-04-26T03:39:35.496 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.496 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.496 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:35.496 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.724557+0000", 2024-04-26T03:39:35.496 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.2758999999999999e-05 2024-04-26T03:39:35.496 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.496 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.496 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:35.496 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.724563+0000", 2024-04-26T03:39:35.497 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.2299999999999996e-06 2024-04-26T03:39:35.497 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.497 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.497 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:35.497 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.724629+0000", 2024-04-26T03:39:35.497 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.5907000000000004e-05 2024-04-26T03:39:35.497 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.497 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.497 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:35.497 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.724806+0000", 2024-04-26T03:39:35.497 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00017629200000000001 2024-04-26T03:39:35.497 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.497 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.497 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:35.497 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.726160+0000", 2024-04-26T03:39:35.498 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0013545289999999999 2024-04-26T03:39:35.498 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.498 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.498 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:35.498 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.739782+0000", 2024-04-26T03:39:35.498 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.013621416000000001 2024-04-26T03:39:35.498 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.498 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.498 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:35.498 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.739954+0000", 2024-04-26T03:39:35.498 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000172483 2024-04-26T03:39:35.498 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.498 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.498 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:35.499 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.740040+0000", 2024-04-26T03:39:35.499 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.6028999999999999e-05 2024-04-26T03:39:35.499 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:35.499 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:35.499 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:35.499 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.499 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.499 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:439 3.14s0 3:2d740be2:::smithi09235267-28:head [append 0~688128] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-04-26T03:39:35.499 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:04.727745+0000", 2024-04-26T03:39:35.499 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 30.71953572, 2024-04-26T03:39:35.499 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.015468773999999999, 2024-04-26T03:39:35.499 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:35.499 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:35.499 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:35.499 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:35.500 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:35.500 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:35.500 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 439 2024-04-26T03:39:35.500 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.500 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:35.500 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.500 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:35.500 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.727745+0000", 2024-04-26T03:39:35.500 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:35.500 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.500 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.500 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:35.500 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.727745+0000", 2024-04-26T03:39:35.500 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:35.501 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.501 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.501 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:35.501 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.727746+0000", 2024-04-26T03:39:35.501 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.5239999999999999e-06 2024-04-26T03:39:35.501 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.501 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.501 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:35.501 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.728329+0000", 2024-04-26T03:39:35.501 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00058246000000000003 2024-04-26T03:39:35.501 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.501 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.501 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:35.501 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.728340+0000", 2024-04-26T03:39:35.501 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1599e-05 2024-04-26T03:39:35.502 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.502 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.502 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:35.502 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.728346+0000", 2024-04-26T03:39:35.502 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.4310000000000002e-06 2024-04-26T03:39:35.502 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.502 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.502 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:35.502 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.733661+0000", 2024-04-26T03:39:35.502 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0053146790000000001 2024-04-26T03:39:35.502 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.502 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.502 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:35.502 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.733770+0000", 2024-04-26T03:39:35.503 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000109372 2024-04-26T03:39:35.503 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.503 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.503 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:35.503 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.734582+0000", 2024-04-26T03:39:35.503 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00081156700000000002 2024-04-26T03:39:35.503 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.503 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.503 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:35.503 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.742177+0000", 2024-04-26T03:39:35.503 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0075957860000000002 2024-04-26T03:39:35.503 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.503 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.503 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:35.503 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.743148+0000", 2024-04-26T03:39:35.504 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00097102400000000002 2024-04-26T03:39:35.504 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.504 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.504 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:35.504 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.743214+0000", 2024-04-26T03:39:35.504 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.5332000000000004e-05 2024-04-26T03:39:35.504 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:35.504 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:35.504 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:35.504 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.504 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.504 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:440 3.14s0 3:2d740be2:::smithi09235267-28:head [append 0~81920] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-04-26T03:39:35.504 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:04.728360+0000", 2024-04-26T03:39:35.504 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 30.718920763, 2024-04-26T03:39:35.505 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.015584582, 2024-04-26T03:39:35.505 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:35.505 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:35.505 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:35.505 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:35.505 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:35.505 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:35.505 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 440 2024-04-26T03:39:35.505 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.505 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:35.505 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.505 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:35.505 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.728360+0000", 2024-04-26T03:39:35.505 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:35.506 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.506 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.506 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:35.506 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.728360+0000", 2024-04-26T03:39:35.506 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:35.506 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.506 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.506 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:35.506 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.728360+0000", 2024-04-26T03:39:35.506 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.1399999999999997e-07 2024-04-26T03:39:35.506 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.506 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.506 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:35.506 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.728409+0000", 2024-04-26T03:39:35.506 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.8939e-05 2024-04-26T03:39:35.507 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.507 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.507 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:35.507 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.728418+0000", 2024-04-26T03:39:35.507 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.8100000000000004e-06 2024-04-26T03:39:35.507 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.507 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.507 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:35.507 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.728422+0000", 2024-04-26T03:39:35.507 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.9430000000000002e-06 2024-04-26T03:39:35.507 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.507 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.507 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:35.507 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.740077+0000", 2024-04-26T03:39:35.508 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.011654912 2024-04-26T03:39:35.508 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.508 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.508 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:35.508 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.740144+0000", 2024-04-26T03:39:35.508 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.6656999999999996e-05 2024-04-26T03:39:35.508 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.508 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.508 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:35.508 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.740515+0000", 2024-04-26T03:39:35.508 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00037100799999999998 2024-04-26T03:39:35.508 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.508 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.508 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:35.508 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.742297+0000", 2024-04-26T03:39:35.509 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.001782217 2024-04-26T03:39:35.509 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.509 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.509 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:35.509 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.743902+0000", 2024-04-26T03:39:35.509 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.001605569 2024-04-26T03:39:35.509 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.509 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.509 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:35.509 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.743944+0000", 2024-04-26T03:39:35.509 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.2012999999999999e-05 2024-04-26T03:39:35.509 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:35.509 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:35.509 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:35.510 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.510 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.510 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:441 3.14s0 3:2d740be2:::smithi09235267-28:head [setxattr _header (56) in=7b] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-04-26T03:39:35.510 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:04.728430+0000", 2024-04-26T03:39:35.510 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 30.718850679999999, 2024-04-26T03:39:35.510 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.015636728999999999, 2024-04-26T03:39:35.510 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:35.510 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:35.510 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:35.510 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:35.510 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:35.510 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:35.510 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 441 2024-04-26T03:39:35.510 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.511 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:35.511 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.511 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:35.511 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.728430+0000", 2024-04-26T03:39:35.511 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:35.511 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.511 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.511 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:35.511 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.728430+0000", 2024-04-26T03:39:35.511 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:35.511 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.511 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.511 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:35.511 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.728430+0000", 2024-04-26T03:39:35.511 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.7300000000000002e-07 2024-04-26T03:39:35.512 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.512 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.512 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:35.512 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.728433+0000", 2024-04-26T03:39:35.512 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.202e-06 2024-04-26T03:39:35.512 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.512 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.512 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:35.512 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.728438+0000", 2024-04-26T03:39:35.512 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.6280000000000004e-06 2024-04-26T03:39:35.512 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.512 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.512 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:35.512 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.728441+0000", 2024-04-26T03:39:35.513 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.9790000000000002e-06 2024-04-26T03:39:35.513 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.513 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.513 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:35.513 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.741303+0000", 2024-04-26T03:39:35.513 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.01286177 2024-04-26T03:39:35.513 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.513 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.513 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:35.513 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.741438+0000", 2024-04-26T03:39:35.513 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00013489599999999999 2024-04-26T03:39:35.513 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.513 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.513 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:35.513 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.741973+0000", 2024-04-26T03:39:35.514 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00053482699999999998 2024-04-26T03:39:35.514 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.514 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.514 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:35.514 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.742703+0000", 2024-04-26T03:39:35.514 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00073040699999999997 2024-04-26T03:39:35.514 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.514 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.514 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:35.514 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.744014+0000", 2024-04-26T03:39:35.514 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.001310819 2024-04-26T03:39:35.514 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.514 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.514 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:35.515 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.744067+0000", 2024-04-26T03:39:35.515 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.2828000000000003e-05 2024-04-26T03:39:35.515 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:35.515 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:35.515 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:35.515 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.515 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.515 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:442 3.14s0 3:2d740be2:::smithi09235267-28:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e49)", 2024-04-26T03:39:35.515 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:04.728445+0000", 2024-04-26T03:39:35.515 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 30.718835328000001, 2024-04-26T03:39:35.515 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.016969079000000001, 2024-04-26T03:39:35.515 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:35.515 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:35.515 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "started", 2024-04-26T03:39:35.515 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:35.516 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:35.516 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:35.516 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 442 2024-04-26T03:39:35.516 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.516 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:35.516 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.516 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:35.516 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.728445+0000", 2024-04-26T03:39:35.516 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:35.516 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.516 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.516 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:35.516 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.728445+0000", 2024-04-26T03:39:35.516 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:35.516 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.517 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.517 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:35.517 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.728446+0000", 2024-04-26T03:39:35.517 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.9299999999999999e-07 2024-04-26T03:39:35.517 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.517 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.517 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:35.517 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.728447+0000", 2024-04-26T03:39:35.517 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.3710000000000001e-06 2024-04-26T03:39:35.517 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.517 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.517 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:35.517 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.728452+0000", 2024-04-26T03:39:35.517 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.7990000000000001e-06 2024-04-26T03:39:35.517 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.518 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.518 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:35.518 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.728454+0000", 2024-04-26T03:39:35.518 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.717e-06 2024-04-26T03:39:35.518 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.518 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.518 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:35.518 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.742339+0000", 2024-04-26T03:39:35.518 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.013884931 2024-04-26T03:39:35.518 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.518 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.518 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:35.518 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.742421+0000", 2024-04-26T03:39:35.519 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.1386999999999999e-05 2024-04-26T03:39:35.519 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.519 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.519 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:35.519 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.744351+0000", 2024-04-26T03:39:35.519 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.001930153 2024-04-26T03:39:35.519 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.519 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.519 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:35.519 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.744430+0000", 2024-04-26T03:39:35.519 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.9515e-05 2024-04-26T03:39:35.519 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.519 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.519 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:35.520 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.745414+0000", 2024-04-26T03:39:35.520 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00098391300000000011 2024-04-26T03:39:35.520 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:35.520 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:35.520 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:35.520 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.520 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.520 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.10s4 60/48 ECSubWrite(tid=274, reqid=client.4556.0:705, at_version=60'22, trim_to=0'0, roll_forward_to=60'18))", 2024-04-26T03:39:35.520 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:15.966885+0000", 2024-04-26T03:39:35.520 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 19.480395244, 2024-04-26T03:39:35.520 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.014214999000000001, 2024-04-26T03:39:35.520 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:35.520 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:35.520 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:35.520 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:35.521 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.521 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:35.521 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.966885+0000", 2024-04-26T03:39:35.521 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:35.521 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.521 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.521 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:35.521 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.966885+0000", 2024-04-26T03:39:35.521 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:35.521 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.521 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.521 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:35.521 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.966886+0000", 2024-04-26T03:39:35.521 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.9500000000000001e-07 2024-04-26T03:39:35.521 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.522 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.522 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:35.522 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.967191+0000", 2024-04-26T03:39:35.522 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00030450300000000002 2024-04-26T03:39:35.522 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.522 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.522 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:35.522 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.967204+0000", 2024-04-26T03:39:35.522 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.3642000000000001e-05 2024-04-26T03:39:35.522 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.522 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.522 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:35.522 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.967212+0000", 2024-04-26T03:39:35.522 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.9850000000000006e-06 2024-04-26T03:39:35.523 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.523 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.523 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:35.523 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.968412+0000", 2024-04-26T03:39:35.523 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0011998499999999999 2024-04-26T03:39:35.523 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.523 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.523 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:35.523 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.968421+0000", 2024-04-26T03:39:35.523 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.4989999999999994e-06 2024-04-26T03:39:35.523 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.523 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.523 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:35.523 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.981073+0000", 2024-04-26T03:39:35.523 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.012652527 2024-04-26T03:39:35.524 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.524 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.524 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:35.524 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.981100+0000", 2024-04-26T03:39:35.524 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.7197999999999999e-05 2024-04-26T03:39:35.524 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:35.524 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:35.524 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:35.524 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.524 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.524 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:754 3.14s0 3:2ee2abb1:::smithi09235267-12:head [append 0~688128] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:35.524 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:20.729310+0000", 2024-04-26T03:39:35.524 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 14.717970205, 2024-04-26T03:39:35.524 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0196145, 2024-04-26T03:39:35.525 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:35.525 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:35.525 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:35.525 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:35.525 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:35.525 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:35.525 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 754 2024-04-26T03:39:35.525 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.525 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:35.525 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.525 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:35.525 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.729310+0000", 2024-04-26T03:39:35.525 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:35.525 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.525 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.526 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:35.526 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.729310+0000", 2024-04-26T03:39:35.526 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:35.526 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.526 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.526 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:35.526 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.729312+0000", 2024-04-26T03:39:35.526 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.0679999999999999e-06 2024-04-26T03:39:35.526 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.526 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.526 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:35.526 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.730088+0000", 2024-04-26T03:39:35.526 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00077547900000000003 2024-04-26T03:39:35.526 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.526 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.527 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:35.527 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.730109+0000", 2024-04-26T03:39:35.527 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.0707999999999999e-05 2024-04-26T03:39:35.527 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.527 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.527 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:35.527 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.730118+0000", 2024-04-26T03:39:35.527 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.5880000000000004e-06 2024-04-26T03:39:35.527 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.527 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.527 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:35.527 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.730179+0000", 2024-04-26T03:39:35.527 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.0545999999999999e-05 2024-04-26T03:39:35.527 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.528 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.528 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:35.528 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.730260+0000", 2024-04-26T03:39:35.528 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.1324999999999998e-05 2024-04-26T03:39:35.528 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.528 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.528 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:35.528 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.731473+0000", 2024-04-26T03:39:35.528 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.001212917 2024-04-26T03:39:35.528 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.528 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.528 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:35.528 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.748526+0000", 2024-04-26T03:39:35.528 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.017053467999999999 2024-04-26T03:39:35.528 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.529 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.529 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:35.529 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.748849+0000", 2024-04-26T03:39:35.529 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00032208099999999999 2024-04-26T03:39:35.529 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.529 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.529 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:35.529 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.748925+0000", 2024-04-26T03:39:35.529 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.6320000000000001e-05 2024-04-26T03:39:35.529 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:35.529 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:35.529 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:35.529 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.529 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.529 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:755 3.14s0 3:2ee2abb1:::smithi09235267-12:head [append 0~720896] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:35.530 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:20.733275+0000", 2024-04-26T03:39:35.530 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 14.714005364, 2024-04-26T03:39:35.530 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.019559626, 2024-04-26T03:39:35.530 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:35.530 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:35.530 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:35.530 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:35.530 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:35.530 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:35.530 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 755 2024-04-26T03:39:35.530 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.530 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:35.530 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.530 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:35.531 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.733275+0000", 2024-04-26T03:39:35.531 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:35.531 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.531 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.531 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:35.531 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.733275+0000", 2024-04-26T03:39:35.531 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:35.531 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.531 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.531 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:35.531 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.733277+0000", 2024-04-26T03:39:35.531 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.283e-06 2024-04-26T03:39:35.531 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.531 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.531 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:35.532 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.733948+0000", 2024-04-26T03:39:35.532 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000671348 2024-04-26T03:39:35.532 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.532 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.532 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:35.532 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.733962+0000", 2024-04-26T03:39:35.532 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.4144999999999999e-05 2024-04-26T03:39:35.532 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.532 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.532 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:35.532 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.733971+0000", 2024-04-26T03:39:35.532 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.7849999999999997e-06 2024-04-26T03:39:35.532 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.533 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.533 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:35.533 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.737490+0000", 2024-04-26T03:39:35.533 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0035195790000000001 2024-04-26T03:39:35.533 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.533 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.533 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:35.533 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.737615+0000", 2024-04-26T03:39:35.533 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00012457899999999999 2024-04-26T03:39:35.533 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.533 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.533 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:35.533 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.738785+0000", 2024-04-26T03:39:35.533 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.001170405 2024-04-26T03:39:35.534 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.534 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.534 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:35.534 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.748528+0000", 2024-04-26T03:39:35.534 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0097424030000000002 2024-04-26T03:39:35.534 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.534 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.534 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:35.534 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.752780+0000", 2024-04-26T03:39:35.534 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0042521879999999996 2024-04-26T03:39:35.534 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.534 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.534 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:35.534 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.752835+0000", 2024-04-26T03:39:35.535 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.4911000000000003e-05 2024-04-26T03:39:35.535 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:35.535 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:35.535 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:35.535 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.535 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.535 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:756 3.14s0 3:2ee2abb1:::smithi09235267-12:head [append 0~475136] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:35.535 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:20.735986+0000", 2024-04-26T03:39:35.535 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 14.711294315, 2024-04-26T03:39:35.535 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.020284473000000001, 2024-04-26T03:39:35.535 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:35.535 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:35.535 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:35.535 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:35.536 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:35.536 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:35.536 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 756 2024-04-26T03:39:35.536 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.536 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:35.536 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.536 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:35.536 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.735986+0000", 2024-04-26T03:39:35.536 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:35.536 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.536 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.536 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:35.536 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.735986+0000", 2024-04-26T03:39:35.536 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:35.536 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.537 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.537 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:35.537 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.735988+0000", 2024-04-26T03:39:35.537 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.773e-06 2024-04-26T03:39:35.537 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.537 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.537 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:35.537 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.736400+0000", 2024-04-26T03:39:35.537 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000412311 2024-04-26T03:39:35.537 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.537 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.537 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:35.537 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.736413+0000", 2024-04-26T03:39:35.537 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.2554e-05 2024-04-26T03:39:35.538 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.538 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.538 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:35.538 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.736419+0000", 2024-04-26T03:39:35.538 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.5429999999999999e-06 2024-04-26T03:39:35.538 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.538 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.538 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:35.538 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.744075+0000", 2024-04-26T03:39:35.538 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0076556050000000002 2024-04-26T03:39:35.538 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.538 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.538 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:35.538 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.744181+0000", 2024-04-26T03:39:35.539 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000105784 2024-04-26T03:39:35.539 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.539 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.539 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:35.539 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.745066+0000", 2024-04-26T03:39:35.539 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00088487699999999995 2024-04-26T03:39:35.539 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.539 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.539 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:35.539 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.749042+0000", 2024-04-26T03:39:35.539 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.003976757 2024-04-26T03:39:35.539 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.539 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.539 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:35.540 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.756213+0000", 2024-04-26T03:39:35.540 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0071703009999999996 2024-04-26T03:39:35.540 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.540 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.540 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:35.540 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.756271+0000", 2024-04-26T03:39:35.540 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.7967999999999998e-05 2024-04-26T03:39:35.540 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:35.540 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:35.540 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:35.540 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.540 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.540 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:757 3.14s0 3:2ee2abb1:::smithi09235267-12:head [append 0~360448] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:35.540 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:20.737966+0000", 2024-04-26T03:39:35.541 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 14.709314897, 2024-04-26T03:39:35.541 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.018472444000000001, 2024-04-26T03:39:35.541 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:35.541 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:35.541 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:35.541 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:35.541 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:35.541 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:35.541 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 757 2024-04-26T03:39:35.541 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.541 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:35.541 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.541 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:35.542 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.737966+0000", 2024-04-26T03:39:35.542 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:35.542 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.542 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.542 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:35.542 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.737966+0000", 2024-04-26T03:39:35.542 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:35.542 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.542 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.542 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:35.542 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.737967+0000", 2024-04-26T03:39:35.542 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.031e-06 2024-04-26T03:39:35.542 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.542 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.543 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:35.543 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.738361+0000", 2024-04-26T03:39:35.543 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000394163 2024-04-26T03:39:35.543 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.543 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.543 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:35.543 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.738372+0000", 2024-04-26T03:39:35.543 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1545999999999999e-05 2024-04-26T03:39:35.543 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.543 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.543 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:35.543 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.738380+0000", 2024-04-26T03:39:35.543 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.6029999999999999e-06 2024-04-26T03:39:35.543 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.544 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.544 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:35.544 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.749098+0000", 2024-04-26T03:39:35.544 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.010717585 2024-04-26T03:39:35.544 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.544 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.544 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:35.544 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.749165+0000", 2024-04-26T03:39:35.544 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.7364000000000001e-05 2024-04-26T03:39:35.544 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.544 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.544 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:35.544 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.749659+0000", 2024-04-26T03:39:35.544 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00049434999999999995 2024-04-26T03:39:35.544 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.545 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.545 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:35.545 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.753323+0000", 2024-04-26T03:39:35.545 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0036639889999999999 2024-04-26T03:39:35.545 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.545 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.545 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:35.545 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.756392+0000", 2024-04-26T03:39:35.545 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0030688410000000001 2024-04-26T03:39:35.545 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.545 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.545 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:35.545 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.756438+0000", 2024-04-26T03:39:35.545 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.5972e-05 2024-04-26T03:39:35.546 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:35.546 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:35.546 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:35.546 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.546 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.546 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:758 3.14s0 3:2ee2abb1:::smithi09235267-12:head [setxattr _header (57) in=7b] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:35.546 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:20.738397+0000", 2024-04-26T03:39:35.546 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 14.708884094, 2024-04-26T03:39:35.546 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.018504256, 2024-04-26T03:39:35.546 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:35.546 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:35.546 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:35.546 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:35.547 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:35.547 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:35.547 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 758 2024-04-26T03:39:35.547 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.547 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:35.547 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.547 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:35.547 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.738397+0000", 2024-04-26T03:39:35.547 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:35.547 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.547 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.547 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:35.547 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.738397+0000", 2024-04-26T03:39:35.547 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:35.548 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.548 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.548 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:35.548 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.738397+0000", 2024-04-26T03:39:35.548 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.2900000000000002e-07 2024-04-26T03:39:35.548 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.548 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.548 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:35.548 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.738402+0000", 2024-04-26T03:39:35.548 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.9150000000000004e-06 2024-04-26T03:39:35.548 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.548 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.548 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:35.548 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.738409+0000", 2024-04-26T03:39:35.549 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.8419999999999999e-06 2024-04-26T03:39:35.549 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.549 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.549 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:35.549 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.738414+0000", 2024-04-26T03:39:35.549 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.1800000000000004e-06 2024-04-26T03:39:35.549 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.549 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.549 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:35.549 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.751944+0000", 2024-04-26T03:39:35.549 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.013529617000000001 2024-04-26T03:39:35.549 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.549 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.549 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:35.549 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.752019+0000", 2024-04-26T03:39:35.550 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.4784999999999996e-05 2024-04-26T03:39:35.550 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.550 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.550 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:35.550 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.752320+0000", 2024-04-26T03:39:35.550 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000301052 2024-04-26T03:39:35.550 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.550 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.550 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:35.550 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.753331+0000", 2024-04-26T03:39:35.550 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0010112610000000001 2024-04-26T03:39:35.550 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.550 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.550 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:35.551 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.756849+0000", 2024-04-26T03:39:35.551 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.003518017 2024-04-26T03:39:35.551 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.551 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.551 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:35.551 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.756901+0000", 2024-04-26T03:39:35.551 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.1758000000000001e-05 2024-04-26T03:39:35.551 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:35.551 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:35.551 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:35.551 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.551 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.551 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:759 3.14s0 3:2ee2abb1:::smithi09235267-12:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:35.551 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:20.738422+0000", 2024-04-26T03:39:35.551 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 14.708858638000001, 2024-04-26T03:39:35.552 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.020008317000000001, 2024-04-26T03:39:35.552 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:35.552 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:35.552 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "started", 2024-04-26T03:39:35.552 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:35.552 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:35.552 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:35.552 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 759 2024-04-26T03:39:35.552 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.552 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:35.552 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.552 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:35.552 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.738422+0000", 2024-04-26T03:39:35.552 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:35.553 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.553 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.553 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:35.553 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.738422+0000", 2024-04-26T03:39:35.553 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:35.553 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.553 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.553 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:35.553 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.738423+0000", 2024-04-26T03:39:35.553 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.4199999999999996e-07 2024-04-26T03:39:35.553 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.553 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.553 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:35.553 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.738425+0000", 2024-04-26T03:39:35.553 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.6910000000000002e-06 2024-04-26T03:39:35.554 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.554 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.554 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:35.554 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.738432+0000", 2024-04-26T03:39:35.554 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.2629999999999997e-06 2024-04-26T03:39:35.554 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.554 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.554 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:35.554 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.738438+0000", 2024-04-26T03:39:35.554 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.2360000000000004e-06 2024-04-26T03:39:35.554 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.554 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.554 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:35.554 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.752936+0000", 2024-04-26T03:39:35.555 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.014498363 2024-04-26T03:39:35.555 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.555 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.555 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:35.555 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.753003+0000", 2024-04-26T03:39:35.555 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.7168000000000005e-05 2024-04-26T03:39:35.555 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.555 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.555 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:35.555 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.757220+0000", 2024-04-26T03:39:35.555 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0042164799999999999 2024-04-26T03:39:35.555 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.555 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.555 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:35.556 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.757311+0000", 2024-04-26T03:39:35.556 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.0828000000000005e-05 2024-04-26T03:39:35.556 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.556 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.556 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:35.556 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.758430+0000", 2024-04-26T03:39:35.556 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.001119746 2024-04-26T03:39:35.556 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:35.556 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:35.556 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:35.556 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:35.556 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:35.556 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:39:35.558 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-04-26T03:39:35.558 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:39:35.556+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:39:35.659 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:39:35.658+0000 7f60d3c4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31757) UID: 0 2024-04-26T03:39:35.690 INFO:teuthology.orchestra.run.smithi028.stderr:noscrub is set 2024-04-26T03:39:35.690 INFO:teuthology.orchestra.run.smithi028.stderr:reweighted osd.7 to 1 (10000) 2024-04-26T03:39:35.694 INFO:tasks.rados.rados.0.smithi092.stdout:347: done (1 left) 2024-04-26T03:39:35.694 INFO:tasks.rados.rados.0.smithi092.stdout:348: done (0 left) 2024-04-26T03:39:35.694 INFO:tasks.rados.rados.0.smithi092.stdout:349: copy_from oid 5 from oid 4 current snap is 26 2024-04-26T03:39:35.694 INFO:tasks.rados.rados.0.smithi092.stdout:350: copy_from oid 19 from oid 24 current snap is 26 2024-04-26T03:39:35.694 INFO:tasks.rados.rados.0.smithi092.stdout:351: rollback oid 15 current snap is 26 2024-04-26T03:39:35.694 INFO:tasks.rados.rados.0.smithi092.stdout:rollback oid 15 to 23 2024-04-26T03:39:35.694 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 37 current snap is 26 2024-04-26T03:39:35.694 INFO:tasks.rados.rados.0.smithi092.stdout:352: seq_num 155 ranges {688128=212992} 2024-04-26T03:39:35.699 INFO:tasks.rados.rados.0.smithi092.stdout:352: writing smithi09235267-37 from 688128 to 901120 tid 1 2024-04-26T03:39:35.699 INFO:tasks.rados.rados.0.smithi092.stdout:353: delete oid 31 current snap is 26 2024-04-26T03:39:35.699 INFO:tasks.rados.rados.0.smithi092.stdout:351: finishing rollback tid 1 to smithi09235267-15 2024-04-26T03:39:35.699 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 15 v 0 (ObjNum 129 snap 0 seq_num 3583367168) dirty dne 2024-04-26T03:39:35.703 INFO:tasks.rados.rados.0.smithi092.stdout:351: done (4 left) 2024-04-26T03:39:35.704 INFO:tasks.rados.rados.0.smithi092.stdout:353: done (3 left) 2024-04-26T03:39:35.704 INFO:tasks.rados.rados.0.smithi092.stdout:354: snap_create 2024-04-26T03:39:35.711 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 8 1 2024-04-26T03:39:35.712 INFO:tasks.rados.rados.0.smithi092.stdout:352: finishing write tid 1 to smithi09235267-37 2024-04-26T03:39:35.712 INFO:tasks.rados.rados.0.smithi092.stdout:352: finishing write tid 2 to smithi09235267-37 2024-04-26T03:39:35.714 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:39:35.714 INFO:teuthology.orchestra.run.smithi028.stdout: "ops": [], 2024-04-26T03:39:35.714 INFO:teuthology.orchestra.run.smithi028.stdout: "num_ops": 0 2024-04-26T03:39:35.714 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:39:35.715 INFO:tasks.rados.rados.0.smithi092.stdout:352: finishing write tid 3 to smithi09235267-37 2024-04-26T03:39:35.715 INFO:tasks.rados.rados.0.smithi092.stdout:352: oid 37 updating version 0 to 84 2024-04-26T03:39:35.715 INFO:tasks.rados.rados.0.smithi092.stdout:352: oid 37 version 84 is already newer than 83 2024-04-26T03:39:35.715 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 37 v 84 (ObjNum 155 snap 26 seq_num 155) dirty exists 2024-04-26T03:39:35.715 INFO:tasks.rados.rados.0.smithi092.stdout:352: left oid 37 (ObjNum 155 snap 26 seq_num 155) 2024-04-26T03:39:35.725 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-04-26T03:39:35.735 INFO:tasks.rados.rados.0.smithi092.stdout:350: finishing copy_from racing read to smithi09235267-19 2024-04-26T03:39:35.737 INFO:tasks.rados.rados.0.smithi092.stdout:350: finishing copy_from to smithi09235267-19 2024-04-26T03:39:35.737 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 19 v 46 (ObjNum 78 snap 6 seq_num 78) dirty exists 2024-04-26T03:39:35.764 INFO:tasks.rados.rados.0.smithi092.stdout:349: finishing copy_from racing read to smithi09235267-5 2024-04-26T03:39:35.766 INFO:tasks.ceph.osd.8.smithi028.stderr:2024-04-26T03:39:35.764+0000 7f5bb75d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 31968) UID: 0 2024-04-26T03:39:35.780 INFO:tasks.rados.rados.0.smithi092.stdout:349: finishing copy_from to smithi09235267-5 2024-04-26T03:39:35.781 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 5 v 50 (ObjNum 127 snap 16 seq_num 127) dirty exists 2024-04-26T03:39:35.860 INFO:tasks.ceph.osd.14.smithi066.stderr:2024-04-26T03:39:35.859+0000 7f5395f08640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 31765) UID: 0 2024-04-26T03:39:35.874 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:39:35.874 INFO:teuthology.orchestra.run.smithi028.stdout: "ops": [], 2024-04-26T03:39:35.874 INFO:teuthology.orchestra.run.smithi028.stdout: "complaint_time": 30, 2024-04-26T03:39:35.875 INFO:teuthology.orchestra.run.smithi028.stdout: "num_blocked_ops": 0 2024-04-26T03:39:35.875 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:39:35.883 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-04-26T03:39:35.961 INFO:tasks.ceph.osd.4.smithi028.stderr:2024-04-26T03:39:35.959+0000 7fd9511b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31963) UID: 0 2024-04-26T03:39:35.995 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:39:35.995 INFO:teuthology.orchestra.run.smithi028.stdout: "size": 20, 2024-04-26T03:39:35.995 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 600, 2024-04-26T03:39:35.995 INFO:teuthology.orchestra.run.smithi028.stdout: "ops": [ 2024-04-26T03:39:35.995 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.995 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.es6 35/32 ECSubWrite(tid=2, reqid=client.4556.0:58, at_version=35'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:39:35.995 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.512937+0000", 2024-04-26T03:39:35.995 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 44.476769140999998, 2024-04-26T03:39:35.995 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0046369269999999999, 2024-04-26T03:39:35.995 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:35.995 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:35.996 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:35.996 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:35.996 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.996 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:35.996 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.512937+0000", 2024-04-26T03:39:35.996 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:35.996 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.996 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.996 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:35.996 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.512937+0000", 2024-04-26T03:39:35.996 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:35.996 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.996 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.996 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:35.997 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.512937+0000", 2024-04-26T03:39:35.997 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.8099999999999999e-07 2024-04-26T03:39:35.997 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.997 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.997 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:35.997 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.513175+0000", 2024-04-26T03:39:35.997 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000237851 2024-04-26T03:39:35.997 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.997 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.997 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:35.997 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.513196+0000", 2024-04-26T03:39:35.997 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.0344999999999999e-05 2024-04-26T03:39:35.997 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.997 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.997 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:35.998 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.513207+0000", 2024-04-26T03:39:35.998 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1083999999999999e-05 2024-04-26T03:39:35.998 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.998 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.998 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:35.998 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.513309+0000", 2024-04-26T03:39:35.998 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000102321 2024-04-26T03:39:35.998 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.998 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.998 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:35.998 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.513318+0000", 2024-04-26T03:39:35.998 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.5219999999999996e-06 2024-04-26T03:39:35.998 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.998 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.998 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:35.999 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.517543+0000", 2024-04-26T03:39:35.999 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0042245160000000002 2024-04-26T03:39:35.999 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.999 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.999 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:35.999 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.517574+0000", 2024-04-26T03:39:35.999 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.1007000000000003e-05 2024-04-26T03:39:35.999 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:35.999 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:35.999 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:35.999 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:35.999 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:35.999 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.es6 35/32 ECSubWrite(tid=16, reqid=client.4556.0:85, at_version=35'5, trim_to=0'0, roll_forward_to=35'4))", 2024-04-26T03:39:35.999 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.542343+0000", 2024-04-26T03:39:36.000 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 44.447363576999997, 2024-04-26T03:39:36.000 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0051706219999999997, 2024-04-26T03:39:36.000 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:36.000 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:36.000 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:36.000 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:36.000 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.000 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:36.000 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.542343+0000", 2024-04-26T03:39:36.000 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:36.000 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.000 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.000 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:36.000 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.542343+0000", 2024-04-26T03:39:36.001 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:36.001 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.001 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.001 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:36.001 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.542344+0000", 2024-04-26T03:39:36.001 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.488e-06 2024-04-26T03:39:36.001 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.001 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.001 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:36.001 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.542511+0000", 2024-04-26T03:39:36.001 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000166844 2024-04-26T03:39:36.001 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.001 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.001 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:36.001 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.542528+0000", 2024-04-26T03:39:36.002 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.6548000000000001e-05 2024-04-26T03:39:36.002 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.002 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.002 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:36.002 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.542535+0000", 2024-04-26T03:39:36.002 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.7610000000000007e-06 2024-04-26T03:39:36.002 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.002 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.002 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:36.002 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.542610+0000", 2024-04-26T03:39:36.002 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.4632e-05 2024-04-26T03:39:36.002 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.002 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.002 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:36.003 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.542620+0000", 2024-04-26T03:39:36.003 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0108e-05 2024-04-26T03:39:36.003 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.003 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.003 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:36.003 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.547469+0000", 2024-04-26T03:39:36.003 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0048486129999999999 2024-04-26T03:39:36.003 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.003 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.003 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:36.003 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.547513+0000", 2024-04-26T03:39:36.003 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.4628e-05 2024-04-26T03:39:36.003 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:36.003 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:36.004 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:36.004 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.004 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.004 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.es6 35/32 ECSubWrite(tid=24, reqid=client.4556.0:124, at_version=35'7, trim_to=0'0, roll_forward_to=35'6))", 2024-04-26T03:39:36.004 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.596283+0000", 2024-04-26T03:39:36.004 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 44.393423278, 2024-04-26T03:39:36.004 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0047651710000000003, 2024-04-26T03:39:36.004 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:36.004 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:36.004 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:36.004 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:36.004 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.004 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:36.004 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.596283+0000", 2024-04-26T03:39:36.004 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:36.005 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.005 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.005 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:36.005 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.596283+0000", 2024-04-26T03:39:36.005 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:36.005 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.005 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.005 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:36.005 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.596284+0000", 2024-04-26T03:39:36.005 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0950000000000001e-06 2024-04-26T03:39:36.005 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.005 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.005 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:36.005 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.596595+0000", 2024-04-26T03:39:36.005 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00031093400000000002 2024-04-26T03:39:36.006 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.006 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.006 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:36.006 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.596610+0000", 2024-04-26T03:39:36.006 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.5264000000000002e-05 2024-04-26T03:39:36.006 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.006 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.006 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:36.006 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.596617+0000", 2024-04-26T03:39:36.006 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.0859999999999996e-06 2024-04-26T03:39:36.006 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.006 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.006 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:36.006 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.596669+0000", 2024-04-26T03:39:36.006 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.1436e-05 2024-04-26T03:39:36.007 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.007 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.007 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:36.007 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.596675+0000", 2024-04-26T03:39:36.007 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.4030000000000004e-06 2024-04-26T03:39:36.007 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.007 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.007 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:36.007 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601010+0000", 2024-04-26T03:39:36.007 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0043352030000000001 2024-04-26T03:39:36.007 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.007 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.007 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:36.007 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601048+0000", 2024-04-26T03:39:36.008 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.7750000000000003e-05 2024-04-26T03:39:36.008 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:36.008 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:36.008 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:36.008 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.008 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.008 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.bs3 35/32 ECSubWrite(tid=20, reqid=client.4556.0:127, at_version=35'7, trim_to=0'0, roll_forward_to=35'6))", 2024-04-26T03:39:36.008 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.601363+0000", 2024-04-26T03:39:36.008 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 44.388343321000001, 2024-04-26T03:39:36.008 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.007091913, 2024-04-26T03:39:36.008 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:36.008 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:36.008 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:36.008 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:36.009 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.009 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:36.009 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601363+0000", 2024-04-26T03:39:36.009 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:36.009 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.009 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.009 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:36.009 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601363+0000", 2024-04-26T03:39:36.009 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:36.009 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.009 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.009 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:36.009 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601364+0000", 2024-04-26T03:39:36.009 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.54e-07 2024-04-26T03:39:36.009 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.010 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.010 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:36.010 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601498+0000", 2024-04-26T03:39:36.010 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00013417600000000001 2024-04-26T03:39:36.010 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.010 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.010 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:36.010 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601514+0000", 2024-04-26T03:39:36.010 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.6371999999999998e-05 2024-04-26T03:39:36.010 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.010 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.010 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:36.010 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601523+0000", 2024-04-26T03:39:36.011 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.0189999999999995e-06 2024-04-26T03:39:36.011 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.011 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.011 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:36.011 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601589+0000", 2024-04-26T03:39:36.011 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.5924000000000001e-05 2024-04-26T03:39:36.011 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.011 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.011 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:36.011 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.601598+0000", 2024-04-26T03:39:36.011 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.6940000000000001e-06 2024-04-26T03:39:36.011 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.011 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.011 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:36.011 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.608417+0000", 2024-04-26T03:39:36.012 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.006819121 2024-04-26T03:39:36.012 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.012 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.012 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:36.012 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.608455+0000", 2024-04-26T03:39:36.012 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.7753000000000003e-05 2024-04-26T03:39:36.012 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:36.012 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:36.012 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:36.012 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.012 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.012 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.bs3 35/32 ECSubWrite(tid=22, reqid=client.4556.0:133, at_version=35'9, trim_to=0'0, roll_forward_to=35'7))", 2024-04-26T03:39:36.012 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.610898+0000", 2024-04-26T03:39:36.012 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 44.378808112999998, 2024-04-26T03:39:36.012 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0048565170000000003, 2024-04-26T03:39:36.013 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:36.013 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:36.013 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:36.013 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:36.013 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.013 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:36.013 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.610898+0000", 2024-04-26T03:39:36.013 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:36.013 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.013 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.013 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:36.013 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.610898+0000", 2024-04-26T03:39:36.013 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:36.013 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.014 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.014 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:36.014 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.610899+0000", 2024-04-26T03:39:36.014 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.9500000000000001e-07 2024-04-26T03:39:36.014 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.014 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.014 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:36.014 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.611030+0000", 2024-04-26T03:39:36.014 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00013066099999999999 2024-04-26T03:39:36.014 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.014 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.014 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:36.014 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.611046+0000", 2024-04-26T03:39:36.014 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.6056e-05 2024-04-26T03:39:36.015 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.015 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.015 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:36.015 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.611053+0000", 2024-04-26T03:39:36.015 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.1999999999999997e-06 2024-04-26T03:39:36.015 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.015 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.015 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:36.015 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.611119+0000", 2024-04-26T03:39:36.015 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.5839000000000004e-05 2024-04-26T03:39:36.015 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.015 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.015 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:36.015 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.611136+0000", 2024-04-26T03:39:36.016 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.7070000000000001e-05 2024-04-26T03:39:36.016 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.016 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.016 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:36.016 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.615722+0000", 2024-04-26T03:39:36.016 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.004586499 2024-04-26T03:39:36.016 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.016 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.016 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:36.016 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.615755+0000", 2024-04-26T03:39:36.016 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.2296999999999999e-05 2024-04-26T03:39:36.016 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:36.016 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:36.016 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:36.016 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.017 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.017 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.es6 35/32 ECSubWrite(tid=34, reqid=client.4556.0:177, at_version=35'9, trim_to=0'0, roll_forward_to=35'8))", 2024-04-26T03:39:36.017 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.731787+0000", 2024-04-26T03:39:36.017 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 44.257918832999998, 2024-04-26T03:39:36.017 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0075575629999999998, 2024-04-26T03:39:36.017 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:36.017 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:36.017 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:36.017 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:36.017 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.017 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:36.017 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.731787+0000", 2024-04-26T03:39:36.017 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:36.017 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.018 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.018 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:36.018 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.731787+0000", 2024-04-26T03:39:36.018 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:36.018 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.018 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.018 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:36.018 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.731788+0000", 2024-04-26T03:39:36.018 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1200000000000001e-06 2024-04-26T03:39:36.018 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.018 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.018 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:36.018 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.732056+0000", 2024-04-26T03:39:36.018 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00026767099999999998 2024-04-26T03:39:36.018 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.019 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.019 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:36.019 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.732072+0000", 2024-04-26T03:39:36.019 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.6242000000000001e-05 2024-04-26T03:39:36.019 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.019 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.019 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:36.019 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.732083+0000", 2024-04-26T03:39:36.019 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0135e-05 2024-04-26T03:39:36.019 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.019 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.019 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:36.019 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.732152+0000", 2024-04-26T03:39:36.019 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.9325000000000005e-05 2024-04-26T03:39:36.019 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.020 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.020 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:36.020 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.732163+0000", 2024-04-26T03:39:36.020 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0728e-05 2024-04-26T03:39:36.020 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.020 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.020 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:36.020 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.739313+0000", 2024-04-26T03:39:36.020 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0071506599999999997 2024-04-26T03:39:36.020 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.020 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.020 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:36.020 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.739345+0000", 2024-04-26T03:39:36.020 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.1681999999999999e-05 2024-04-26T03:39:36.021 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:36.021 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:36.021 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:36.021 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.021 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.021 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.es6 35/32 ECSubWrite(tid=39, reqid=client.4556.0:207, at_version=35'12, trim_to=0'0, roll_forward_to=35'11))", 2024-04-26T03:39:36.021 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.807524+0000", 2024-04-26T03:39:36.021 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 44.182182406999999, 2024-04-26T03:39:36.021 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0081817109999999995, 2024-04-26T03:39:36.021 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:36.021 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:36.021 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:36.021 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:36.021 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.021 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:36.022 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.807524+0000", 2024-04-26T03:39:36.022 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:36.022 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.022 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.022 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:36.022 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.807524+0000", 2024-04-26T03:39:36.022 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:36.022 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.022 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.022 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:36.022 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.807525+0000", 2024-04-26T03:39:36.022 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.6700000000000002e-07 2024-04-26T03:39:36.023 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.023 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.023 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:36.023 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.808020+0000", 2024-04-26T03:39:36.023 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00049521799999999996 2024-04-26T03:39:36.023 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.023 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.023 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:36.023 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.808031+0000", 2024-04-26T03:39:36.023 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0862999999999999e-05 2024-04-26T03:39:36.023 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.023 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.023 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:36.023 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.808037+0000", 2024-04-26T03:39:36.024 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.438e-06 2024-04-26T03:39:36.024 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.024 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.024 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:36.024 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.808102+0000", 2024-04-26T03:39:36.024 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.4388999999999996e-05 2024-04-26T03:39:36.024 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.024 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.024 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:36.024 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.808114+0000", 2024-04-26T03:39:36.024 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.2211e-05 2024-04-26T03:39:36.024 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.024 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.024 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:36.025 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.815665+0000", 2024-04-26T03:39:36.025 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0075510259999999997 2024-04-26T03:39:36.025 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.025 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.025 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:36.025 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.815706+0000", 2024-04-26T03:39:36.025 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.0698999999999998e-05 2024-04-26T03:39:36.025 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:36.025 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:36.026 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:36.026 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.026 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.026 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.es6 35/32 ECSubWrite(tid=40, reqid=client.4556.0:208, at_version=35'13, trim_to=0'0, roll_forward_to=35'11))", 2024-04-26T03:39:36.026 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.811551+0000", 2024-04-26T03:39:36.026 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 44.178155414000003, 2024-04-26T03:39:36.026 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0045656179999999996, 2024-04-26T03:39:36.026 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:36.026 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:36.026 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:36.026 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:36.026 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.027 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:36.027 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.811551+0000", 2024-04-26T03:39:36.027 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:36.027 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.027 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.027 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:36.027 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.811551+0000", 2024-04-26T03:39:36.027 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:36.027 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.027 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.027 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:36.027 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.811551+0000", 2024-04-26T03:39:36.027 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.3900000000000005e-07 2024-04-26T03:39:36.027 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.027 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.028 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:36.028 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.811779+0000", 2024-04-26T03:39:36.028 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000227541 2024-04-26T03:39:36.028 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.028 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.028 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:36.028 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.811790+0000", 2024-04-26T03:39:36.028 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1076e-05 2024-04-26T03:39:36.028 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.028 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.028 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:36.028 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.811796+0000", 2024-04-26T03:39:36.028 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.643e-06 2024-04-26T03:39:36.028 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.029 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.029 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:36.029 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.813542+0000", 2024-04-26T03:39:36.029 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0017465969999999999 2024-04-26T03:39:36.029 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.029 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.029 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:36.029 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.813549+0000", 2024-04-26T03:39:36.029 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.5359999999999998e-06 2024-04-26T03:39:36.029 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.029 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.029 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:36.029 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.816088+0000", 2024-04-26T03:39:36.029 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0025395750000000001 2024-04-26T03:39:36.030 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.030 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.030 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:36.030 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.816116+0000", 2024-04-26T03:39:36.030 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.8110999999999999e-05 2024-04-26T03:39:36.030 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:36.030 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:36.030 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:36.030 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.030 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.030 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.es6 43/32 ECSubWrite(tid=48, reqid=client.4556.0:301, at_version=43'16, trim_to=0'0, roll_forward_to=35'14))", 2024-04-26T03:39:36.030 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:59.531146+0000", 2024-04-26T03:39:36.030 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 36.458560124000002, 2024-04-26T03:39:36.030 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.004955175, 2024-04-26T03:39:36.030 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:36.031 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:36.031 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:36.031 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:36.031 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.031 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:36.031 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.531146+0000", 2024-04-26T03:39:36.031 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:36.031 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.031 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.031 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:36.031 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.531146+0000", 2024-04-26T03:39:36.031 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:36.031 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.031 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.031 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:36.032 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.531146+0000", 2024-04-26T03:39:36.032 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.49e-07 2024-04-26T03:39:36.032 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.032 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.032 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:36.032 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.531483+0000", 2024-04-26T03:39:36.032 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00033656099999999998 2024-04-26T03:39:36.032 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.032 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.032 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:36.032 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.531497+0000", 2024-04-26T03:39:36.032 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.378e-05 2024-04-26T03:39:36.032 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.033 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.033 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:36.033 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.531506+0000", 2024-04-26T03:39:36.033 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.2159999999999995e-06 2024-04-26T03:39:36.033 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.033 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.033 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:36.033 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.531587+0000", 2024-04-26T03:39:36.033 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.1180999999999994e-05 2024-04-26T03:39:36.033 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.033 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.033 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:36.033 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.531599+0000", 2024-04-26T03:39:36.033 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.2169e-05 2024-04-26T03:39:36.033 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.034 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.034 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:36.034 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.536067+0000", 2024-04-26T03:39:36.034 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0044680099999999997 2024-04-26T03:39:36.034 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.034 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.034 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:36.034 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.536101+0000", 2024-04-26T03:39:36.034 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.4109000000000002e-05 2024-04-26T03:39:36.034 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:36.034 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:36.034 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:36.034 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.034 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.034 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.es6 43/32 ECSubWrite(tid=49, reqid=client.4556.0:302, at_version=43'17, trim_to=0'0, roll_forward_to=35'14))", 2024-04-26T03:39:36.035 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:59.537150+0000", 2024-04-26T03:39:36.035 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 36.452556424999997, 2024-04-26T03:39:36.035 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0058389130000000003, 2024-04-26T03:39:36.035 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:36.035 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:36.035 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:36.035 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:36.035 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.035 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:36.035 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.537150+0000", 2024-04-26T03:39:36.035 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:36.035 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.035 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.035 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:36.035 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.537150+0000", 2024-04-26T03:39:36.036 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:36.036 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.036 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.036 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:36.036 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.537151+0000", 2024-04-26T03:39:36.036 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.92e-07 2024-04-26T03:39:36.036 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.036 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.036 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:36.036 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.537707+0000", 2024-04-26T03:39:36.036 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00055628199999999996 2024-04-26T03:39:36.036 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.036 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.036 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:36.037 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.537720+0000", 2024-04-26T03:39:36.037 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.3434999999999999e-05 2024-04-26T03:39:36.037 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.037 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.037 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:36.037 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.537728+0000", 2024-04-26T03:39:36.037 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.8290000000000008e-06 2024-04-26T03:39:36.037 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.037 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.037 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:36.037 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.537787+0000", 2024-04-26T03:39:36.037 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.9249000000000001e-05 2024-04-26T03:39:36.037 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.038 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.038 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:36.038 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.537798+0000", 2024-04-26T03:39:36.038 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0997e-05 2024-04-26T03:39:36.038 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.038 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.038 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:36.038 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.542961+0000", 2024-04-26T03:39:36.038 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0051625320000000001 2024-04-26T03:39:36.038 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.038 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.038 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:36.039 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.542989+0000", 2024-04-26T03:39:36.039 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.7797000000000002e-05 2024-04-26T03:39:36.039 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:36.039 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:36.039 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:36.039 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.039 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.039 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.cs5 45/32 ECSubWrite(tid=53, reqid=client.4556.0:338, at_version=45'21, trim_to=0'0, roll_forward_to=42'19))", 2024-04-26T03:39:36.039 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:01.572023+0000", 2024-04-26T03:39:36.039 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 34.417683332999999, 2024-04-26T03:39:36.039 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0057687980000000003, 2024-04-26T03:39:36.039 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:36.039 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:36.040 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:36.040 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:36.040 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.040 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:36.040 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.572023+0000", 2024-04-26T03:39:36.040 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:36.040 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.040 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.040 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:36.040 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.572023+0000", 2024-04-26T03:39:36.040 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:36.040 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.040 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.040 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:36.040 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.572023+0000", 2024-04-26T03:39:36.041 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.8599999999999999e-07 2024-04-26T03:39:36.041 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.041 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.041 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:36.041 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.572451+0000", 2024-04-26T03:39:36.041 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00042800899999999997 2024-04-26T03:39:36.041 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.041 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.041 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:36.041 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.572468+0000", 2024-04-26T03:39:36.041 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.6991999999999999e-05 2024-04-26T03:39:36.041 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.041 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.041 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:36.042 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.572480+0000", 2024-04-26T03:39:36.042 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1338000000000001e-05 2024-04-26T03:39:36.042 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.042 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.042 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:36.042 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.572569+0000", 2024-04-26T03:39:36.042 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.9571e-05 2024-04-26T03:39:36.042 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.042 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.042 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:36.042 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.572582+0000", 2024-04-26T03:39:36.042 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.295e-05 2024-04-26T03:39:36.043 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.043 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.043 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:36.043 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.577759+0000", 2024-04-26T03:39:36.043 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0051768300000000003 2024-04-26T03:39:36.043 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.043 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.043 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:36.043 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.577792+0000", 2024-04-26T03:39:36.043 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.2722000000000003e-05 2024-04-26T03:39:36.043 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:36.043 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:36.043 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:36.043 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.044 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.044 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.es6 45/32 ECSubWrite(tid=62, reqid=client.4556.0:349, at_version=45'21, trim_to=0'0, roll_forward_to=43'19))", 2024-04-26T03:39:36.044 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:01.632894+0000", 2024-04-26T03:39:36.044 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 34.356812679000001, 2024-04-26T03:39:36.044 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0051349860000000002, 2024-04-26T03:39:36.044 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:36.044 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:36.044 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:36.044 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:36.044 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.044 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:36.044 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.632894+0000", 2024-04-26T03:39:36.044 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:36.044 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.045 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.045 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:36.045 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.632894+0000", 2024-04-26T03:39:36.045 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:36.045 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.045 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.045 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:36.045 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.632894+0000", 2024-04-26T03:39:36.045 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.6100000000000002e-07 2024-04-26T03:39:36.045 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.045 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.045 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:36.045 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.633352+0000", 2024-04-26T03:39:36.045 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00045790799999999998 2024-04-26T03:39:36.046 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.046 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.046 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:36.046 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.633364+0000", 2024-04-26T03:39:36.046 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.2157999999999999e-05 2024-04-26T03:39:36.046 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.046 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.046 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:36.046 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.633372+0000", 2024-04-26T03:39:36.046 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.9589999999999995e-06 2024-04-26T03:39:36.046 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.046 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.047 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:36.047 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.633431+0000", 2024-04-26T03:39:36.047 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.8690999999999998e-05 2024-04-26T03:39:36.047 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.047 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.047 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:36.047 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.633437+0000", 2024-04-26T03:39:36.047 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.798e-06 2024-04-26T03:39:36.047 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.047 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.047 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:36.047 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.637987+0000", 2024-04-26T03:39:36.047 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0045496130000000001 2024-04-26T03:39:36.047 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.048 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.048 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:36.048 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.638029+0000", 2024-04-26T03:39:36.048 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.1498000000000001e-05 2024-04-26T03:39:36.048 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:36.048 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:36.048 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:36.048 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.048 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.048 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.es6 45/32 ECSubWrite(tid=63, reqid=client.4556.0:350, at_version=45'22, trim_to=0'0, roll_forward_to=43'19))", 2024-04-26T03:39:36.048 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:01.637512+0000", 2024-04-26T03:39:36.048 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 34.352194371000003, 2024-04-26T03:39:36.049 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0059848239999999997, 2024-04-26T03:39:36.049 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:36.049 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:36.049 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:36.049 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:36.049 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.049 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:36.049 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.637512+0000", 2024-04-26T03:39:36.049 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:36.049 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.049 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.049 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:36.049 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.637512+0000", 2024-04-26T03:39:36.050 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:36.050 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.050 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.050 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:36.050 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.637513+0000", 2024-04-26T03:39:36.050 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.001e-06 2024-04-26T03:39:36.050 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.050 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.050 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:36.050 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.638154+0000", 2024-04-26T03:39:36.050 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00064122099999999998 2024-04-26T03:39:36.050 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.050 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.050 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:36.051 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.638164+0000", 2024-04-26T03:39:36.051 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0325e-05 2024-04-26T03:39:36.051 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.051 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.051 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:36.051 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.638170+0000", 2024-04-26T03:39:36.051 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.6980000000000003e-06 2024-04-26T03:39:36.051 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.051 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.051 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:36.051 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.638223+0000", 2024-04-26T03:39:36.051 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.2667000000000002e-05 2024-04-26T03:39:36.052 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.052 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.052 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:36.052 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.638232+0000", 2024-04-26T03:39:36.052 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.9439999999999994e-06 2024-04-26T03:39:36.052 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.052 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.052 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:36.052 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.643458+0000", 2024-04-26T03:39:36.052 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0052263850000000001 2024-04-26T03:39:36.052 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.052 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.052 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:36.052 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.643497+0000", 2024-04-26T03:39:36.053 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.8583e-05 2024-04-26T03:39:36.053 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:36.053 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:36.053 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:36.053 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.053 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.053 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.bs3 45/32 ECSubWrite(tid=68, reqid=client.4556.0:361, at_version=45'31, trim_to=0'0, roll_forward_to=43'30))", 2024-04-26T03:39:36.053 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:01.715320+0000", 2024-04-26T03:39:36.053 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 34.274385829000003, 2024-04-26T03:39:36.053 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0047019460000000003, 2024-04-26T03:39:36.053 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:36.053 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:36.053 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:36.053 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:36.054 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.054 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:36.054 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.715320+0000", 2024-04-26T03:39:36.054 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:36.054 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.054 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.054 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:36.054 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.715320+0000", 2024-04-26T03:39:36.054 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:36.054 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.054 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.054 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:36.055 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.715322+0000", 2024-04-26T03:39:36.055 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.248e-06 2024-04-26T03:39:36.055 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.055 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.055 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:36.055 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.715385+0000", 2024-04-26T03:39:36.055 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.3443999999999998e-05 2024-04-26T03:39:36.055 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.055 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.055 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:36.055 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.715406+0000", 2024-04-26T03:39:36.055 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.0573000000000001e-05 2024-04-26T03:39:36.055 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.056 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.056 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:36.056 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.715418+0000", 2024-04-26T03:39:36.056 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1960000000000001e-05 2024-04-26T03:39:36.056 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.056 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.056 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:36.056 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.715525+0000", 2024-04-26T03:39:36.056 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000107694 2024-04-26T03:39:36.056 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.056 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.056 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:36.056 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.715537+0000", 2024-04-26T03:39:36.057 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1749999999999999e-05 2024-04-26T03:39:36.057 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.057 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.057 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:36.057 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.719962+0000", 2024-04-26T03:39:36.057 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0044252349999999996 2024-04-26T03:39:36.057 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.057 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.057 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:36.057 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.720022+0000", 2024-04-26T03:39:36.057 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.0041999999999999e-05 2024-04-26T03:39:36.057 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:36.058 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:36.058 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:36.058 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.058 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.058 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.bs3 60/32 ECSubWrite(tid=128, reqid=client.4556.0:685, at_version=60'75, trim_to=0'0, roll_forward_to=60'73))", 2024-04-26T03:39:36.058 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:15.940662+0000", 2024-04-26T03:39:36.058 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 20.049044425000002, 2024-04-26T03:39:36.058 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0069063650000000002, 2024-04-26T03:39:36.058 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:36.058 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:36.058 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:36.058 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:36.059 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.059 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:36.059 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.940662+0000", 2024-04-26T03:39:36.059 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:36.059 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.059 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.059 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:36.059 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.940662+0000", 2024-04-26T03:39:36.059 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:36.059 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.059 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.059 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:36.059 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.940663+0000", 2024-04-26T03:39:36.059 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.037e-06 2024-04-26T03:39:36.060 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.060 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.060 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:36.060 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.940747+0000", 2024-04-26T03:39:36.060 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.3844000000000006e-05 2024-04-26T03:39:36.060 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.060 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.060 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:36.060 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.940763+0000", 2024-04-26T03:39:36.060 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.6506000000000001e-05 2024-04-26T03:39:36.060 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.060 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.061 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:36.061 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.940772+0000", 2024-04-26T03:39:36.061 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.4100000000000008e-06 2024-04-26T03:39:36.061 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.061 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.061 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:36.061 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.940861+0000", 2024-04-26T03:39:36.061 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.9896999999999997e-05 2024-04-26T03:39:36.061 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.061 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.061 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:36.061 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.940873+0000", 2024-04-26T03:39:36.061 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1344e-05 2024-04-26T03:39:36.061 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.062 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.062 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:36.062 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.947531+0000", 2024-04-26T03:39:36.062 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0066581390000000004 2024-04-26T03:39:36.062 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.062 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.062 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:36.062 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.947568+0000", 2024-04-26T03:39:36.062 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.7187999999999997e-05 2024-04-26T03:39:36.062 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:36.062 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:36.062 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:36.062 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.063 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.063 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.bs3 65/32 ECSubWrite(tid=135, reqid=client.4556.0:745, at_version=65'81, trim_to=0'0, roll_forward_to=64'79))", 2024-04-26T03:39:36.063 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:20.715978+0000", 2024-04-26T03:39:36.063 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 15.273728, 2024-04-26T03:39:36.063 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.007376657, 2024-04-26T03:39:36.063 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:36.063 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:36.063 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:36.063 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:36.063 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.063 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:36.063 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.715978+0000", 2024-04-26T03:39:36.063 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:36.064 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.064 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.064 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:36.064 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.715978+0000", 2024-04-26T03:39:36.064 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:36.064 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.064 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.064 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:36.064 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.715978+0000", 2024-04-26T03:39:36.064 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.4499999999999998e-07 2024-04-26T03:39:36.064 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.064 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.064 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:36.064 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.716165+0000", 2024-04-26T03:39:36.065 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0001863 2024-04-26T03:39:36.065 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.065 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.065 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:36.065 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.716178+0000", 2024-04-26T03:39:36.065 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.3005e-05 2024-04-26T03:39:36.065 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.065 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.065 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:36.065 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.716187+0000", 2024-04-26T03:39:36.065 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.5279999999999992e-06 2024-04-26T03:39:36.065 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.065 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.066 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:36.066 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.716320+0000", 2024-04-26T03:39:36.066 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00013288899999999999 2024-04-26T03:39:36.066 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.066 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.066 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:36.066 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.716332+0000", 2024-04-26T03:39:36.066 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1997000000000001e-05 2024-04-26T03:39:36.066 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.066 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.066 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:36.066 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.723322+0000", 2024-04-26T03:39:36.067 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0069894759999999997 2024-04-26T03:39:36.067 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.067 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.067 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:36.067 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.723355+0000", 2024-04-26T03:39:36.067 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.3216999999999997e-05 2024-04-26T03:39:36.067 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:36.067 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:36.067 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:36.067 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.067 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.067 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.bs3 65/32 ECSubWrite(tid=137, reqid=client.4556.0:747, at_version=65'83, trim_to=0'0, roll_forward_to=65'81))", 2024-04-26T03:39:36.067 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:20.728761+0000", 2024-04-26T03:39:36.068 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 15.260945264, 2024-04-26T03:39:36.068 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0055263029999999998, 2024-04-26T03:39:36.068 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:36.068 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:36.068 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:36.068 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:36.068 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.068 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:36.068 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.728761+0000", 2024-04-26T03:39:36.068 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:36.068 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.068 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.068 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:36.069 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.728761+0000", 2024-04-26T03:39:36.069 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:36.069 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.069 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.069 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:36.069 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.728764+0000", 2024-04-26T03:39:36.069 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.2490000000000001e-06 2024-04-26T03:39:36.069 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.069 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.069 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:36.069 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.728897+0000", 2024-04-26T03:39:36.069 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00013275900000000001 2024-04-26T03:39:36.070 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.070 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.070 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:36.070 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.728914+0000", 2024-04-26T03:39:36.070 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.7187e-05 2024-04-26T03:39:36.070 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.070 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.070 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:36.070 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.728923+0000", 2024-04-26T03:39:36.070 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.3920000000000003e-06 2024-04-26T03:39:36.070 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.070 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.070 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:36.071 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.728996+0000", 2024-04-26T03:39:36.071 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.3875000000000007e-05 2024-04-26T03:39:36.071 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.071 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.071 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:36.071 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.729007+0000", 2024-04-26T03:39:36.071 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0916e-05 2024-04-26T03:39:36.071 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.071 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.071 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:36.071 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.734257+0000", 2024-04-26T03:39:36.071 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0052494409999999997 2024-04-26T03:39:36.072 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.072 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.072 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:36.072 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.734287+0000", 2024-04-26T03:39:36.072 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.0484000000000002e-05 2024-04-26T03:39:36.072 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:36.072 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:36.072 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:36.072 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.072 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.072 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.es6 65/32 ECSubWrite(tid=127, reqid=client.4556.0:768, at_version=65'50, trim_to=0'0, roll_forward_to=63'48))", 2024-04-26T03:39:36.072 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:20.765003+0000", 2024-04-26T03:39:36.072 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 15.224703181000001, 2024-04-26T03:39:36.073 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0053955619999999996, 2024-04-26T03:39:36.073 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:36.073 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:36.073 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:36.073 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:36.073 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.073 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:36.073 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.765003+0000", 2024-04-26T03:39:36.073 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:36.073 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.073 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.073 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:36.073 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.765003+0000", 2024-04-26T03:39:36.073 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:36.074 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.074 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.074 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:36.074 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.765003+0000", 2024-04-26T03:39:36.074 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.4999999999999998e-07 2024-04-26T03:39:36.074 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.074 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.074 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:36.074 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.765522+0000", 2024-04-26T03:39:36.074 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00051831700000000002 2024-04-26T03:39:36.074 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.074 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.074 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:36.075 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.765533+0000", 2024-04-26T03:39:36.075 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1294000000000001e-05 2024-04-26T03:39:36.075 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.075 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.075 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:36.075 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.765540+0000", 2024-04-26T03:39:36.075 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.6159999999999998e-06 2024-04-26T03:39:36.075 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.075 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.075 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:36.075 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.765605+0000", 2024-04-26T03:39:36.075 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.5735999999999995e-05 2024-04-26T03:39:36.075 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.076 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.076 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:36.076 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.765618+0000", 2024-04-26T03:39:36.076 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.2792000000000001e-05 2024-04-26T03:39:36.076 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.076 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.076 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:36.076 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.770374+0000", 2024-04-26T03:39:36.076 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0047560329999999998 2024-04-26T03:39:36.076 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.076 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.076 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:36.076 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.770399+0000", 2024-04-26T03:39:36.076 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.4423999999999999e-05 2024-04-26T03:39:36.077 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:36.077 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:36.077 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:36.077 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.077 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.077 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.es6 65/32 ECSubWrite(tid=129, reqid=client.4556.0:770, at_version=65'52, trim_to=0'0, roll_forward_to=65'50))", 2024-04-26T03:39:36.077 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:20.779240+0000", 2024-04-26T03:39:36.077 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 15.210466173, 2024-04-26T03:39:36.077 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0055201820000000002, 2024-04-26T03:39:36.077 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:36.077 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:36.078 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:36.078 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:36.078 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.078 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:36.078 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.779240+0000", 2024-04-26T03:39:36.078 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:36.078 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.078 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.078 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:36.078 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.779240+0000", 2024-04-26T03:39:36.078 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:36.078 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.078 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.079 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:36.079 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.779241+0000", 2024-04-26T03:39:36.079 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.4600000000000004e-07 2024-04-26T03:39:36.079 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.079 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.079 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:36.079 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.779790+0000", 2024-04-26T03:39:36.079 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00054915699999999999 2024-04-26T03:39:36.079 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.079 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.079 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:36.079 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.779801+0000", 2024-04-26T03:39:36.079 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0847e-05 2024-04-26T03:39:36.080 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.080 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.080 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:36.080 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.779806+0000", 2024-04-26T03:39:36.080 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.2449999999999998e-06 2024-04-26T03:39:36.080 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.080 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.080 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:36.080 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.779853+0000", 2024-04-26T03:39:36.080 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.6592999999999999e-05 2024-04-26T03:39:36.080 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.080 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.080 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:36.080 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.779858+0000", 2024-04-26T03:39:36.081 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.7350000000000001e-06 2024-04-26T03:39:36.081 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.081 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.081 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:36.081 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.784731+0000", 2024-04-26T03:39:36.081 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0048723680000000002 2024-04-26T03:39:36.081 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.081 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.081 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:36.081 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.784760+0000", 2024-04-26T03:39:36.082 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.9590999999999998e-05 2024-04-26T03:39:36.082 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:36.082 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:36.082 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:36.082 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.082 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.082 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.cs5 75/32 ECSubWrite(tid=220, reqid=osd.11.0:220, at_version=75'58, trim_to=0'0, roll_forward_to=73'56))", 2024-04-26T03:39:36.082 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:30.691864+0000", 2024-04-26T03:39:36.082 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 5.2978422690000002, 2024-04-26T03:39:36.082 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0046289060000000003, 2024-04-26T03:39:36.082 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:36.082 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:36.082 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:36.083 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:36.083 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.083 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:36.083 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:30.691864+0000", 2024-04-26T03:39:36.083 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:36.083 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.083 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.083 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:36.083 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:30.691864+0000", 2024-04-26T03:39:36.083 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:36.083 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.083 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.083 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:36.084 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:30.691864+0000", 2024-04-26T03:39:36.084 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.0899999999999997e-07 2024-04-26T03:39:36.084 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.084 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.084 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:36.084 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:30.691890+0000", 2024-04-26T03:39:36.084 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.6194e-05 2024-04-26T03:39:36.084 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.084 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.084 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:36.084 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:30.691910+0000", 2024-04-26T03:39:36.084 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.9870000000000001e-05 2024-04-26T03:39:36.084 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.085 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.085 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:36.085 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:30.691921+0000", 2024-04-26T03:39:36.085 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0868e-05 2024-04-26T03:39:36.085 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.085 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.085 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:36.085 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:30.691991+0000", 2024-04-26T03:39:36.085 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.0017000000000004e-05 2024-04-26T03:39:36.085 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.085 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.085 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:36.085 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:30.692000+0000", 2024-04-26T03:39:36.085 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.2369999999999998e-06 2024-04-26T03:39:36.086 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.086 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.086 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:36.086 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:30.696441+0000", 2024-04-26T03:39:36.086 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.004440612 2024-04-26T03:39:36.086 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:36.086 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:36.086 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:36.086 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:30.696493+0000", 2024-04-26T03:39:36.086 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.1799000000000003e-05 2024-04-26T03:39:36.086 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:36.086 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:36.086 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:36.086 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:36.087 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:36.087 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:39:36.088 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-26T03:39:36.089 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:39:36.087+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:39:36.188 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-04-26T03:39:36.187+0000 7fe9223c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31954) UID: 0 2024-04-26T03:39:36.208 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-26T03:39:36.208 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-04-26T03:39:36.208 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-04-26T03:39:36.208 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-26T03:39:36.217 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-26T03:39:36.302 INFO:tasks.ceph.osd.9.smithi042.stderr:2024-04-26T03:39:36.300+0000 7fdac63b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 31960) UID: 0 2024-04-26T03:39:36.365 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-26T03:39:36.366 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-04-26T03:39:36.366 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-04-26T03:39:36.366 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-04-26T03:39:36.366 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-26T03:39:36.374 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-26T03:39:36.391 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:39:36.389+0000 7fba0d2ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31961) UID: 0 2024-04-26T03:39:36.491 INFO:tasks.ceph.osd.8.smithi028.stderr:2024-04-26T03:39:36.490+0000 7f5bb75d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 31968) UID: 0 2024-04-26T03:39:36.523 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-26T03:39:36.523 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-04-26T03:39:36.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-04-26T03:39:36.523 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-04-26T03:39:36.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.523 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:13 3.0s0 3:0a8403ef:::smithi09235267-5:head [append 0~81920,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:36.524 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.484308+0000", 2024-04-26T03:39:36.524 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.036426106999997, 2024-04-26T03:39:36.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017722486999999999, 2024-04-26T03:39:36.524 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:36.524 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:36.524 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:36.524 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:36.524 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:36.524 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:36.524 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 13 2024-04-26T03:39:36.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.524 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:36.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:36.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484308+0000", 2024-04-26T03:39:36.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:36.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:36.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484308+0000", 2024-04-26T03:39:36.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:36.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:36.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484311+0000", 2024-04-26T03:39:36.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8890000000000002e-06 2024-04-26T03:39:36.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:36.526 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484792+0000", 2024-04-26T03:39:36.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00048166600000000001 2024-04-26T03:39:36.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:36.526 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484806+0000", 2024-04-26T03:39:36.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3709999999999999e-05 2024-04-26T03:39:36.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:36.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484816+0000", 2024-04-26T03:39:36.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7000000000000003e-06 2024-04-26T03:39:36.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:36.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484975+0000", 2024-04-26T03:39:36.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015943199999999999 2024-04-26T03:39:36.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:36.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.485147+0000", 2024-04-26T03:39:36.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000171888 2024-04-26T03:39:36.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:36.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.486224+0000", 2024-04-26T03:39:36.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010766720000000001 2024-04-26T03:39:36.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:36.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.489541+0000", 2024-04-26T03:39:36.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033176210000000002 2024-04-26T03:39:36.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:39:36.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.501934+0000", 2024-04-26T03:39:36.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012392633 2024-04-26T03:39:36.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:36.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.502030+0000", 2024-04-26T03:39:36.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6275999999999998e-05 2024-04-26T03:39:36.529 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:36.529 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:36.529 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:36.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.530 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:14 3.0s0 3:0a8403ef:::smithi09235267-5:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:36.530 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.484901+0000", 2024-04-26T03:39:36.530 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.035832999999997, 2024-04-26T03:39:36.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017354086000000001, 2024-04-26T03:39:36.530 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:36.530 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:36.530 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:36.530 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:36.530 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:36.530 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:36.530 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 14 2024-04-26T03:39:36.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.531 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:36.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:36.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484901+0000", 2024-04-26T03:39:36.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:36.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:36.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484901+0000", 2024-04-26T03:39:36.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:36.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:36.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484902+0000", 2024-04-26T03:39:36.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1069999999999999e-06 2024-04-26T03:39:36.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:36.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484909+0000", 2024-04-26T03:39:36.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.323e-06 2024-04-26T03:39:36.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:36.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484919+0000", 2024-04-26T03:39:36.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0069e-05 2024-04-26T03:39:36.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:36.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484930+0000", 2024-04-26T03:39:36.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1092e-05 2024-04-26T03:39:36.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:36.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.487836+0000", 2024-04-26T03:39:36.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0029053569999999999 2024-04-26T03:39:36.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:36.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.487949+0000", 2024-04-26T03:39:36.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000112822 2024-04-26T03:39:36.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:36.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.488660+0000", 2024-04-26T03:39:36.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00071102899999999996 2024-04-26T03:39:36.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:36.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.490973+0000", 2024-04-26T03:39:36.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023132510000000001 2024-04-26T03:39:36.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:39:36.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.502166+0000", 2024-04-26T03:39:36.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011193639999999999 2024-04-26T03:39:36.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:36.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.502255+0000", 2024-04-26T03:39:36.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8395999999999999e-05 2024-04-26T03:39:36.535 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:36.535 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:36.535 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:36.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.535 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:15 3.0s0 3:0a8403ef:::smithi09235267-5:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:36.535 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.484947+0000", 2024-04-26T03:39:36.536 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.035787227, 2024-04-26T03:39:36.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024468274000000002, 2024-04-26T03:39:36.536 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:36.536 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:36.536 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-04-26T03:39:36.536 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:36.536 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:36.536 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:36.536 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15 2024-04-26T03:39:36.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.536 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:36.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:36.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484947+0000", 2024-04-26T03:39:36.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:36.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:36.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484947+0000", 2024-04-26T03:39:36.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:36.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:36.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484947+0000", 2024-04-26T03:39:36.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0999999999999997e-07 2024-04-26T03:39:36.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:36.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484952+0000", 2024-04-26T03:39:36.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.549e-06 2024-04-26T03:39:36.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:36.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484962+0000", 2024-04-26T03:39:36.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0064999999999999e-05 2024-04-26T03:39:36.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:36.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.484972+0000", 2024-04-26T03:39:36.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7529999999999996e-06 2024-04-26T03:39:36.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:36.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.489714+0000", 2024-04-26T03:39:36.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0047418649999999996 2024-04-26T03:39:36.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:36.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.489836+0000", 2024-04-26T03:39:36.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012253399999999999 2024-04-26T03:39:36.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:36.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.502300+0000", 2024-04-26T03:39:36.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012464098 2024-04-26T03:39:36.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:36.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.502390+0000", 2024-04-26T03:39:36.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9954e-05 2024-04-26T03:39:36.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:36.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.509415+0000", 2024-04-26T03:39:36.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0070245459999999996 2024-04-26T03:39:36.540 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:36.540 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:36.540 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:36.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.541 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:40 3.0s0 3:0e9b8628:::smithi09235267-14:head [append 0~229376,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:36.541 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.495863+0000", 2024-04-26T03:39:36.541 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.024870733999997, 2024-04-26T03:39:36.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013720777999999999, 2024-04-26T03:39:36.541 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:36.541 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:36.541 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:36.541 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:36.541 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:36.541 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:36.541 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 40 2024-04-26T03:39:36.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.541 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:36.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:36.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.495863+0000", 2024-04-26T03:39:36.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:36.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:36.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.495863+0000", 2024-04-26T03:39:36.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:36.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:36.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.495865+0000", 2024-04-26T03:39:36.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4090000000000001e-06 2024-04-26T03:39:36.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:36.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496307+0000", 2024-04-26T03:39:36.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00044159800000000002 2024-04-26T03:39:36.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:36.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496322+0000", 2024-04-26T03:39:36.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4935e-05 2024-04-26T03:39:36.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:36.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496338+0000", 2024-04-26T03:39:36.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6155e-05 2024-04-26T03:39:36.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:36.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496408+0000", 2024-04-26T03:39:36.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0183000000000002e-05 2024-04-26T03:39:36.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:36.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496716+0000", 2024-04-26T03:39:36.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000308024 2024-04-26T03:39:36.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:36.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.497681+0000", 2024-04-26T03:39:36.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00096483300000000001 2024-04-26T03:39:36.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:36.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.505243+0000", 2024-04-26T03:39:36.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0075613329999999999 2024-04-26T03:39:36.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:39:36.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.509516+0000", 2024-04-26T03:39:36.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0042732480000000003 2024-04-26T03:39:36.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:36.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.509584+0000", 2024-04-26T03:39:36.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8059999999999996e-05 2024-04-26T03:39:36.545 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:36.546 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:36.546 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:36.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.546 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:41 3.0s0 3:0e9b8628:::smithi09235267-14:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:36.546 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.496478+0000", 2024-04-26T03:39:36.546 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.024255474, 2024-04-26T03:39:36.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018225094000000001, 2024-04-26T03:39:36.546 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:36.546 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:36.546 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:36.546 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:36.547 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:36.547 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:36.547 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 41 2024-04-26T03:39:36.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.547 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:36.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:36.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496478+0000", 2024-04-26T03:39:36.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:36.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:36.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496478+0000", 2024-04-26T03:39:36.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:36.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:36.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496479+0000", 2024-04-26T03:39:36.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7700000000000003e-07 2024-04-26T03:39:36.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:36.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496563+0000", 2024-04-26T03:39:36.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3778999999999999e-05 2024-04-26T03:39:36.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:36.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496574+0000", 2024-04-26T03:39:36.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1144999999999999e-05 2024-04-26T03:39:36.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:36.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496586+0000", 2024-04-26T03:39:36.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1457999999999999e-05 2024-04-26T03:39:36.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:36.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.503975+0000", 2024-04-26T03:39:36.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0073896480000000004 2024-04-26T03:39:36.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:36.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504089+0000", 2024-04-26T03:39:36.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011359299999999999 2024-04-26T03:39:36.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:36.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504968+0000", 2024-04-26T03:39:36.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00087954500000000002 2024-04-26T03:39:36.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:36.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.508613+0000", 2024-04-26T03:39:36.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0036446500000000001 2024-04-26T03:39:36.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-04-26T03:39:36.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.514613+0000", 2024-04-26T03:39:36.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0060000000000000001 2024-04-26T03:39:36.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:36.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.514703+0000", 2024-04-26T03:39:36.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0399e-05 2024-04-26T03:39:36.551 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:36.551 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:36.551 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:36.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.551 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4556.0:42 3.0s0 3:0e9b8628:::smithi09235267-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:36.552 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.496943+0000", 2024-04-26T03:39:36.552 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.023790697999999, 2024-04-26T03:39:36.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021178255999999999, 2024-04-26T03:39:36.552 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:36.552 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:36.552 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-04-26T03:39:36.552 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-04-26T03:39:36.552 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4556", 2024-04-26T03:39:36.552 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:36.552 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 42 2024-04-26T03:39:36.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.552 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:36.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:36.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496943+0000", 2024-04-26T03:39:36.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:36.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:36.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496943+0000", 2024-04-26T03:39:36.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:36.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:36.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496944+0000", 2024-04-26T03:39:36.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9000000000000005e-07 2024-04-26T03:39:36.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:36.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496951+0000", 2024-04-26T03:39:36.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6470000000000001e-06 2024-04-26T03:39:36.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:36.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496962+0000", 2024-04-26T03:39:36.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1618e-05 2024-04-26T03:39:36.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:36.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.496971+0000", 2024-04-26T03:39:36.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6170000000000007e-06 2024-04-26T03:39:36.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:36.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.505538+0000", 2024-04-26T03:39:36.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0085666530000000005 2024-04-26T03:39:36.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:36.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.505620+0000", 2024-04-26T03:39:36.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2669000000000004e-05 2024-04-26T03:39:36.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:36.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.515261+0000", 2024-04-26T03:39:36.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0096405210000000009 2024-04-26T03:39:36.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-04-26T03:39:36.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.515342+0000", 2024-04-26T03:39:36.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1642000000000002e-05 2024-04-26T03:39:36.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:36.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.518121+0000", 2024-04-26T03:39:36.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002778899 2024-04-26T03:39:36.556 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:36.556 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:36.556 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:36.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.557 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds3 35/32 ECSubWrite(tid=1, reqid=client.4556.0:44, at_version=35'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:39:36.557 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.504056+0000", 2024-04-26T03:39:36.557 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.016677610000002, 2024-04-26T03:39:36.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0085269190000000009, 2024-04-26T03:39:36.557 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:36.557 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:36.557 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:36.557 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:36.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:36.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504056+0000", 2024-04-26T03:39:36.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:36.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:36.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504056+0000", 2024-04-26T03:39:36.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:36.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:36.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504057+0000", 2024-04-26T03:39:36.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2600000000000001e-07 2024-04-26T03:39:36.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:36.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504078+0000", 2024-04-26T03:39:36.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1250000000000002e-05 2024-04-26T03:39:36.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:36.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504092+0000", 2024-04-26T03:39:36.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3370000000000001e-05 2024-04-26T03:39:36.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:36.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504102+0000", 2024-04-26T03:39:36.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0047000000000001e-05 2024-04-26T03:39:36.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:36.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504408+0000", 2024-04-26T03:39:36.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030663799999999997 2024-04-26T03:39:36.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:36.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504417+0000", 2024-04-26T03:39:36.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8400000000000001e-06 2024-04-26T03:39:36.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:36.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.512551+0000", 2024-04-26T03:39:36.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0081339870000000005 2024-04-26T03:39:36.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:36.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.512583+0000", 2024-04-26T03:39:36.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1961000000000001e-05 2024-04-26T03:39:36.561 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:36.561 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:36.561 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:36.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.561 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.bs4 48/32 ECSubWrite(tid=78, reqid=osd.4.0:78, at_version=48'40, trim_to=0'0, roll_forward_to=47'36))", 2024-04-26T03:39:36.561 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:04.557110+0000", 2024-04-26T03:39:36.561 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.963623411, 2024-04-26T03:39:36.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0084124439999999998, 2024-04-26T03:39:36.561 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:36.561 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:36.561 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:36.562 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:36.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:36.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.557110+0000", 2024-04-26T03:39:36.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:36.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:36.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.557110+0000", 2024-04-26T03:39:36.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:36.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:36.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.557111+0000", 2024-04-26T03:39:36.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7199999999999997e-07 2024-04-26T03:39:36.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:36.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.557136+0000", 2024-04-26T03:39:36.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4270999999999999e-05 2024-04-26T03:39:36.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:36.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.557146+0000", 2024-04-26T03:39:36.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0832999999999999e-05 2024-04-26T03:39:36.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:36.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.557155+0000", 2024-04-26T03:39:36.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3960000000000006e-06 2024-04-26T03:39:36.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:36.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.557237+0000", 2024-04-26T03:39:36.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2167000000000001e-05 2024-04-26T03:39:36.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:36.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.557248+0000", 2024-04-26T03:39:36.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1283e-05 2024-04-26T03:39:36.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:36.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.565491+0000", 2024-04-26T03:39:36.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0082427439999999998 2024-04-26T03:39:36.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:36.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.565523+0000", 2024-04-26T03:39:36.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1778e-05 2024-04-26T03:39:36.565 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:36.565 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:36.565 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:36.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.566 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.13s4 50/44 ECSubWrite(tid=74, reqid=client.4556.0:446, at_version=50'37, trim_to=0'0, roll_forward_to=50'36))", 2024-04-26T03:39:36.566 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:05.702160+0000", 2024-04-26T03:39:36.566 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 30.818573998000002, 2024-04-26T03:39:36.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.008210953, 2024-04-26T03:39:36.566 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:36.566 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:36.566 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:36.566 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:36.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:36.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.702160+0000", 2024-04-26T03:39:36.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:36.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:36.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.702160+0000", 2024-04-26T03:39:36.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:36.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:36.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.702161+0000", 2024-04-26T03:39:36.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3900000000000003e-07 2024-04-26T03:39:36.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:36.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.702542+0000", 2024-04-26T03:39:36.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038117199999999999 2024-04-26T03:39:36.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:36.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.702556+0000", 2024-04-26T03:39:36.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3720000000000001e-05 2024-04-26T03:39:36.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:36.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.702562+0000", 2024-04-26T03:39:36.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3899999999999998e-06 2024-04-26T03:39:36.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:36.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.702632+0000", 2024-04-26T03:39:36.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9956999999999994e-05 2024-04-26T03:39:36.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:36.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.702644+0000", 2024-04-26T03:39:36.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1627000000000001e-05 2024-04-26T03:39:36.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:36.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.710338+0000", 2024-04-26T03:39:36.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0076939809999999999 2024-04-26T03:39:36.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:36.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.710371+0000", 2024-04-26T03:39:36.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3167000000000003e-05 2024-04-26T03:39:36.570 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:36.570 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:36.570 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:36.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.570 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.18s2 51/44 ECSubWrite(tid=216, reqid=client.4556.0:485, at_version=51'46, trim_to=0'0, roll_forward_to=50'43))", 2024-04-26T03:39:36.570 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:06.709835+0000", 2024-04-26T03:39:36.570 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.810899179, 2024-04-26T03:39:36.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0077202110000000003, 2024-04-26T03:39:36.570 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:36.570 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:36.570 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:36.571 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:36.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:36.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.709835+0000", 2024-04-26T03:39:36.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:36.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:36.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.709835+0000", 2024-04-26T03:39:36.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:36.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:36.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.709835+0000", 2024-04-26T03:39:36.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.23e-07 2024-04-26T03:39:36.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:36.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.710337+0000", 2024-04-26T03:39:36.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00050177999999999996 2024-04-26T03:39:36.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:36.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.710351+0000", 2024-04-26T03:39:36.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3573e-05 2024-04-26T03:39:36.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:36.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.710358+0000", 2024-04-26T03:39:36.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7959999999999998e-06 2024-04-26T03:39:36.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:36.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.710421+0000", 2024-04-26T03:39:36.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3349000000000006e-05 2024-04-26T03:39:36.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:36.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.710431+0000", 2024-04-26T03:39:36.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9210000000000004e-06 2024-04-26T03:39:36.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:36.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.717526+0000", 2024-04-26T03:39:36.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0070952710000000002 2024-04-26T03:39:36.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:36.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.717555+0000", 2024-04-26T03:39:36.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8697999999999999e-05 2024-04-26T03:39:36.574 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:36.574 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:36.574 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:36.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.575 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds3 56/32 ECSubWrite(tid=29, reqid=client.4556.0:608, at_version=56'26, trim_to=0'0, roll_forward_to=50'23))", 2024-04-26T03:39:36.575 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:11.731789+0000", 2024-04-26T03:39:36.575 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 24.788944334, 2024-04-26T03:39:36.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012144034999999999, 2024-04-26T03:39:36.575 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:36.575 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:36.575 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:36.575 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:36.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:36.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.731789+0000", 2024-04-26T03:39:36.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:36.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:36.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.731789+0000", 2024-04-26T03:39:36.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:36.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:36.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.731790+0000", 2024-04-26T03:39:36.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4000000000000001e-07 2024-04-26T03:39:36.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:36.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.732064+0000", 2024-04-26T03:39:36.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00027422299999999998 2024-04-26T03:39:36.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:36.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.732077+0000", 2024-04-26T03:39:36.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3016999999999999e-05 2024-04-26T03:39:36.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:36.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.732084+0000", 2024-04-26T03:39:36.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5780000000000004e-06 2024-04-26T03:39:36.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:36.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.732145+0000", 2024-04-26T03:39:36.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1008000000000003e-05 2024-04-26T03:39:36.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:36.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.732157+0000", 2024-04-26T03:39:36.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1875000000000001e-05 2024-04-26T03:39:36.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:36.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.743838+0000", 2024-04-26T03:39:36.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011680885 2024-04-26T03:39:36.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:36.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.743933+0000", 2024-04-26T03:39:36.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5709000000000001e-05 2024-04-26T03:39:36.579 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:36.579 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:36.579 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:36.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.579 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.13s6 56/55 ECSubWrite(tid=104, reqid=client.4556.0:604, at_version=56'47, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:39:36.579 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:11.774744+0000", 2024-04-26T03:39:36.579 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 24.745989968, 2024-04-26T03:39:36.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.016770859999999999, 2024-04-26T03:39:36.579 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:36.580 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:36.580 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:36.580 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:36.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:36.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.774744+0000", 2024-04-26T03:39:36.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:36.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:36.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.774744+0000", 2024-04-26T03:39:36.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:36.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:36.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.774746+0000", 2024-04-26T03:39:36.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3829999999999999e-06 2024-04-26T03:39:36.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:36.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.776913+0000", 2024-04-26T03:39:36.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0021663659999999999 2024-04-26T03:39:36.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:36.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.776926+0000", 2024-04-26T03:39:36.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3059e-05 2024-04-26T03:39:36.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:36.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.776932+0000", 2024-04-26T03:39:36.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9699999999999996e-06 2024-04-26T03:39:36.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:36.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.776980+0000", 2024-04-26T03:39:36.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8177000000000003e-05 2024-04-26T03:39:36.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:36.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.776986+0000", 2024-04-26T03:39:36.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.122e-06 2024-04-26T03:39:36.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:36.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.791479+0000", 2024-04-26T03:39:36.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.014492777 2024-04-26T03:39:36.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:36.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.791515+0000", 2024-04-26T03:39:36.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6006000000000001e-05 2024-04-26T03:39:36.583 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:36.583 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:36.583 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:36.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.584 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDPGPush(3.13s6 56/55 [PushOp(3:c8a9e28b:::smithi09235267-30:head, version: 49'32, data_included: [0~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 4, recovery_info: ObjectRecoveryInfo(3:c8a9e28b:::smithi09235267-30:head@49'32, size: 2768896, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-04-26T03:39:36.584 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:11.811532+0000", 2024-04-26T03:39:36.584 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 24.709201668999999, 2024-04-26T03:39:36.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010374899999999999, 2024-04-26T03:39:36.584 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:36.584 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:36.584 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:36.584 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:36.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:36.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.811532+0000", 2024-04-26T03:39:36.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:36.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:36.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.811532+0000", 2024-04-26T03:39:36.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:36.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.811533+0000", 2024-04-26T03:39:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0800000000000005e-07 2024-04-26T03:39:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.812185+0000", 2024-04-26T03:39:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00065258999999999998 2024-04-26T03:39:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.812203+0000", 2024-04-26T03:39:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7367000000000002e-05 2024-04-26T03:39:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.812213+0000", 2024-04-26T03:39:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0431999999999999e-05 2024-04-26T03:39:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.816475+0000", 2024-04-26T03:39:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0042618659999999996 2024-04-26T03:39:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:36.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.821907+0000", 2024-04-26T03:39:36.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0054321370000000001 2024-04-26T03:39:36.588 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:36.588 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:36.588 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:36.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.588 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.as5 65/32 ECSubWrite(tid=120, reqid=client.4556.0:760, at_version=65'70, trim_to=0'0, roll_forward_to=64'68))", 2024-04-26T03:39:36.588 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:20.749517+0000", 2024-04-26T03:39:36.588 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.771216946999999, 2024-04-26T03:39:36.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0090199089999999996, 2024-04-26T03:39:36.588 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:36.588 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:36.588 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:36.588 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:36.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:36.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.749517+0000", 2024-04-26T03:39:36.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:36.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:36.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.749517+0000", 2024-04-26T03:39:36.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:36.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:36.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.749517+0000", 2024-04-26T03:39:36.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3700000000000001e-07 2024-04-26T03:39:36.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:36.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.749876+0000", 2024-04-26T03:39:36.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00035863799999999999 2024-04-26T03:39:36.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:36.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.749953+0000", 2024-04-26T03:39:36.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6791000000000004e-05 2024-04-26T03:39:36.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:36.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.749965+0000", 2024-04-26T03:39:36.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2159e-05 2024-04-26T03:39:36.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:36.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.750155+0000", 2024-04-26T03:39:36.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019052399999999999 2024-04-26T03:39:36.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:36.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.750173+0000", 2024-04-26T03:39:36.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8142999999999999e-05 2024-04-26T03:39:36.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:36.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.758502+0000", 2024-04-26T03:39:36.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0083287299999999995 2024-04-26T03:39:36.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:36.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.758537+0000", 2024-04-26T03:39:36.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4586999999999999e-05 2024-04-26T03:39:36.592 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:36.592 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:36.592 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:36.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.592 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds3 65/32 ECSubWrite(tid=53, reqid=client.4556.0:774, at_version=65'51, trim_to=0'0, roll_forward_to=63'49))", 2024-04-26T03:39:36.592 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:20.782051+0000", 2024-04-26T03:39:36.592 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.738682357, 2024-04-26T03:39:36.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.014196823000000001, 2024-04-26T03:39:36.593 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:36.593 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:36.593 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:36.593 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:36.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:36.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.782051+0000", 2024-04-26T03:39:36.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:36.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:36.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.782051+0000", 2024-04-26T03:39:36.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:36.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:36.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.782052+0000", 2024-04-26T03:39:36.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0400000000000002e-07 2024-04-26T03:39:36.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:36.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.782454+0000", 2024-04-26T03:39:36.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00040211000000000001 2024-04-26T03:39:36.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:36.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.782471+0000", 2024-04-26T03:39:36.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6866000000000001e-05 2024-04-26T03:39:36.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:36.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.782480+0000", 2024-04-26T03:39:36.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3840000000000003e-06 2024-04-26T03:39:36.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:36.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.782539+0000", 2024-04-26T03:39:36.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8488e-05 2024-04-26T03:39:36.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:36.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.782545+0000", 2024-04-26T03:39:36.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6340000000000003e-06 2024-04-26T03:39:36.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:36.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.796204+0000", 2024-04-26T03:39:36.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.01365923 2024-04-26T03:39:36.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:36.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.796248+0000", 2024-04-26T03:39:36.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3807000000000003e-05 2024-04-26T03:39:36.596 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:36.596 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:36.597 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:36.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.597 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds3 65/32 ECSubWrite(tid=54, reqid=client.4556.0:775, at_version=65'52, trim_to=0'0, roll_forward_to=63'49))", 2024-04-26T03:39:36.597 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:20.790269+0000", 2024-04-26T03:39:36.597 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.730464857999999, 2024-04-26T03:39:36.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011371925, 2024-04-26T03:39:36.597 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:36.597 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:36.597 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:36.597 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:36.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:36.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.790269+0000", 2024-04-26T03:39:36.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:36.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:36.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.790269+0000", 2024-04-26T03:39:36.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:36.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:36.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.790270+0000", 2024-04-26T03:39:36.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2810000000000001e-06 2024-04-26T03:39:36.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:36.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.790660+0000", 2024-04-26T03:39:36.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00039025699999999997 2024-04-26T03:39:36.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:36.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.790702+0000", 2024-04-26T03:39:36.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1925999999999998e-05 2024-04-26T03:39:36.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:36.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.790735+0000", 2024-04-26T03:39:36.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2833999999999998e-05 2024-04-26T03:39:36.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:36.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.790832+0000", 2024-04-26T03:39:36.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6279000000000004e-05 2024-04-26T03:39:36.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:36.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.790843+0000", 2024-04-26T03:39:36.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1236000000000001e-05 2024-04-26T03:39:36.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:36.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.801595+0000", 2024-04-26T03:39:36.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010752467999999999 2024-04-26T03:39:36.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:36.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.801641+0000", 2024-04-26T03:39:36.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5643999999999999e-05 2024-04-26T03:39:36.601 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:36.601 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:36.601 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:36.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.601 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds3 65/32 ECSubWrite(tid=55, reqid=client.4556.0:776, at_version=65'53, trim_to=0'0, roll_forward_to=63'49))", 2024-04-26T03:39:36.601 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:20.796125+0000", 2024-04-26T03:39:36.601 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.72460834, 2024-04-26T03:39:36.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0079654329999999992, 2024-04-26T03:39:36.601 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:36.601 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:36.601 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:36.602 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:36.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:36.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.796125+0000", 2024-04-26T03:39:36.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:36.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:36.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.796125+0000", 2024-04-26T03:39:36.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:36.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:36.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.796129+0000", 2024-04-26T03:39:36.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7680000000000002e-06 2024-04-26T03:39:36.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:36.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.796414+0000", 2024-04-26T03:39:36.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000285176 2024-04-26T03:39:36.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:36.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.796439+0000", 2024-04-26T03:39:36.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4484000000000001e-05 2024-04-26T03:39:36.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:36.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.796451+0000", 2024-04-26T03:39:36.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2126999999999999e-05 2024-04-26T03:39:36.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:36.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.796530+0000", 2024-04-26T03:39:36.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9370999999999996e-05 2024-04-26T03:39:36.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:36.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.796541+0000", 2024-04-26T03:39:36.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0149999999999999e-05 2024-04-26T03:39:36.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:36.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.804060+0000", 2024-04-26T03:39:36.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0075198720000000004 2024-04-26T03:39:36.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:36.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.804091+0000", 2024-04-26T03:39:36.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0485e-05 2024-04-26T03:39:36.605 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:36.605 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:36.605 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:36.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.605 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.fs5 65/62 ECSubWrite(tid=179, reqid=client.4556.0:783, at_version=65'61, trim_to=0'0, roll_forward_to=65'59))", 2024-04-26T03:39:36.605 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:20.833861+0000", 2024-04-26T03:39:36.605 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.686872798, 2024-04-26T03:39:36.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0087749960000000002, 2024-04-26T03:39:36.606 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:36.606 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:36.606 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:36.606 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:36.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:36.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.833861+0000", 2024-04-26T03:39:36.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:36.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:36.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.833861+0000", 2024-04-26T03:39:36.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:36.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:36.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.833862+0000", 2024-04-26T03:39:36.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.850000000000001e-07 2024-04-26T03:39:36.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:36.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.834542+0000", 2024-04-26T03:39:36.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00067977000000000001 2024-04-26T03:39:36.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:36.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.834559+0000", 2024-04-26T03:39:36.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7496e-05 2024-04-26T03:39:36.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:36.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.834568+0000", 2024-04-26T03:39:36.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9320000000000002e-06 2024-04-26T03:39:36.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:36.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.834668+0000", 2024-04-26T03:39:36.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9345000000000005e-05 2024-04-26T03:39:36.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:36.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.834680+0000", 2024-04-26T03:39:36.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2611e-05 2024-04-26T03:39:36.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:36.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.842565+0000", 2024-04-26T03:39:36.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0078843999999999997 2024-04-26T03:39:36.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:36.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.842636+0000", 2024-04-26T03:39:36.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1457000000000004e-05 2024-04-26T03:39:36.609 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:36.609 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:36.610 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:36.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.610 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.as5 65/32 ECSubWrite(tid=128, reqid=client.4556.0:790, at_version=65'76, trim_to=0'0, roll_forward_to=65'74))", 2024-04-26T03:39:36.610 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:20.851305+0000", 2024-04-26T03:39:36.610 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.669428676000001, 2024-04-26T03:39:36.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0096478870000000008, 2024-04-26T03:39:36.610 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:36.610 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:36.610 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:36.610 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:36.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:36.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.851305+0000", 2024-04-26T03:39:36.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:36.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:36.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.851305+0000", 2024-04-26T03:39:36.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:36.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:36.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.851307+0000", 2024-04-26T03:39:36.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6470000000000001e-06 2024-04-26T03:39:36.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:36.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.851854+0000", 2024-04-26T03:39:36.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00054689799999999998 2024-04-26T03:39:36.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:36.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.851865+0000", 2024-04-26T03:39:36.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1174e-05 2024-04-26T03:39:36.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:36.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.851870+0000", 2024-04-26T03:39:36.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5040000000000002e-06 2024-04-26T03:39:36.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:36.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.851923+0000", 2024-04-26T03:39:36.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2964000000000003e-05 2024-04-26T03:39:36.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:36.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.851931+0000", 2024-04-26T03:39:36.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.272e-06 2024-04-26T03:39:36.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:36.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.860913+0000", 2024-04-26T03:39:36.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0089822849999999996 2024-04-26T03:39:36.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:36.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.860953+0000", 2024-04-26T03:39:36.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0142999999999998e-05 2024-04-26T03:39:36.614 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:36.614 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:36.614 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:36.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.614 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.8s2 80/63 ECSubWrite(tid=321, reqid=client.4556.0:897, at_version=80'111, trim_to=0'0, roll_forward_to=80'110))", 2024-04-26T03:39:36.614 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:35.716771+0000", 2024-04-26T03:39:36.614 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.80396251600000002, 2024-04-26T03:39:36.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018807276000000001, 2024-04-26T03:39:36.614 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:36.614 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:36.614 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:36.615 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:36.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:36.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:35.716771+0000", 2024-04-26T03:39:36.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:36.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:36.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:35.716771+0000", 2024-04-26T03:39:36.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:36.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:36.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:35.716773+0000", 2024-04-26T03:39:36.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.677e-06 2024-04-26T03:39:36.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:36.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:35.719804+0000", 2024-04-26T03:39:36.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0030310139999999998 2024-04-26T03:39:36.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:36.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:35.719818+0000", 2024-04-26T03:39:36.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.381e-05 2024-04-26T03:39:36.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:36.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:35.719829+0000", 2024-04-26T03:39:36.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0917999999999999e-05 2024-04-26T03:39:36.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:36.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:35.719901+0000", 2024-04-26T03:39:36.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1888e-05 2024-04-26T03:39:36.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:36.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:35.719910+0000", 2024-04-26T03:39:36.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9500000000000007e-06 2024-04-26T03:39:36.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:36.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:35.735547+0000", 2024-04-26T03:39:36.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015637343000000001 2024-04-26T03:39:36.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:36.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:36.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:36.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:35.735579+0000", 2024-04-26T03:39:36.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1676e-05 2024-04-26T03:39:36.618 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:36.618 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:36.618 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:36.618 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:36.618 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:36.618 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-26T03:39:36.620 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:39:36.618+0000 7fba0d2ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31961) UID: 0 2024-04-26T03:39:36.620 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-26T03:39:36.692 INFO:teuthology.orchestra.run.smithi028.stderr:reweighted osd.8 to 1 (10000) 2024-04-26T03:39:36.695 INFO:tasks.rados.rados.0.smithi092.stdout:349: done (3 left) 2024-04-26T03:39:36.695 INFO:tasks.rados.rados.0.smithi092.stdout:350: done (2 left) 2024-04-26T03:39:36.695 INFO:tasks.rados.rados.0.smithi092.stdout:352: done (1 left) 2024-04-26T03:39:36.695 INFO:tasks.rados.rados.0.smithi092.stdout:354: done (0 left) 2024-04-26T03:39:36.695 INFO:tasks.rados.rados.0.smithi092.stdout:355: delete oid 49 current snap is 27 2024-04-26T03:39:36.704 INFO:tasks.rados.rados.0.smithi092.stdout:355: done (0 left) 2024-04-26T03:39:36.704 INFO:tasks.rados.rados.0.smithi092.stdout:356: setattr oid 1 current snap is 27 2024-04-26T03:39:36.705 INFO:tasks.rados.rados.0.smithi092.stdout:357: snap_create 2024-04-26T03:39:36.709 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 9 1 2024-04-26T03:39:36.710 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 1 v 65 (ObjNum 65 snap 0 seq_num 1330664049) dirty exists 2024-04-26T03:39:36.720 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:39:36.719+0000 7fb78c544640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31956) UID: 0 2024-04-26T03:39:36.730 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-26T03:39:36.730 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-04-26T03:39:36.731 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-04-26T03:39:36.731 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-26T03:39:36.739 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-26T03:39:36.823 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-04-26T03:39:36.822+0000 7fe9223c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31954) UID: 0 2024-04-26T03:39:36.887 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-26T03:39:36.887 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-04-26T03:39:36.887 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-04-26T03:39:36.887 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-04-26T03:39:36.887 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-26T03:39:36.897 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-26T03:39:36.941 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:39:36.938+0000 7fb78c544640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31956) UID: 0 2024-04-26T03:39:37.024 INFO:tasks.ceph.osd.12.smithi028.stderr:2024-04-26T03:39:37.022+0000 7f114fe14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 31967) UID: 0 2024-04-26T03:39:37.048 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-26T03:39:37.049 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-04-26T03:39:37.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-04-26T03:39:37.049 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-04-26T03:39:37.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.049 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds4 35/32 ECSubWrite(tid=7, reqid=client.4556.0:64, at_version=35'5, trim_to=0'0, roll_forward_to=35'4))", 2024-04-26T03:39:37.049 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.528665+0000", 2024-04-26T03:39:37.049 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.517757957999997, 2024-04-26T03:39:37.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0061053569999999996, 2024-04-26T03:39:37.049 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:37.049 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:37.049 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:37.049 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:37.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:37.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528665+0000", 2024-04-26T03:39:37.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:37.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528665+0000", 2024-04-26T03:39:37.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:37.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528666+0000", 2024-04-26T03:39:37.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.071e-06 2024-04-26T03:39:37.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:37.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528902+0000", 2024-04-26T03:39:37.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00023567299999999999 2024-04-26T03:39:37.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:37.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528920+0000", 2024-04-26T03:39:37.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8252000000000001e-05 2024-04-26T03:39:37.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:37.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.528930+0000", 2024-04-26T03:39:37.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7329999999999998e-06 2024-04-26T03:39:37.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.052 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:37.052 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.529014+0000", 2024-04-26T03:39:37.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3945999999999999e-05 2024-04-26T03:39:37.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.052 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:37.052 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.529031+0000", 2024-04-26T03:39:37.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6960999999999999e-05 2024-04-26T03:39:37.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.052 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:37.052 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.534732+0000", 2024-04-26T03:39:37.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0057011630000000004 2024-04-26T03:39:37.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:37.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.534771+0000", 2024-04-26T03:39:37.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8557999999999999e-05 2024-04-26T03:39:37.053 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.053 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:37.053 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.053 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.4s6 40/32 ECSubWrite(tid=22, reqid=client.4556.0:262, at_version=40'14, trim_to=0'0, roll_forward_to=35'12))", 2024-04-26T03:39:37.053 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:56.521608+0000", 2024-04-26T03:39:37.054 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 40.524814781000003, 2024-04-26T03:39:37.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.006347615, 2024-04-26T03:39:37.054 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:37.054 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:37.054 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:37.054 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:37.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:37.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.521608+0000", 2024-04-26T03:39:37.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:37.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.521608+0000", 2024-04-26T03:39:37.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:37.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.521609+0000", 2024-04-26T03:39:37.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9000000000000001e-07 2024-04-26T03:39:37.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:37.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.521956+0000", 2024-04-26T03:39:37.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00034728800000000001 2024-04-26T03:39:37.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:37.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.521972+0000", 2024-04-26T03:39:37.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6150999999999998e-05 2024-04-26T03:39:37.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:37.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.521980+0000", 2024-04-26T03:39:37.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1990000000000006e-06 2024-04-26T03:39:37.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:37.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.522048+0000", 2024-04-26T03:39:37.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8235e-05 2024-04-26T03:39:37.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:37.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.522055+0000", 2024-04-26T03:39:37.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0639999999999996e-06 2024-04-26T03:39:37.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:37.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.527920+0000", 2024-04-26T03:39:37.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0058647300000000003 2024-04-26T03:39:37.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:37.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:56.527956+0000", 2024-04-26T03:39:37.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5757999999999999e-05 2024-04-26T03:39:37.058 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.058 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:37.058 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.058 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.2s1 49/44 ECSubWrite(tid=32, reqid=client.4556.0:433, at_version=49'20, trim_to=0'0, roll_forward_to=48'19))", 2024-04-26T03:39:37.058 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:04.699918+0000", 2024-04-26T03:39:37.058 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.346505237999999, 2024-04-26T03:39:37.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0082878350000000003, 2024-04-26T03:39:37.058 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:37.059 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:37.059 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:37.059 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:37.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:37.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.699918+0000", 2024-04-26T03:39:37.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:37.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.699918+0000", 2024-04-26T03:39:37.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:37.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.699918+0000", 2024-04-26T03:39:37.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6300000000000005e-07 2024-04-26T03:39:37.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:37.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.700410+0000", 2024-04-26T03:39:37.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00049148099999999999 2024-04-26T03:39:37.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:37.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.700422+0000", 2024-04-26T03:39:37.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2466000000000001e-05 2024-04-26T03:39:37.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:37.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.700431+0000", 2024-04-26T03:39:37.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2479999999999996e-06 2024-04-26T03:39:37.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:37.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.700559+0000", 2024-04-26T03:39:37.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012855599999999999 2024-04-26T03:39:37.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:37.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.700570+0000", 2024-04-26T03:39:37.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0652000000000001e-05 2024-04-26T03:39:37.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:37.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.708166+0000", 2024-04-26T03:39:37.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0075965770000000002 2024-04-26T03:39:37.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:37.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.708206+0000", 2024-04-26T03:39:37.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9292000000000003e-05 2024-04-26T03:39:37.063 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.063 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:37.063 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.063 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.18s6 49/44 ECSubWrite(tid=206, reqid=client.4556.0:443, at_version=49'36, trim_to=0'0, roll_forward_to=49'34))", 2024-04-26T03:39:37.063 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:04.734641+0000", 2024-04-26T03:39:37.063 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.311782336, 2024-04-26T03:39:37.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0066362670000000004, 2024-04-26T03:39:37.064 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:37.064 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:37.064 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:37.064 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:37.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:37.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.734641+0000", 2024-04-26T03:39:37.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:37.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.734641+0000", 2024-04-26T03:39:37.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:37.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.734642+0000", 2024-04-26T03:39:37.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.516e-06 2024-04-26T03:39:37.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:37.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.735354+0000", 2024-04-26T03:39:37.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00071165699999999998 2024-04-26T03:39:37.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:37.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.735370+0000", 2024-04-26T03:39:37.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6175e-05 2024-04-26T03:39:37.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:37.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.735377+0000", 2024-04-26T03:39:37.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1559999999999998e-06 2024-04-26T03:39:37.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:37.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.735445+0000", 2024-04-26T03:39:37.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7557000000000004e-05 2024-04-26T03:39:37.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:37.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.735455+0000", 2024-04-26T03:39:37.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0165e-05 2024-04-26T03:39:37.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:37.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.741242+0000", 2024-04-26T03:39:37.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.005787309 2024-04-26T03:39:37.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:37.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.741277+0000", 2024-04-26T03:39:37.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4731999999999999e-05 2024-04-26T03:39:37.068 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.068 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:37.068 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.068 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.14s6 49/44 ECSubWrite(tid=36, reqid=client.4556.0:439, at_version=49'25, trim_to=0'0, roll_forward_to=46'22))", 2024-04-26T03:39:37.068 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:04.734819+0000", 2024-04-26T03:39:37.068 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.311604561000003, 2024-04-26T03:39:37.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0079486820000000003, 2024-04-26T03:39:37.068 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:37.068 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:37.068 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:37.068 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:37.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:37.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.734819+0000", 2024-04-26T03:39:37.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:37.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.734819+0000", 2024-04-26T03:39:37.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:37.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.734819+0000", 2024-04-26T03:39:37.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9999999999999997e-07 2024-04-26T03:39:37.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:37.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.735094+0000", 2024-04-26T03:39:37.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00027495999999999998 2024-04-26T03:39:37.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:37.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.735111+0000", 2024-04-26T03:39:37.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6872999999999999e-05 2024-04-26T03:39:37.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:37.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.735122+0000", 2024-04-26T03:39:37.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1055e-05 2024-04-26T03:39:37.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:37.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.735189+0000", 2024-04-26T03:39:37.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7075000000000004e-05 2024-04-26T03:39:37.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:37.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.735199+0000", 2024-04-26T03:39:37.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8479999999999996e-06 2024-04-26T03:39:37.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:37.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.742740+0000", 2024-04-26T03:39:37.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0075408699999999999 2024-04-26T03:39:37.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:37.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.742767+0000", 2024-04-26T03:39:37.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7300999999999998e-05 2024-04-26T03:39:37.072 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.072 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:37.073 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.073 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.18s6 51/44 ECSubWrite(tid=215, reqid=client.4556.0:484, at_version=51'45, trim_to=0'0, roll_forward_to=50'43))", 2024-04-26T03:39:37.073 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:06.705341+0000", 2024-04-26T03:39:37.073 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 30.341082544999999, 2024-04-26T03:39:37.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0089840560000000007, 2024-04-26T03:39:37.073 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:37.073 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:37.073 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:37.073 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:37.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:37.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.705341+0000", 2024-04-26T03:39:37.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:37.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.705341+0000", 2024-04-26T03:39:37.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:37.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.705341+0000", 2024-04-26T03:39:37.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7700000000000001e-07 2024-04-26T03:39:37.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:37.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.705617+0000", 2024-04-26T03:39:37.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000276655 2024-04-26T03:39:37.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:37.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.705633+0000", 2024-04-26T03:39:37.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.537e-05 2024-04-26T03:39:37.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:37.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.705643+0000", 2024-04-26T03:39:37.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0516000000000001e-05 2024-04-26T03:39:37.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:37.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.705722+0000", 2024-04-26T03:39:37.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8232999999999994e-05 2024-04-26T03:39:37.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:37.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.705735+0000", 2024-04-26T03:39:37.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3645000000000001e-05 2024-04-26T03:39:37.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:37.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.714285+0000", 2024-04-26T03:39:37.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0085501989999999996 2024-04-26T03:39:37.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:37.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:06.714325+0000", 2024-04-26T03:39:37.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9261000000000002e-05 2024-04-26T03:39:37.077 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.077 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:37.077 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.077 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds4 56/32 ECSubWrite(tid=28, reqid=client.4556.0:607, at_version=56'25, trim_to=0'0, roll_forward_to=50'23))", 2024-04-26T03:39:37.077 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:11.724542+0000", 2024-04-26T03:39:37.078 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.321881044000001, 2024-04-26T03:39:37.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.014413491, 2024-04-26T03:39:37.078 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:37.078 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:37.078 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:37.078 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:37.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:37.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.724542+0000", 2024-04-26T03:39:37.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:37.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.724542+0000", 2024-04-26T03:39:37.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:37.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.724542+0000", 2024-04-26T03:39:37.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8700000000000001e-07 2024-04-26T03:39:37.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:37.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.725098+0000", 2024-04-26T03:39:37.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000555465 2024-04-26T03:39:37.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:37.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.725111+0000", 2024-04-26T03:39:37.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3043e-05 2024-04-26T03:39:37.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:37.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.725120+0000", 2024-04-26T03:39:37.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0710000000000001e-06 2024-04-26T03:39:37.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:37.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.725201+0000", 2024-04-26T03:39:37.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0914000000000005e-05 2024-04-26T03:39:37.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:37.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.725211+0000", 2024-04-26T03:39:37.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7440000000000002e-06 2024-04-26T03:39:37.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:37.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.738695+0000", 2024-04-26T03:39:37.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013484489000000001 2024-04-26T03:39:37.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:37.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.738956+0000", 2024-04-26T03:39:37.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026037800000000002 2024-04-26T03:39:37.082 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.082 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:37.082 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.082 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds4 56/32 ECSubWrite(tid=29, reqid=client.4556.0:608, at_version=56'26, trim_to=0'0, roll_forward_to=50'23))", 2024-04-26T03:39:37.082 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:11.731977+0000", 2024-04-26T03:39:37.082 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.314445893999999, 2024-04-26T03:39:37.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012989461000000001, 2024-04-26T03:39:37.082 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:37.082 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:37.082 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:37.082 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:37.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:37.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.731977+0000", 2024-04-26T03:39:37.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:37.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.731977+0000", 2024-04-26T03:39:37.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:37.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.731978+0000", 2024-04-26T03:39:37.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8100000000000001e-07 2024-04-26T03:39:37.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:37.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.732398+0000", 2024-04-26T03:39:37.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00041989499999999999 2024-04-26T03:39:37.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:37.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.732413+0000", 2024-04-26T03:39:37.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4861999999999999e-05 2024-04-26T03:39:37.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:37.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.732423+0000", 2024-04-26T03:39:37.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.026e-05 2024-04-26T03:39:37.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:37.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.732866+0000", 2024-04-26T03:39:37.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00044276299999999999 2024-04-26T03:39:37.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:37.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.732879+0000", 2024-04-26T03:39:37.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2632999999999999e-05 2024-04-26T03:39:37.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:37.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.744874+0000", 2024-04-26T03:39:37.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011995530000000001 2024-04-26T03:39:37.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:37.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.744967+0000", 2024-04-26T03:39:37.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2637000000000001e-05 2024-04-26T03:39:37.086 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.087 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:37.087 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.087 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.5s6 56/44 ECSubWrite(tid=70, reqid=client.4556.0:615, at_version=56'57, trim_to=0'0, roll_forward_to=52'55))", 2024-04-26T03:39:37.087 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:11.733862+0000", 2024-04-26T03:39:37.087 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.312560706999999, 2024-04-26T03:39:37.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011141698, 2024-04-26T03:39:37.087 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:37.087 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:37.087 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:37.087 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:37.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:37.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.733862+0000", 2024-04-26T03:39:37.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:37.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.733862+0000", 2024-04-26T03:39:37.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:37.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.733863+0000", 2024-04-26T03:39:37.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7100000000000001e-07 2024-04-26T03:39:37.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:37.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.733978+0000", 2024-04-26T03:39:37.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000115505 2024-04-26T03:39:37.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:37.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.733991+0000", 2024-04-26T03:39:37.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3251999999999999e-05 2024-04-26T03:39:37.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:37.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.733998+0000", 2024-04-26T03:39:37.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.392e-06 2024-04-26T03:39:37.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:37.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.738752+0000", 2024-04-26T03:39:37.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0047539269999999998 2024-04-26T03:39:37.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:37.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.738762+0000", 2024-04-26T03:39:37.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0429e-05 2024-04-26T03:39:37.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:37.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.744977+0000", 2024-04-26T03:39:37.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0062144380000000001 2024-04-26T03:39:37.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:37.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.745004+0000", 2024-04-26T03:39:37.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7583999999999999e-05 2024-04-26T03:39:37.091 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.091 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:37.091 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.091 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds4 56/32 ECSubWrite(tid=30, reqid=client.4556.0:609, at_version=56'27, trim_to=0'0, roll_forward_to=50'23))", 2024-04-26T03:39:37.091 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:11.737431+0000", 2024-04-26T03:39:37.091 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.308991882000001, 2024-04-26T03:39:37.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0080512009999999991, 2024-04-26T03:39:37.091 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:37.092 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:37.092 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:37.092 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:37.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:37.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.737431+0000", 2024-04-26T03:39:37.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:37.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.737431+0000", 2024-04-26T03:39:37.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:37.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.737433+0000", 2024-04-26T03:39:37.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.717e-06 2024-04-26T03:39:37.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:37.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.737960+0000", 2024-04-26T03:39:37.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00052740199999999995 2024-04-26T03:39:37.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:37.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.738089+0000", 2024-04-26T03:39:37.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012901699999999999 2024-04-26T03:39:37.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:37.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.738102+0000", 2024-04-26T03:39:37.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2456999999999999e-05 2024-04-26T03:39:37.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:37.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.739001+0000", 2024-04-26T03:39:37.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00089964699999999995 2024-04-26T03:39:37.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:37.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.739011+0000", 2024-04-26T03:39:37.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6800000000000005e-06 2024-04-26T03:39:37.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:37.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.745459+0000", 2024-04-26T03:39:37.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0064480750000000002 2024-04-26T03:39:37.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:37.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.745482+0000", 2024-04-26T03:39:37.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3206000000000001e-05 2024-04-26T03:39:37.095 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.095 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:37.095 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.096 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds4 60/32 ECSubWrite(tid=44, reqid=client.4556.0:690, at_version=60'42, trim_to=0'0, roll_forward_to=60'40))", 2024-04-26T03:39:37.096 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:15.945751+0000", 2024-04-26T03:39:37.096 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 21.100671996999999, 2024-04-26T03:39:37.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0063316780000000003, 2024-04-26T03:39:37.096 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:37.096 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:37.096 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:37.096 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:37.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:37.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.945751+0000", 2024-04-26T03:39:37.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:37.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.945751+0000", 2024-04-26T03:39:37.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:37.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.945752+0000", 2024-04-26T03:39:37.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5000000000000002e-07 2024-04-26T03:39:37.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:37.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.946392+0000", 2024-04-26T03:39:37.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00063980299999999999 2024-04-26T03:39:37.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:37.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.946402+0000", 2024-04-26T03:39:37.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0674e-05 2024-04-26T03:39:37.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:37.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.946409+0000", 2024-04-26T03:39:37.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2580000000000001e-06 2024-04-26T03:39:37.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:37.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.946465+0000", 2024-04-26T03:39:37.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6110000000000003e-05 2024-04-26T03:39:37.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:37.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.946477+0000", 2024-04-26T03:39:37.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2092e-05 2024-04-26T03:39:37.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:37.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.952052+0000", 2024-04-26T03:39:37.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0055748830000000001 2024-04-26T03:39:37.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:37.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.952083+0000", 2024-04-26T03:39:37.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1207999999999997e-05 2024-04-26T03:39:37.100 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.100 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:37.100 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.100 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds4 60/32 ECSubWrite(tid=45, reqid=client.4556.0:691, at_version=60'43, trim_to=0'0, roll_forward_to=60'41))", 2024-04-26T03:39:37.100 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:15.954483+0000", 2024-04-26T03:39:37.100 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 21.091940421, 2024-04-26T03:39:37.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0060120410000000001, 2024-04-26T03:39:37.100 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:37.100 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:37.100 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:37.100 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:37.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:37.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.954483+0000", 2024-04-26T03:39:37.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:37.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.954483+0000", 2024-04-26T03:39:37.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:37.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.954483+0000", 2024-04-26T03:39:37.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.44e-07 2024-04-26T03:39:37.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:37.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.954871+0000", 2024-04-26T03:39:37.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000387842 2024-04-26T03:39:37.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:37.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.954885+0000", 2024-04-26T03:39:37.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.397e-05 2024-04-26T03:39:37.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:37.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.954898+0000", 2024-04-26T03:39:37.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2578999999999999e-05 2024-04-26T03:39:37.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:37.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.954957+0000", 2024-04-26T03:39:37.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9021000000000003e-05 2024-04-26T03:39:37.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:37.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.954968+0000", 2024-04-26T03:39:37.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.146e-05 2024-04-26T03:39:37.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:37.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.960457+0000", 2024-04-26T03:39:37.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0054888719999999997 2024-04-26T03:39:37.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:37.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.960495+0000", 2024-04-26T03:39:37.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7653e-05 2024-04-26T03:39:37.104 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.104 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:37.104 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.104 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.14s6 65/44 ECSubWrite(tid=59, reqid=client.4556.0:754, at_version=65'37, trim_to=0'0, roll_forward_to=52'35))", 2024-04-26T03:39:37.104 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:20.731540+0000", 2024-04-26T03:39:37.104 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 16.314883024, 2024-04-26T03:39:37.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0060911840000000004, 2024-04-26T03:39:37.105 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:37.105 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:37.105 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:37.105 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:37.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:37.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.731540+0000", 2024-04-26T03:39:37.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:37.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.731540+0000", 2024-04-26T03:39:37.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:37.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.731540+0000", 2024-04-26T03:39:37.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.23e-07 2024-04-26T03:39:37.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:37.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.732062+0000", 2024-04-26T03:39:37.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00052151900000000002 2024-04-26T03:39:37.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:37.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.732074+0000", 2024-04-26T03:39:37.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2619999999999999e-05 2024-04-26T03:39:37.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:37.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.732087+0000", 2024-04-26T03:39:37.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2894000000000001e-05 2024-04-26T03:39:37.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:37.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.732159+0000", 2024-04-26T03:39:37.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2105000000000002e-05 2024-04-26T03:39:37.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:37.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.732168+0000", 2024-04-26T03:39:37.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0159999999999997e-06 2024-04-26T03:39:37.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:37.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.737587+0000", 2024-04-26T03:39:37.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0054181079999999996 2024-04-26T03:39:37.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:37.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.737631+0000", 2024-04-26T03:39:37.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4699e-05 2024-04-26T03:39:37.108 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.108 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:37.108 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.109 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds4 65/32 ECSubWrite(tid=53, reqid=client.4556.0:774, at_version=65'51, trim_to=0'0, roll_forward_to=63'49))", 2024-04-26T03:39:37.109 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:20.782153+0000", 2024-04-26T03:39:37.109 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 16.264270223, 2024-04-26T03:39:37.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0087244720000000005, 2024-04-26T03:39:37.109 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:37.109 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:37.109 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:37.109 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:37.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:37.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.782153+0000", 2024-04-26T03:39:37.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:37.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.782153+0000", 2024-04-26T03:39:37.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:37.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.782153+0000", 2024-04-26T03:39:37.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.39e-07 2024-04-26T03:39:37.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:37.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.782787+0000", 2024-04-26T03:39:37.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00063373399999999995 2024-04-26T03:39:37.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:37.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.782803+0000", 2024-04-26T03:39:37.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5841e-05 2024-04-26T03:39:37.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:37.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.782813+0000", 2024-04-26T03:39:37.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0083e-05 2024-04-26T03:39:37.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:37.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.782908+0000", 2024-04-26T03:39:37.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4667999999999996e-05 2024-04-26T03:39:37.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:37.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.782918+0000", 2024-04-26T03:39:37.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0861e-05 2024-04-26T03:39:37.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:37.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.790839+0000", 2024-04-26T03:39:37.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0079204960000000008 2024-04-26T03:39:37.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:37.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.790877+0000", 2024-04-26T03:39:37.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8449999999999999e-05 2024-04-26T03:39:37.113 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.113 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:37.113 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.113 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds4 65/32 ECSubWrite(tid=54, reqid=client.4556.0:775, at_version=65'52, trim_to=0'0, roll_forward_to=63'49))", 2024-04-26T03:39:37.113 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:20.790281+0000", 2024-04-26T03:39:37.113 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 16.256142376, 2024-04-26T03:39:37.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011523500000000001, 2024-04-26T03:39:37.113 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:37.113 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:37.114 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:37.114 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:37.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:37.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.790281+0000", 2024-04-26T03:39:37.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:37.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.790281+0000", 2024-04-26T03:39:37.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:37.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.790282+0000", 2024-04-26T03:39:37.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0920000000000001e-06 2024-04-26T03:39:37.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:37.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.790956+0000", 2024-04-26T03:39:37.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00067404499999999996 2024-04-26T03:39:37.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:37.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.790994+0000", 2024-04-26T03:39:37.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7777000000000001e-05 2024-04-26T03:39:37.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:37.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.791009+0000", 2024-04-26T03:39:37.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4931e-05 2024-04-26T03:39:37.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:37.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.791081+0000", 2024-04-26T03:39:37.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2205000000000004e-05 2024-04-26T03:39:37.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:37.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.791091+0000", 2024-04-26T03:39:37.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0675e-05 2024-04-26T03:39:37.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:37.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.801700+0000", 2024-04-26T03:39:37.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.01060902 2024-04-26T03:39:37.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:37.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.801804+0000", 2024-04-26T03:39:37.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000103755 2024-04-26T03:39:37.117 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.118 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:37.118 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.118 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds4 65/32 ECSubWrite(tid=55, reqid=client.4556.0:776, at_version=65'53, trim_to=0'0, roll_forward_to=63'49))", 2024-04-26T03:39:37.118 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:20.796218+0000", 2024-04-26T03:39:37.118 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 16.250205065999999, 2024-04-26T03:39:37.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0082157159999999996, 2024-04-26T03:39:37.118 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:37.118 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:37.118 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:37.118 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:37.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:37.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.796218+0000", 2024-04-26T03:39:37.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:37.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.796218+0000", 2024-04-26T03:39:37.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:37.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.796219+0000", 2024-04-26T03:39:37.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.009e-06 2024-04-26T03:39:37.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:37.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.796738+0000", 2024-04-26T03:39:37.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00051900000000000004 2024-04-26T03:39:37.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:37.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.796755+0000", 2024-04-26T03:39:37.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6574000000000001e-05 2024-04-26T03:39:37.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:37.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.796764+0000", 2024-04-26T03:39:37.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6830000000000003e-06 2024-04-26T03:39:37.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:37.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.796833+0000", 2024-04-26T03:39:37.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8410000000000004e-05 2024-04-26T03:39:37.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:37.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.796844+0000", 2024-04-26T03:39:37.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1232e-05 2024-04-26T03:39:37.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:37.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.804411+0000", 2024-04-26T03:39:37.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0075670579999999998 2024-04-26T03:39:37.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:37.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.804434+0000", 2024-04-26T03:39:37.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2750000000000001e-05 2024-04-26T03:39:37.122 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.122 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:37.122 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.122 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.fs2 65/62 ECSubWrite(tid=177, reqid=client.4556.0:781, at_version=65'59, trim_to=0'0, roll_forward_to=64'57))", 2024-04-26T03:39:37.123 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:20.821419+0000", 2024-04-26T03:39:37.123 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 16.225004063, 2024-04-26T03:39:37.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.006888567, 2024-04-26T03:39:37.123 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:37.123 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:37.123 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:37.123 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:37.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:37.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.821419+0000", 2024-04-26T03:39:37.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:37.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.821419+0000", 2024-04-26T03:39:37.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:37.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.821419+0000", 2024-04-26T03:39:37.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.91e-07 2024-04-26T03:39:37.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:37.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.822029+0000", 2024-04-26T03:39:37.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00060977700000000004 2024-04-26T03:39:37.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:37.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.822045+0000", 2024-04-26T03:39:37.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5875999999999998e-05 2024-04-26T03:39:37.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:37.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.822054+0000", 2024-04-26T03:39:37.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4350000000000003e-06 2024-04-26T03:39:37.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:37.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.822137+0000", 2024-04-26T03:39:37.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2844999999999997e-05 2024-04-26T03:39:37.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:37.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.822145+0000", 2024-04-26T03:39:37.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.011e-06 2024-04-26T03:39:37.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:37.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.828271+0000", 2024-04-26T03:39:37.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0061262030000000002 2024-04-26T03:39:37.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:37.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.828308+0000", 2024-04-26T03:39:37.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6229000000000002e-05 2024-04-26T03:39:37.127 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.127 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:37.127 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.127 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.fs2 65/62 ECSubWrite(tid=178, reqid=client.4556.0:782, at_version=65'60, trim_to=0'0, roll_forward_to=64'57))", 2024-04-26T03:39:37.127 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:20.829285+0000", 2024-04-26T03:39:37.127 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 16.217137792999999, 2024-04-26T03:39:37.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010418624, 2024-04-26T03:39:37.127 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:37.127 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:37.127 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:37.127 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:37.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:37.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.829285+0000", 2024-04-26T03:39:37.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:37.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.829285+0000", 2024-04-26T03:39:37.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:37.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.829286+0000", 2024-04-26T03:39:37.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0299999999999998e-07 2024-04-26T03:39:37.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:37.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.829548+0000", 2024-04-26T03:39:37.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000262122 2024-04-26T03:39:37.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:37.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.829565+0000", 2024-04-26T03:39:37.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6342e-05 2024-04-26T03:39:37.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:37.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.829573+0000", 2024-04-26T03:39:37.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8470000000000002e-06 2024-04-26T03:39:37.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:37.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.829651+0000", 2024-04-26T03:39:37.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7900999999999998e-05 2024-04-26T03:39:37.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:37.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.829661+0000", 2024-04-26T03:39:37.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9699999999999994e-06 2024-04-26T03:39:37.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:37.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.839616+0000", 2024-04-26T03:39:37.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.009954259 2024-04-26T03:39:37.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:37.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.839704+0000", 2024-04-26T03:39:37.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8380000000000004e-05 2024-04-26T03:39:37.131 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.131 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:37.131 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.131 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.fs2 65/62 ECSubWrite(tid=179, reqid=client.4556.0:783, at_version=65'61, trim_to=0'0, roll_forward_to=65'59))", 2024-04-26T03:39:37.131 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:20.834350+0000", 2024-04-26T03:39:37.131 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 16.212073484000001, 2024-04-26T03:39:37.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.007406719, 2024-04-26T03:39:37.132 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:37.132 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:37.132 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:37.132 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:37.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:37.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.834350+0000", 2024-04-26T03:39:37.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:37.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.834350+0000", 2024-04-26T03:39:37.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:37.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.834351+0000", 2024-04-26T03:39:37.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8700000000000004e-07 2024-04-26T03:39:37.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:37.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.834873+0000", 2024-04-26T03:39:37.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00052216800000000005 2024-04-26T03:39:37.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:37.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.834890+0000", 2024-04-26T03:39:37.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6980999999999999e-05 2024-04-26T03:39:37.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:37.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.834899+0000", 2024-04-26T03:39:37.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5279999999999992e-06 2024-04-26T03:39:37.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:37.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.834970+0000", 2024-04-26T03:39:37.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1066e-05 2024-04-26T03:39:37.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:37.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.834981+0000", 2024-04-26T03:39:37.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0234999999999999e-05 2024-04-26T03:39:37.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:37.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.841717+0000", 2024-04-26T03:39:37.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0067359059999999998 2024-04-26T03:39:37.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:37.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.841756+0000", 2024-04-26T03:39:37.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9848000000000002e-05 2024-04-26T03:39:37.135 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.135 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:37.136 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.136 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.8s6 80/63 ECSubWrite(tid=321, reqid=client.4556.0:897, at_version=80'111, trim_to=0'0, roll_forward_to=80'110))", 2024-04-26T03:39:37.136 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:35.716891+0000", 2024-04-26T03:39:37.136 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 1.3295325789999999, 2024-04-26T03:39:37.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019470023999999999, 2024-04-26T03:39:37.136 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:37.136 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:37.136 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:37.136 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:37.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:37.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:35.716891+0000", 2024-04-26T03:39:37.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:37.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:35.716891+0000", 2024-04-26T03:39:37.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:37.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:35.716893+0000", 2024-04-26T03:39:37.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.007e-06 2024-04-26T03:39:37.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:37.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:35.719882+0000", 2024-04-26T03:39:37.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0029891399999999999 2024-04-26T03:39:37.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:37.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:35.719895+0000", 2024-04-26T03:39:37.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2921999999999999e-05 2024-04-26T03:39:37.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:37.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:35.719904+0000", 2024-04-26T03:39:37.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9290000000000004e-06 2024-04-26T03:39:37.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:37.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:35.719976+0000", 2024-04-26T03:39:37.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2758999999999999e-05 2024-04-26T03:39:37.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:37.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:35.719988+0000", 2024-04-26T03:39:37.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2104e-05 2024-04-26T03:39:37.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:37.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:35.736324+0000", 2024-04-26T03:39:37.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.016335682000000001 2024-04-26T03:39:37.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:37.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:35.736361+0000", 2024-04-26T03:39:37.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6480999999999999e-05 2024-04-26T03:39:37.140 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.140 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:37.140 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.140 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.140 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:37.140 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-26T03:39:37.142 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-04-26T03:39:37.143 INFO:tasks.ceph.osd.8.smithi028.stderr:2024-04-26T03:39:37.140+0000 7f5bb75d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 31968) UID: 0 2024-04-26T03:39:37.243 INFO:tasks.ceph.osd.15.smithi092.stderr:2024-04-26T03:39:37.241+0000 7f6d12dec640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 31824) UID: 0 2024-04-26T03:39:37.250 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-26T03:39:37.250 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-04-26T03:39:37.250 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-04-26T03:39:37.251 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-26T03:39:37.259 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-04-26T03:39:37.344 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-04-26T03:39:37.343+0000 7fee562fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31759) UID: 0 2024-04-26T03:39:37.366 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-26T03:39:37.367 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-04-26T03:39:37.367 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-04-26T03:39:37.367 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-04-26T03:39:37.367 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-26T03:39:37.371 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-26T03:39:37.375 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-04-26T03:39:37.445 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-04-26T03:39:37.443+0000 7fe9223c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31954) UID: 0 2024-04-26T03:39:37.483 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-26T03:39:37.483 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-04-26T03:39:37.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-04-26T03:39:37.483 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-04-26T03:39:37.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.483 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds2 35/32 ECSubWrite(tid=0, reqid=client.4556.0:43, at_version=35'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:39:37.483 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.501816+0000", 2024-04-26T03:39:37.484 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.979084284000002, 2024-04-26T03:39:37.484 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0091953699999999996, 2024-04-26T03:39:37.484 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:37.484 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:37.484 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:37.484 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:37.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.484 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:37.484 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.501816+0000", 2024-04-26T03:39:37.484 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.484 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:37.484 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.501816+0000", 2024-04-26T03:39:37.485 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.485 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.485 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:37.485 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.501816+0000", 2024-04-26T03:39:37.485 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8099999999999998e-07 2024-04-26T03:39:37.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.485 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.485 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:37.485 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.502179+0000", 2024-04-26T03:39:37.485 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036287299999999999 2024-04-26T03:39:37.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.485 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.485 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:37.485 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.502197+0000", 2024-04-26T03:39:37.486 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7765e-05 2024-04-26T03:39:37.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.486 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.486 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:37.486 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.502207+0000", 2024-04-26T03:39:37.486 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6190000000000006e-06 2024-04-26T03:39:37.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.486 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.486 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:37.486 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.502306+0000", 2024-04-26T03:39:37.486 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9655999999999996e-05 2024-04-26T03:39:37.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.486 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.486 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:37.487 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.502315+0000", 2024-04-26T03:39:37.487 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5629999999999992e-06 2024-04-26T03:39:37.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.487 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:37.487 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.510976+0000", 2024-04-26T03:39:37.487 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0086612379999999999 2024-04-26T03:39:37.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.487 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:37.487 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.511011+0000", 2024-04-26T03:39:37.487 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5274999999999997e-05 2024-04-26T03:39:37.487 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.487 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:37.487 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.488 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.488 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds2 35/32 ECSubWrite(tid=1, reqid=client.4556.0:44, at_version=35'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:39:37.488 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.504658+0000", 2024-04-26T03:39:37.488 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.976241967999997, 2024-04-26T03:39:37.488 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0063889319999999999, 2024-04-26T03:39:37.488 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:37.488 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:37.488 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:37.488 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:37.488 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.488 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:37.488 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504658+0000", 2024-04-26T03:39:37.488 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.489 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.489 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:37.489 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504658+0000", 2024-04-26T03:39:37.489 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.489 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.489 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:37.489 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504659+0000", 2024-04-26T03:39:37.489 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3399999999999997e-07 2024-04-26T03:39:37.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.489 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.489 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:37.489 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504681+0000", 2024-04-26T03:39:37.489 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1452999999999999e-05 2024-04-26T03:39:37.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.490 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.490 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:37.490 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504700+0000", 2024-04-26T03:39:37.490 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9772999999999999e-05 2024-04-26T03:39:37.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.490 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.490 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:37.490 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.504710+0000", 2024-04-26T03:39:37.490 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2660000000000007e-06 2024-04-26T03:39:37.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.490 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.490 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:37.490 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.505327+0000", 2024-04-26T03:39:37.490 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00061779000000000001 2024-04-26T03:39:37.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.491 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:37.491 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.505339+0000", 2024-04-26T03:39:37.491 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.147e-05 2024-04-26T03:39:37.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.491 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:37.491 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.511016+0000", 2024-04-26T03:39:37.491 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0056775439999999996 2024-04-26T03:39:37.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.491 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:37.491 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.511047+0000", 2024-04-26T03:39:37.491 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0701999999999998e-05 2024-04-26T03:39:37.492 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.492 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:37.492 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.492 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.492 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.9s3 35/32 ECSubWrite(tid=10, reqid=client.4556.0:76, at_version=35'3, trim_to=0'0, roll_forward_to=35'2))", 2024-04-26T03:39:37.492 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.530631+0000", 2024-04-26T03:39:37.492 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.950269145999997, 2024-04-26T03:39:37.492 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0058993190000000001, 2024-04-26T03:39:37.492 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:37.492 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:37.492 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:37.492 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:37.492 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.493 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:37.493 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.530631+0000", 2024-04-26T03:39:37.493 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.493 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.493 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:37.493 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.530631+0000", 2024-04-26T03:39:37.493 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.493 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.493 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:37.493 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.530632+0000", 2024-04-26T03:39:37.493 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3900000000000004e-07 2024-04-26T03:39:37.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.493 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.494 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:37.494 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.531212+0000", 2024-04-26T03:39:37.494 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000579979 2024-04-26T03:39:37.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.494 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:37.494 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.531238+0000", 2024-04-26T03:39:37.494 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5927000000000001e-05 2024-04-26T03:39:37.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.494 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:37.494 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.531247+0000", 2024-04-26T03:39:37.494 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8519999999999993e-06 2024-04-26T03:39:37.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.495 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:37.495 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.531314+0000", 2024-04-26T03:39:37.495 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7070000000000007e-05 2024-04-26T03:39:37.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.495 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:37.495 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.531322+0000", 2024-04-26T03:39:37.495 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6859999999999995e-06 2024-04-26T03:39:37.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.495 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:37.495 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.536494+0000", 2024-04-26T03:39:37.495 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0051716319999999998 2024-04-26T03:39:37.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.496 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:37.496 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.536530+0000", 2024-04-26T03:39:37.496 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6334000000000001e-05 2024-04-26T03:39:37.496 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.496 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:37.496 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.496 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.2s3 35/32 ECSubWrite(tid=8, reqid=client.4556.0:139, at_version=35'5, trim_to=0'0, roll_forward_to=35'4))", 2024-04-26T03:39:37.496 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:51.621036+0000", 2024-04-26T03:39:37.496 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.859864432000002, 2024-04-26T03:39:37.496 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0056652040000000001, 2024-04-26T03:39:37.496 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:37.497 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:37.497 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:37.497 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:37.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.497 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:37.497 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.621036+0000", 2024-04-26T03:39:37.497 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.497 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:37.497 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.621036+0000", 2024-04-26T03:39:37.497 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:37.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.621037+0000", 2024-04-26T03:39:37.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.78e-07 2024-04-26T03:39:37.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:37.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.621467+0000", 2024-04-26T03:39:37.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00043015799999999998 2024-04-26T03:39:37.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:37.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.621481+0000", 2024-04-26T03:39:37.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3978999999999999e-05 2024-04-26T03:39:37.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.499 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.499 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:37.499 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.621491+0000", 2024-04-26T03:39:37.499 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7010000000000008e-06 2024-04-26T03:39:37.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.499 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.499 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:37.499 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.621571+0000", 2024-04-26T03:39:37.499 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0408000000000007e-05 2024-04-26T03:39:37.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.499 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.499 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:37.499 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.621581+0000", 2024-04-26T03:39:37.499 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0079e-05 2024-04-26T03:39:37.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.500 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:37.500 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.626670+0000", 2024-04-26T03:39:37.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0050891920000000002 2024-04-26T03:39:37.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.500 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:37.500 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:51.626701+0000", 2024-04-26T03:39:37.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0809000000000003e-05 2024-04-26T03:39:37.500 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.500 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:37.500 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.500 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.es5 43/32 ECSubWrite(tid=49, reqid=client.4556.0:302, at_version=43'17, trim_to=0'0, roll_forward_to=35'14))", 2024-04-26T03:39:37.501 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:38:59.537163+0000", 2024-04-26T03:39:37.501 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.943736858999998, 2024-04-26T03:39:37.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0055244370000000001, 2024-04-26T03:39:37.501 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:37.501 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:37.501 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:37.501 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:37.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:37.501 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.537163+0000", 2024-04-26T03:39:37.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:37.501 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.537163+0000", 2024-04-26T03:39:37.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.502 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:37.502 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.537165+0000", 2024-04-26T03:39:37.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7039999999999999e-06 2024-04-26T03:39:37.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.502 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:37.502 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.537336+0000", 2024-04-26T03:39:37.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000170619 2024-04-26T03:39:37.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.502 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:37.503 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.537352+0000", 2024-04-26T03:39:37.503 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5974999999999999e-05 2024-04-26T03:39:37.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.503 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:37.503 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.537361+0000", 2024-04-26T03:39:37.503 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2490000000000007e-06 2024-04-26T03:39:37.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.503 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:37.503 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.537448+0000", 2024-04-26T03:39:37.503 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7148e-05 2024-04-26T03:39:37.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.504 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:37.504 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.537457+0000", 2024-04-26T03:39:37.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6500000000000002e-06 2024-04-26T03:39:37.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.504 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:37.504 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.542650+0000", 2024-04-26T03:39:37.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0051930580000000004 2024-04-26T03:39:37.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.504 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:37.504 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:38:59.542688+0000", 2024-04-26T03:39:37.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8034000000000002e-05 2024-04-26T03:39:37.504 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.504 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:37.505 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.505 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.2s3 49/44 ECSubWrite(tid=32, reqid=client.4556.0:433, at_version=49'20, trim_to=0'0, roll_forward_to=48'19))", 2024-04-26T03:39:37.505 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:04.700091+0000", 2024-04-26T03:39:37.505 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.780809376999997, 2024-04-26T03:39:37.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0091986069999999993, 2024-04-26T03:39:37.505 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:37.505 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:37.505 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:37.505 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:37.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.505 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:37.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.700091+0000", 2024-04-26T03:39:37.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:37.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.700091+0000", 2024-04-26T03:39:37.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:37.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.700091+0000", 2024-04-26T03:39:37.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1600000000000001e-07 2024-04-26T03:39:37.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:37.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.700840+0000", 2024-04-26T03:39:37.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00074861199999999995 2024-04-26T03:39:37.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.507 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:37.507 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.700854+0000", 2024-04-26T03:39:37.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3696999999999999e-05 2024-04-26T03:39:37.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.507 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:37.507 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.700863+0000", 2024-04-26T03:39:37.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2969999999999992e-06 2024-04-26T03:39:37.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.507 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:37.508 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.700940+0000", 2024-04-26T03:39:37.508 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7356999999999998e-05 2024-04-26T03:39:37.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.508 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.508 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:37.508 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.700950+0000", 2024-04-26T03:39:37.508 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7340000000000003e-06 2024-04-26T03:39:37.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.508 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.508 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:37.508 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.709245+0000", 2024-04-26T03:39:37.508 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.008294582 2024-04-26T03:39:37.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.508 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.508 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:37.509 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.709289+0000", 2024-04-26T03:39:37.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4811999999999997e-05 2024-04-26T03:39:37.509 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.509 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:37.509 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.509 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.2s3 49/44 ECSubWrite(tid=34, reqid=client.4556.0:434, at_version=49'23, trim_to=0'0, roll_forward_to=48'19))", 2024-04-26T03:39:37.509 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:04.712922+0000", 2024-04-26T03:39:37.509 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.767978434, 2024-04-26T03:39:37.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0059780120000000004, 2024-04-26T03:39:37.509 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:37.509 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:37.509 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:37.509 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:37.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:37.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.712922+0000", 2024-04-26T03:39:37.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:37.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.712922+0000", 2024-04-26T03:39:37.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:37.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.712923+0000", 2024-04-26T03:39:37.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.237e-06 2024-04-26T03:39:37.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:37.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.713234+0000", 2024-04-26T03:39:37.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00031075200000000002 2024-04-26T03:39:37.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:37.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.713251+0000", 2024-04-26T03:39:37.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6976000000000001e-05 2024-04-26T03:39:37.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:37.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.713260+0000", 2024-04-26T03:39:37.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0119999999999994e-06 2024-04-26T03:39:37.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.512 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:37.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.713314+0000", 2024-04-26T03:39:37.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3947999999999999e-05 2024-04-26T03:39:37.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.512 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:37.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.713320+0000", 2024-04-26T03:39:37.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9989999999999997e-06 2024-04-26T03:39:37.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.512 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:37.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.718862+0000", 2024-04-26T03:39:37.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0055423080000000001 2024-04-26T03:39:37.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.513 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:37.513 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:04.718900+0000", 2024-04-26T03:39:37.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7780000000000001e-05 2024-04-26T03:39:37.513 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.513 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:37.513 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.513 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.2s3 50/44 ECSubWrite(tid=42, reqid=client.4556.0:478, at_version=50'28, trim_to=0'0, roll_forward_to=50'27))", 2024-04-26T03:39:37.513 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:05.912137+0000", 2024-04-26T03:39:37.513 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.568763247, 2024-04-26T03:39:37.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0056262439999999999, 2024-04-26T03:39:37.514 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:37.514 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:37.514 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:37.514 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:37.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:37.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.912137+0000", 2024-04-26T03:39:37.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:37.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.912137+0000", 2024-04-26T03:39:37.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:37.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.912138+0000", 2024-04-26T03:39:37.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0550000000000001e-06 2024-04-26T03:39:37.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:37.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.912745+0000", 2024-04-26T03:39:37.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000606552 2024-04-26T03:39:37.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:37.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.912756+0000", 2024-04-26T03:39:37.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1019000000000001e-05 2024-04-26T03:39:37.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:37.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.912763+0000", 2024-04-26T03:39:37.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3950000000000003e-06 2024-04-26T03:39:37.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:37.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.912834+0000", 2024-04-26T03:39:37.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1049000000000004e-05 2024-04-26T03:39:37.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:37.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.912846+0000", 2024-04-26T03:39:37.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2340999999999999e-05 2024-04-26T03:39:37.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:37.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.917730+0000", 2024-04-26T03:39:37.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0048834259999999997 2024-04-26T03:39:37.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:37.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:05.917763+0000", 2024-04-26T03:39:37.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3407e-05 2024-04-26T03:39:37.517 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.517 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:37.517 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.517 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds2 56/32 ECSubWrite(tid=28, reqid=client.4556.0:607, at_version=56'25, trim_to=0'0, roll_forward_to=50'23))", 2024-04-26T03:39:37.518 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:11.724660+0000", 2024-04-26T03:39:37.518 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.756239785999998, 2024-04-26T03:39:37.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.01307469, 2024-04-26T03:39:37.518 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:37.518 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:37.518 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:37.518 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:37.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:37.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.724660+0000", 2024-04-26T03:39:37.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:37.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.724660+0000", 2024-04-26T03:39:37.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:37.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.724661+0000", 2024-04-26T03:39:37.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9999999999999997e-07 2024-04-26T03:39:37.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:37.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.725284+0000", 2024-04-26T03:39:37.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00062266999999999997 2024-04-26T03:39:37.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:37.520 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.725299+0000", 2024-04-26T03:39:37.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4849e-05 2024-04-26T03:39:37.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:37.520 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.725309+0000", 2024-04-26T03:39:37.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9809999999999998e-06 2024-04-26T03:39:37.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:37.520 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.725390+0000", 2024-04-26T03:39:37.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1000000000000004e-05 2024-04-26T03:39:37.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:37.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.725402+0000", 2024-04-26T03:39:37.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2622e-05 2024-04-26T03:39:37.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:37.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.737637+0000", 2024-04-26T03:39:37.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012234681000000001 2024-04-26T03:39:37.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:37.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.737735+0000", 2024-04-26T03:39:37.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8287000000000003e-05 2024-04-26T03:39:37.521 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.521 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:37.521 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.522 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds2 56/32 ECSubWrite(tid=29, reqid=client.4556.0:608, at_version=56'26, trim_to=0'0, roll_forward_to=50'23))", 2024-04-26T03:39:37.522 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:11.732163+0000", 2024-04-26T03:39:37.522 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.748737240000001, 2024-04-26T03:39:37.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010518913, 2024-04-26T03:39:37.522 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:37.522 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:37.522 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:37.522 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:37.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:37.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.732163+0000", 2024-04-26T03:39:37.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:37.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.732163+0000", 2024-04-26T03:39:37.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:37.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.732164+0000", 2024-04-26T03:39:37.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7300000000000004e-07 2024-04-26T03:39:37.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:37.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.732418+0000", 2024-04-26T03:39:37.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025363600000000001 2024-04-26T03:39:37.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:37.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.732432+0000", 2024-04-26T03:39:37.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.49e-05 2024-04-26T03:39:37.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:37.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.732443+0000", 2024-04-26T03:39:37.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0351e-05 2024-04-26T03:39:37.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:37.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.733300+0000", 2024-04-26T03:39:37.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00085717599999999997 2024-04-26T03:39:37.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:37.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.733310+0000", 2024-04-26T03:39:37.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0210999999999999e-05 2024-04-26T03:39:37.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:37.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.742622+0000", 2024-04-26T03:39:37.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0093113480000000005 2024-04-26T03:39:37.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:37.526 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.742682+0000", 2024-04-26T03:39:37.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0318000000000001e-05 2024-04-26T03:39:37.526 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.526 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:37.526 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.526 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds2 56/32 ECSubWrite(tid=30, reqid=client.4556.0:609, at_version=56'27, trim_to=0'0, roll_forward_to=50'23))", 2024-04-26T03:39:37.526 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:11.737685+0000", 2024-04-26T03:39:37.526 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.743214749, 2024-04-26T03:39:37.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0057102849999999998, 2024-04-26T03:39:37.526 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:37.526 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:37.526 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:37.527 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:37.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:37.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.737685+0000", 2024-04-26T03:39:37.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:37.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.737685+0000", 2024-04-26T03:39:37.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:37.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.737686+0000", 2024-04-26T03:39:37.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8400000000000004e-07 2024-04-26T03:39:37.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:37.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.737927+0000", 2024-04-26T03:39:37.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000241215 2024-04-26T03:39:37.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:37.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.737943+0000", 2024-04-26T03:39:37.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5563999999999999e-05 2024-04-26T03:39:37.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:37.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.737952+0000", 2024-04-26T03:39:37.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2909999999999996e-06 2024-04-26T03:39:37.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:37.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.738028+0000", 2024-04-26T03:39:37.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6087000000000005e-05 2024-04-26T03:39:37.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:37.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.738039+0000", 2024-04-26T03:39:37.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0864e-05 2024-04-26T03:39:37.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:37.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.743355+0000", 2024-04-26T03:39:37.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0053161700000000003 2024-04-26T03:39:37.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:37.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:11.743396+0000", 2024-04-26T03:39:37.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0410000000000001e-05 2024-04-26T03:39:37.530 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.530 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:37.530 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.530 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds2 60/32 ECSubWrite(tid=45, reqid=client.4556.0:691, at_version=60'43, trim_to=0'0, roll_forward_to=60'41))", 2024-04-26T03:39:37.530 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:15.954667+0000", 2024-04-26T03:39:37.531 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 21.52623371, 2024-04-26T03:39:37.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0056884680000000003, 2024-04-26T03:39:37.531 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:37.531 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:37.531 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:37.531 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:37.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:37.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.954667+0000", 2024-04-26T03:39:37.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:37.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.954667+0000", 2024-04-26T03:39:37.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:37.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.954668+0000", 2024-04-26T03:39:37.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.26e-06 2024-04-26T03:39:37.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:37.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.954882+0000", 2024-04-26T03:39:37.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021437299999999999 2024-04-26T03:39:37.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:37.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.954897+0000", 2024-04-26T03:39:37.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5214999999999999e-05 2024-04-26T03:39:37.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:37.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.954906+0000", 2024-04-26T03:39:37.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7919999999999998e-06 2024-04-26T03:39:37.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:37.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.954968+0000", 2024-04-26T03:39:37.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1661999999999993e-05 2024-04-26T03:39:37.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:37.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.954974+0000", 2024-04-26T03:39:37.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1319999999999999e-06 2024-04-26T03:39:37.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:37.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.960325+0000", 2024-04-26T03:39:37.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0053511990000000001 2024-04-26T03:39:37.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:37.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.960355+0000", 2024-04-26T03:39:37.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9835000000000001e-05 2024-04-26T03:39:37.534 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.535 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:37.535 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.535 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.10s1 60/48 ECSubWrite(tid=272, reqid=client.4556.0:703, at_version=60'20, trim_to=0'0, roll_forward_to=60'18))", 2024-04-26T03:39:37.535 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:15.958198+0000", 2024-04-26T03:39:37.535 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 21.522702327000001, 2024-04-26T03:39:37.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0082246890000000003, 2024-04-26T03:39:37.535 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:37.535 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:37.535 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:37.535 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:37.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:37.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.958198+0000", 2024-04-26T03:39:37.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:37.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.958198+0000", 2024-04-26T03:39:37.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:37.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.958199+0000", 2024-04-26T03:39:37.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3e-07 2024-04-26T03:39:37.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:37.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.959083+0000", 2024-04-26T03:39:37.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00088405799999999996 2024-04-26T03:39:37.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:37.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.959097+0000", 2024-04-26T03:39:37.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.456e-05 2024-04-26T03:39:37.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:37.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.959106+0000", 2024-04-26T03:39:37.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3210000000000005e-06 2024-04-26T03:39:37.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:37.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.959164+0000", 2024-04-26T03:39:37.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8700999999999999e-05 2024-04-26T03:39:37.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:37.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.959172+0000", 2024-04-26T03:39:37.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.374e-06 2024-04-26T03:39:37.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:37.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.966393+0000", 2024-04-26T03:39:37.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0072211209999999996 2024-04-26T03:39:37.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:37.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:15.966423+0000", 2024-04-26T03:39:37.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9824000000000001e-05 2024-04-26T03:39:37.539 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.539 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:37.539 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.539 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.es5 65/32 ECSubWrite(tid=127, reqid=client.4556.0:768, at_version=65'50, trim_to=0'0, roll_forward_to=63'48))", 2024-04-26T03:39:37.539 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:20.764878+0000", 2024-04-26T03:39:37.539 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 16.716022504000001, 2024-04-26T03:39:37.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0082696509999999994, 2024-04-26T03:39:37.539 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:37.539 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:37.540 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:37.540 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:37.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:37.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.764878+0000", 2024-04-26T03:39:37.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:37.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.764878+0000", 2024-04-26T03:39:37.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:37.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.764878+0000", 2024-04-26T03:39:37.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1600000000000002e-07 2024-04-26T03:39:37.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:37.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.765111+0000", 2024-04-26T03:39:37.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000232945 2024-04-26T03:39:37.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:37.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.765128+0000", 2024-04-26T03:39:37.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7206000000000001e-05 2024-04-26T03:39:37.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:37.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.765140+0000", 2024-04-26T03:39:37.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1592e-05 2024-04-26T03:39:37.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:37.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.765220+0000", 2024-04-26T03:39:37.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0502999999999998e-05 2024-04-26T03:39:37.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:37.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.765232+0000", 2024-04-26T03:39:37.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1824999999999999e-05 2024-04-26T03:39:37.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:37.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.773109+0000", 2024-04-26T03:39:37.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0078771999999999991 2024-04-26T03:39:37.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:37.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.773147+0000", 2024-04-26T03:39:37.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8064e-05 2024-04-26T03:39:37.543 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.543 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:37.543 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.543 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.es5 65/32 ECSubWrite(tid=129, reqid=client.4556.0:770, at_version=65'52, trim_to=0'0, roll_forward_to=65'50))", 2024-04-26T03:39:37.543 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:20.779267+0000", 2024-04-26T03:39:37.544 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 16.701632994000001, 2024-04-26T03:39:37.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0053205700000000002, 2024-04-26T03:39:37.544 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:37.544 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:37.544 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:37.544 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:37.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:37.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.779267+0000", 2024-04-26T03:39:37.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:37.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.779267+0000", 2024-04-26T03:39:37.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:37.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.779268+0000", 2024-04-26T03:39:37.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2799999999999995e-07 2024-04-26T03:39:37.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:37.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.779379+0000", 2024-04-26T03:39:37.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011136 2024-04-26T03:39:37.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:37.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.779400+0000", 2024-04-26T03:39:37.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0925e-05 2024-04-26T03:39:37.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:37.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.779410+0000", 2024-04-26T03:39:37.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1740000000000006e-06 2024-04-26T03:39:37.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:37.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.779464+0000", 2024-04-26T03:39:37.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4478999999999998e-05 2024-04-26T03:39:37.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:37.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.779470+0000", 2024-04-26T03:39:37.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9909999999999999e-06 2024-04-26T03:39:37.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:37.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.784547+0000", 2024-04-26T03:39:37.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0050772179999999997 2024-04-26T03:39:37.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:37.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.784588+0000", 2024-04-26T03:39:37.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0595e-05 2024-04-26T03:39:37.547 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.547 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:37.548 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.548 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds2 65/32 ECSubWrite(tid=53, reqid=client.4556.0:774, at_version=65'51, trim_to=0'0, roll_forward_to=63'49))", 2024-04-26T03:39:37.548 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:20.782425+0000", 2024-04-26T03:39:37.548 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 16.698474821000001, 2024-04-26T03:39:37.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.014125627999999999, 2024-04-26T03:39:37.548 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:37.548 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:37.548 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:37.548 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:37.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:37.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.782425+0000", 2024-04-26T03:39:37.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:37.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.782425+0000", 2024-04-26T03:39:37.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:37.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.782426+0000", 2024-04-26T03:39:37.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3999999999999997e-07 2024-04-26T03:39:37.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:37.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.782965+0000", 2024-04-26T03:39:37.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00053963399999999999 2024-04-26T03:39:37.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:37.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.782978+0000", 2024-04-26T03:39:37.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2744e-05 2024-04-26T03:39:37.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:37.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.782986+0000", 2024-04-26T03:39:37.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0700000000000007e-06 2024-04-26T03:39:37.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:37.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.783062+0000", 2024-04-26T03:39:37.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5611999999999994e-05 2024-04-26T03:39:37.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:37.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.783072+0000", 2024-04-26T03:39:37.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0566e-05 2024-04-26T03:39:37.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:37.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.796494+0000", 2024-04-26T03:39:37.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013421196999999999 2024-04-26T03:39:37.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:37.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.796551+0000", 2024-04-26T03:39:37.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7465e-05 2024-04-26T03:39:37.552 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.552 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:37.552 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.552 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.ds2 65/32 ECSubWrite(tid=54, reqid=client.4556.0:775, at_version=65'52, trim_to=0'0, roll_forward_to=63'49))", 2024-04-26T03:39:37.552 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:20.790746+0000", 2024-04-26T03:39:37.552 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 16.690154419999999, 2024-04-26T03:39:37.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0091241550000000001, 2024-04-26T03:39:37.553 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:37.553 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:37.553 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:37.553 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:37.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:37.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.790746+0000", 2024-04-26T03:39:37.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:37.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.790746+0000", 2024-04-26T03:39:37.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:37.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.790747+0000", 2024-04-26T03:39:37.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7499999999999999e-07 2024-04-26T03:39:37.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:37.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.791058+0000", 2024-04-26T03:39:37.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00031094299999999999 2024-04-26T03:39:37.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:37.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.791069+0000", 2024-04-26T03:39:37.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.148e-05 2024-04-26T03:39:37.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:37.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.791077+0000", 2024-04-26T03:39:37.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5599999999999996e-06 2024-04-26T03:39:37.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:37.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.791168+0000", 2024-04-26T03:39:37.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1768999999999995e-05 2024-04-26T03:39:37.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:37.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.791179+0000", 2024-04-26T03:39:37.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0217999999999999e-05 2024-04-26T03:39:37.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:37.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.799834+0000", 2024-04-26T03:39:37.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0086549450000000007 2024-04-26T03:39:37.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:37.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.799870+0000", 2024-04-26T03:39:37.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6365000000000002e-05 2024-04-26T03:39:37.556 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.556 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:37.556 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.557 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.fs4 65/62 ECSubWrite(tid=177, reqid=client.4556.0:781, at_version=65'59, trim_to=0'0, roll_forward_to=64'57))", 2024-04-26T03:39:37.557 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:20.821171+0000", 2024-04-26T03:39:37.557 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 16.659729041999999, 2024-04-26T03:39:37.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.006859904, 2024-04-26T03:39:37.557 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:37.557 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:37.557 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:37.557 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:37.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:37.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.821171+0000", 2024-04-26T03:39:37.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:37.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.821171+0000", 2024-04-26T03:39:37.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:37.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.821172+0000", 2024-04-26T03:39:37.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8299999999999996e-07 2024-04-26T03:39:37.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:37.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.821844+0000", 2024-04-26T03:39:37.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00067213600000000004 2024-04-26T03:39:37.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:37.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.821856+0000", 2024-04-26T03:39:37.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1956e-05 2024-04-26T03:39:37.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:37.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.821862+0000", 2024-04-26T03:39:37.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5940000000000001e-06 2024-04-26T03:39:37.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:37.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.821939+0000", 2024-04-26T03:39:37.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7743000000000004e-05 2024-04-26T03:39:37.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:37.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.821952+0000", 2024-04-26T03:39:37.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.273e-05 2024-04-26T03:39:37.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:37.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.827975+0000", 2024-04-26T03:39:37.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00602295 2024-04-26T03:39:37.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:37.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.828031+0000", 2024-04-26T03:39:37.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6112000000000001e-05 2024-04-26T03:39:37.561 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.561 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:37.561 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.561 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.fs4 65/62 ECSubWrite(tid=179, reqid=client.4556.0:783, at_version=65'61, trim_to=0'0, roll_forward_to=65'59))", 2024-04-26T03:39:37.561 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:20.833720+0000", 2024-04-26T03:39:37.561 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 16.647180586000001, 2024-04-26T03:39:37.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0083247359999999992, 2024-04-26T03:39:37.561 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:37.561 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:37.561 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:37.561 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:37.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:37.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.833720+0000", 2024-04-26T03:39:37.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:37.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.833720+0000", 2024-04-26T03:39:37.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:37.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.833720+0000", 2024-04-26T03:39:37.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5000000000000002e-07 2024-04-26T03:39:37.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:37.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.834354+0000", 2024-04-26T03:39:37.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00063369900000000005 2024-04-26T03:39:37.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:37.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.834366+0000", 2024-04-26T03:39:37.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2087e-05 2024-04-26T03:39:37.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:37.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.834377+0000", 2024-04-26T03:39:37.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0579e-05 2024-04-26T03:39:37.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:37.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.834467+0000", 2024-04-26T03:39:37.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0334999999999995e-05 2024-04-26T03:39:37.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:37.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.834479+0000", 2024-04-26T03:39:37.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1685000000000001e-05 2024-04-26T03:39:37.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:37.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.841984+0000", 2024-04-26T03:39:37.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0075050999999999998 2024-04-26T03:39:37.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:37.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:20.842044+0000", 2024-04-26T03:39:37.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0600999999999998e-05 2024-04-26T03:39:37.565 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.565 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:37.565 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.565 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDECSubOpWrite(3.10s1 80/48 ECSubWrite(tid=322, reqid=client.4556.0:895, at_version=80'35, trim_to=0'0, roll_forward_to=75'33))", 2024-04-26T03:39:37.565 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-04-26T03:39:35.740173+0000", 2024-04-26T03:39:37.565 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 1.7407275250000001, 2024-04-26T03:39:37.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026535494999999999, 2024-04-26T03:39:37.566 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-04-26T03:39:37.566 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-04-26T03:39:37.566 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-04-26T03:39:37.566 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-04-26T03:39:37.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-04-26T03:39:37.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:35.740173+0000", 2024-04-26T03:39:37.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-04-26T03:39:37.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:35.740173+0000", 2024-04-26T03:39:37.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-04-26T03:39:37.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-04-26T03:39:37.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:35.740173+0000", 2024-04-26T03:39:37.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5699999999999999e-07 2024-04-26T03:39:37.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-04-26T03:39:37.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:35.743640+0000", 2024-04-26T03:39:37.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003466659 2024-04-26T03:39:37.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-04-26T03:39:37.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:35.743653+0000", 2024-04-26T03:39:37.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3114e-05 2024-04-26T03:39:37.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-04-26T03:39:37.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:35.743660+0000", 2024-04-26T03:39:37.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3549999999999999e-06 2024-04-26T03:39:37.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-04-26T03:39:37.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:35.743717+0000", 2024-04-26T03:39:37.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7011e-05 2024-04-26T03:39:37.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_started", 2024-04-26T03:39:37.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:35.743725+0000", 2024-04-26T03:39:37.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4149999999999999e-06 2024-04-26T03:39:37.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_committed", 2024-04-26T03:39:37.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:35.766674+0000", 2024-04-26T03:39:37.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.022948567999999999 2024-04-26T03:39:37.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-26T03:39:37.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-26T03:39:37.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-04-26T03:39:37.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-04-26T03:39:35.766708+0000", 2024-04-26T03:39:37.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4215999999999999e-05 2024-04-26T03:39:37.569 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.569 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:37.569 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.569 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-26T03:39:37.569 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-26T03:39:37.569 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-26T03:39:37.571 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0: {} 2024-04-26T03:39:37.571 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1: {} 2024-04-26T03:39:37.571 INFO:teuthology.orchestra.run.smithi028.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T03:39:37.572 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2: {} 2024-04-26T03:39:37.572 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-04-26T03:39:37.573 INFO:teuthology.orchestra.run.smithi028.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T03:39:37.573 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3: {} 2024-04-26T03:39:37.573 INFO:teuthology.orchestra.run.smithi028.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T03:39:37.574 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4: {} 2024-04-26T03:39:37.574 INFO:teuthology.orchestra.run.smithi028.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T03:39:37.574 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:39:37.570+0000 7f60d3c4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31757) UID: 0 2024-04-26T03:39:37.589 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5: {} 2024-04-26T03:39:37.589 INFO:teuthology.orchestra.run.smithi028.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T03:39:37.608 INFO:teuthology.orchestra.run.smithi028.stdout:osd.6: {} 2024-04-26T03:39:37.608 INFO:teuthology.orchestra.run.smithi028.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T03:39:37.625 INFO:teuthology.orchestra.run.smithi028.stdout:osd.7: {} 2024-04-26T03:39:37.625 INFO:teuthology.orchestra.run.smithi028.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T03:39:37.642 INFO:teuthology.orchestra.run.smithi028.stdout:osd.8: {} 2024-04-26T03:39:37.642 INFO:teuthology.orchestra.run.smithi028.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T03:39:37.659 INFO:teuthology.orchestra.run.smithi028.stdout:osd.9: {} 2024-04-26T03:39:37.659 INFO:teuthology.orchestra.run.smithi028.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T03:39:37.672 INFO:tasks.ceph.osd.14.smithi066.stderr:2024-04-26T03:39:37.671+0000 7f5395f08640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 31765) UID: 0 2024-04-26T03:39:37.678 INFO:teuthology.orchestra.run.smithi028.stdout:osd.10: {} 2024-04-26T03:39:37.678 INFO:teuthology.orchestra.run.smithi028.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T03:39:37.693 INFO:teuthology.orchestra.run.smithi028.stderr:reweighted osd.9 to 1 (10000) 2024-04-26T03:39:37.697 INFO:tasks.rados.rados.0.smithi092.stdout:356: done (1 left) 2024-04-26T03:39:37.698 INFO:tasks.rados.rados.0.smithi092.stdout:357: done (0 left) 2024-04-26T03:39:37.698 INFO:tasks.rados.rados.0.smithi092.stdout:358: snap_create 2024-04-26T03:39:37.698 INFO:teuthology.orchestra.run.smithi028.stdout:osd.11: {} 2024-04-26T03:39:37.698 INFO:teuthology.orchestra.run.smithi028.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T03:39:37.708 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 10 1 2024-04-26T03:39:37.712 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-26T03:39:37.717 INFO:teuthology.orchestra.run.smithi028.stdout:osd.12: {} 2024-04-26T03:39:37.718 INFO:teuthology.orchestra.run.smithi028.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T03:39:37.722 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-26T03:39:37.722 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-04-26T03:39:37.722 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-04-26T03:39:37.722 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-26T03:39:37.731 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-04-26T03:39:37.736 INFO:teuthology.orchestra.run.smithi028.stdout:osd.13: {} 2024-04-26T03:39:37.736 INFO:teuthology.orchestra.run.smithi028.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T03:39:37.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:39:37.735+0000 7f1266ffd640 0 -- 172.21.15.28:0/2379546724 >> [v2:172.21.15.66:6824/1534905602,v1:172.21.15.66:6825/1534905602] conn(0x7f126821d530 msgr2=0x7f126821f920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-26T03:39:37.755 INFO:teuthology.orchestra.run.smithi028.stdout:osd.14: {} 2024-04-26T03:39:37.755 INFO:teuthology.orchestra.run.smithi028.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T03:39:37.773 INFO:teuthology.orchestra.run.smithi028.stdout:osd.15: {} 2024-04-26T03:39:37.773 INFO:teuthology.orchestra.run.smithi028.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T03:39:37.774 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-04-26T03:39:37.772+0000 7f745118d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 31821) UID: 0 2024-04-26T03:39:37.838 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:37.838 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:37.847 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-04-26T03:39:37.874 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-04-26T03:39:37.873+0000 7f745118d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 31821) UID: 0 2024-04-26T03:39:37.954 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:37.954 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:37.962 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-26T03:39:38.007 INFO:tasks.ceph.osd.14.smithi066.stderr:2024-04-26T03:39:38.006+0000 7f5395f08640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 31765) UID: 0 2024-04-26T03:39:38.077 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:39:38.076+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:39:38.082 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:38.082 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:38.092 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-26T03:39:38.178 INFO:tasks.ceph.osd.9.smithi042.stderr:2024-04-26T03:39:38.177+0000 7fdac63b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 31960) UID: 0 2024-04-26T03:39:38.240 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:38.240 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:38.249 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-26T03:39:38.279 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:39:38.278+0000 7fba0d2ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31961) UID: 0 2024-04-26T03:39:38.356 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:38.356 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:38.366 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-26T03:39:38.380 INFO:tasks.ceph.osd.9.smithi042.stderr:2024-04-26T03:39:38.379+0000 7fdac63b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 31960) UID: 0 2024-04-26T03:39:38.473 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:38.473 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:38.481 INFO:tasks.ceph.osd.9.smithi042.stderr:2024-04-26T03:39:38.480+0000 7fdac63b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 31960) UID: 0 2024-04-26T03:39:38.482 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-26T03:39:38.581 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:39:38.580+0000 7fb78c544640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31956) UID: 0 2024-04-26T03:39:38.589 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:38.589 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:38.598 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-26T03:39:38.683 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-04-26T03:39:38.682+0000 7fe9223c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31954) UID: 0 2024-04-26T03:39:38.695 INFO:teuthology.orchestra.run.smithi028.stderr:nodeep-scrub is set 2024-04-26T03:39:38.698 INFO:teuthology.orchestra.run.smithi028.stderr:reweighted osd.10 to 1 (10000) 2024-04-26T03:39:38.698 INFO:tasks.rados.rados.0.smithi092.stdout:358: done (0 left) 2024-04-26T03:39:38.699 INFO:tasks.rados.rados.0.smithi092.stdout:359: rollback oid 15 current snap is 29 2024-04-26T03:39:38.699 INFO:tasks.rados.rados.0.smithi092.stdout:rollback oid 15 to 18 2024-04-26T03:39:38.699 INFO:tasks.rados.rados.0.smithi092.stdout:360: setattr oid 21 current snap is 29 2024-04-26T03:39:38.700 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 50 current snap is 29 2024-04-26T03:39:38.700 INFO:tasks.rados.rados.0.smithi092.stdout:361: seq_num 158 ranges {1638400=262144} 2024-04-26T03:39:38.709 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:38.710 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:38.711 INFO:tasks.rados.rados.0.smithi092.stdout:361: writing smithi09235267-50 from 1638400 to 1900544 tid 1 2024-04-26T03:39:38.711 INFO:tasks.rados.rados.0.smithi092.stdout:362: copy_from oid 40 from oid 23 current snap is 29 2024-04-26T03:39:38.711 INFO:tasks.rados.rados.0.smithi092.stdout:359: finishing rollback tid 1 to smithi09235267-15 2024-04-26T03:39:38.711 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 15 v 0 (ObjNum 161 snap 0 seq_num 2187344954) dirty dne 2024-04-26T03:39:38.711 INFO:tasks.rados.rados.0.smithi092.stdout:359: done (3 left) 2024-04-26T03:39:38.711 INFO:tasks.rados.rados.0.smithi092.stdout:363: read oid 8 snap -1 2024-04-26T03:39:38.711 INFO:tasks.rados.rados.0.smithi092.stdout:363: expect (ObjNum 7 snap 0 seq_num 7) 2024-04-26T03:39:38.711 INFO:tasks.rados.rados.0.smithi092.stdout:364: read oid 42 snap 27 2024-04-26T03:39:38.711 INFO:tasks.rados.rados.0.smithi092.stdout:364: expect (ObjNum 60 snap 0 seq_num 60) 2024-04-26T03:39:38.712 INFO:tasks.rados.rados.0.smithi092.stdout:365: delete oid 4 current snap is 29 2024-04-26T03:39:38.712 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 21 v 116 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-04-26T03:39:38.717 INFO:tasks.rados.rados.0.smithi092.stdout:363: expect (ObjNum 7 snap 0 seq_num 7) 2024-04-26T03:39:38.722 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-04-26T03:39:38.724 INFO:tasks.rados.rados.0.smithi092.stdout:360: done (5 left) 2024-04-26T03:39:38.724 INFO:tasks.rados.rados.0.smithi092.stdout:363: done (4 left) 2024-04-26T03:39:38.724 INFO:tasks.rados.rados.0.smithi092.stdout:365: done (3 left) 2024-04-26T03:39:38.724 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 15 current snap is 29 2024-04-26T03:39:38.724 INFO:tasks.rados.rados.0.smithi092.stdout:366: seq_num 160 ranges {0=294912} 2024-04-26T03:39:38.724 INFO:tasks.rados.rados.0.smithi092.stdout:366: writing smithi09235267-15 from 0 to 294912 tid 1 2024-04-26T03:39:38.724 INFO:tasks.rados.rados.0.smithi092.stdout:367: setattr oid 13 current snap is 29 2024-04-26T03:39:38.724 INFO:tasks.rados.rados.0.smithi092.stdout:361: finishing write tid 1 to smithi09235267-50 2024-04-26T03:39:38.724 INFO:tasks.rados.rados.0.smithi092.stdout:361: finishing write tid 2 to smithi09235267-50 2024-04-26T03:39:38.724 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 11 1 2024-04-26T03:39:38.726 INFO:tasks.rados.rados.0.smithi092.stdout:361: finishing write tid 3 to smithi09235267-50 2024-04-26T03:39:38.726 INFO:tasks.rados.rados.0.smithi092.stdout:361: oid 50 updating version 0 to 133 2024-04-26T03:39:38.726 INFO:tasks.rados.rados.0.smithi092.stdout:361: oid 50 version 133 is already newer than 132 2024-04-26T03:39:38.726 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 50 v 133 (ObjNum 158 snap 29 seq_num 158) dirty exists 2024-04-26T03:39:38.726 INFO:tasks.rados.rados.0.smithi092.stdout:361: left oid 50 (ObjNum 158 snap 29 seq_num 158) 2024-04-26T03:39:38.726 INFO:tasks.rados.rados.0.smithi092.stdout:361: done (4 left) 2024-04-26T03:39:38.726 INFO:tasks.rados.rados.0.smithi092.stdout:368: copy_from oid 28 from oid 25 current snap is 29 2024-04-26T03:39:38.726 INFO:tasks.rados.rados.0.smithi092.stdout:369: rollback oid 48 current snap is 29 2024-04-26T03:39:38.727 INFO:tasks.rados.rados.0.smithi092.stdout:rollback oid 48 to 27 2024-04-26T03:39:38.727 INFO:tasks.rados.rados.0.smithi092.stdout:370: copy_from oid 19 from oid 35 current snap is 29 2024-04-26T03:39:38.727 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 32 current snap is 29 2024-04-26T03:39:38.727 INFO:tasks.rados.rados.0.smithi092.stdout:371: seq_num 161 ranges {2129920=311296} 2024-04-26T03:39:38.740 INFO:tasks.rados.rados.0.smithi092.stdout:371: writing smithi09235267-32 from 2129920 to 2441216 tid 1 2024-04-26T03:39:38.740 INFO:tasks.rados.rados.0.smithi092.stdout:372: snap_remove snap 23 2024-04-26T03:39:38.809 INFO:tasks.ceph.osd.8.smithi028.stderr:2024-04-26T03:39:38.807+0000 7f5bb75d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 31968) UID: 0 2024-04-26T03:39:38.838 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:38.839 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:38.847 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-04-26T03:39:38.886 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-04-26T03:39:38.884+0000 7fdc2d21c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31819) UID: 0 2024-04-26T03:39:38.954 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:38.954 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:38.964 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-04-26T03:39:38.986 INFO:tasks.ceph.osd.9.smithi042.stderr:2024-04-26T03:39:38.985+0000 7fdac63b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 31960) UID: 0 2024-04-26T03:39:39.071 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:39.071 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:39.080 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-04-26T03:39:39.088 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-04-26T03:39:39.086+0000 7fdc2d21c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31819) UID: 0 2024-04-26T03:39:39.186 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:39.187 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:39.188 INFO:tasks.ceph.osd.12.smithi028.stderr:2024-04-26T03:39:39.187+0000 7f114fe14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 31967) UID: 0 2024-04-26T03:39:39.195 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-04-26T03:39:39.289 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:39:39.288+0000 7fb78c544640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31956) UID: 0 2024-04-26T03:39:39.302 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:39.302 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:39.311 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-04-26T03:39:39.391 INFO:tasks.ceph.osd.9.smithi042.stderr:2024-04-26T03:39:39.389+0000 7fdac63b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 31960) UID: 0 2024-04-26T03:39:39.417 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:39.417 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:39.426 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-26T03:39:39.491 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:39:39.490+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:39:39.543 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:39.543 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:39.552 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-26T03:39:39.595 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-04-26T03:39:39.594+0000 7f745118d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 31821) UID: 0 2024-04-26T03:39:39.655 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:39.655 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:39.663 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-26T03:39:39.690 INFO:teuthology.orchestra.run.smithi028.stderr:reweighted osd.11 to 1 (10000) 2024-04-26T03:39:39.695 INFO:tasks.rados.rados.0.smithi092.stdout:372: done (8 left) 2024-04-26T03:39:39.695 INFO:tasks.rados.rados.0.smithi092.stdout:373: read oid 49 snap 22 2024-04-26T03:39:39.695 INFO:tasks.rados.rados.0.smithi092.stdout:373: expect deleted 2024-04-26T03:39:39.695 INFO:tasks.rados.rados.0.smithi092.stdout:370: finishing copy_from to smithi09235267-19 2024-04-26T03:39:39.695 INFO:tasks.rados.rados.0.smithi092.stdout:370: got expected ENOENT (src dne) 2024-04-26T03:39:39.695 INFO:tasks.rados.rados.0.smithi092.stdout:370: finishing copy_from racing read to smithi09235267-19 2024-04-26T03:39:39.696 INFO:tasks.rados.rados.0.smithi092.stdout:366: finishing write tid 1 to smithi09235267-15 2024-04-26T03:39:39.696 INFO:tasks.rados.rados.0.smithi092.stdout:366: finishing write tid 2 to smithi09235267-15 2024-04-26T03:39:39.696 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 13 v 118 (ObjNum 132 snap 17 seq_num 132) dirty exists 2024-04-26T03:39:39.696 INFO:tasks.rados.rados.0.smithi092.stdout:366: finishing write tid 3 to smithi09235267-15 2024-04-26T03:39:39.696 INFO:tasks.rados.rados.0.smithi092.stdout:366: oid 15 updating version 0 to 79 2024-04-26T03:39:39.696 INFO:tasks.rados.rados.0.smithi092.stdout:366: oid 15 updating version 79 to 80 2024-04-26T03:39:39.696 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 15 v 80 (ObjNum 160 snap 29 seq_num 160) dirty exists 2024-04-26T03:39:39.696 INFO:tasks.rados.rados.0.smithi092.stdout:366: left oid 15 (ObjNum 160 snap 29 seq_num 160) 2024-04-26T03:39:39.696 INFO:tasks.rados.rados.0.smithi092.stdout:364: expect (ObjNum 60 snap 0 seq_num 60) 2024-04-26T03:39:39.697 INFO:tasks.ceph.osd.15.smithi092.stderr:2024-04-26T03:39:39.694+0000 7f6d12dec640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 31824) UID: 0 2024-04-26T03:39:39.716 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 12 1 2024-04-26T03:39:39.786 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-26T03:39:39.796 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:39:39.794+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:39:39.805 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:39.805 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:39.808 INFO:tasks.rados.rados.0.smithi092.stdout:362: finishing copy_from racing read to smithi09235267-40 2024-04-26T03:39:39.809 INFO:tasks.rados.rados.0.smithi092.stdout:362: finishing copy_from to smithi09235267-40 2024-04-26T03:39:39.809 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 40 v 55 (ObjNum 110 snap 8 seq_num 110) dirty exists 2024-04-26T03:39:39.809 INFO:tasks.rados.rados.0.smithi092.stdout:368: finishing copy_from racing read to smithi09235267-28 2024-04-26T03:39:39.809 INFO:tasks.rados.rados.0.smithi092.stdout:368: finishing copy_from to smithi09235267-28 2024-04-26T03:39:39.809 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 28 v 80 (ObjNum 139 snap 17 seq_num 139) dirty exists 2024-04-26T03:39:39.809 INFO:tasks.rados.rados.0.smithi092.stdout:362: done (8 left) 2024-04-26T03:39:39.809 INFO:tasks.rados.rados.0.smithi092.stdout:364: done (7 left) 2024-04-26T03:39:39.809 INFO:tasks.rados.rados.0.smithi092.stdout:366: done (6 left) 2024-04-26T03:39:39.809 INFO:tasks.rados.rados.0.smithi092.stdout:367: done (5 left) 2024-04-26T03:39:39.809 INFO:tasks.rados.rados.0.smithi092.stdout:368: done (4 left) 2024-04-26T03:39:39.809 INFO:tasks.rados.rados.0.smithi092.stdout:370: done (3 left) 2024-04-26T03:39:39.809 INFO:tasks.rados.rados.0.smithi092.stdout:374: copy_from oid 21 from oid 10 current snap is 29 2024-04-26T03:39:39.809 INFO:tasks.rados.rados.0.smithi092.stdout:369: finishing rollback tid 0 to smithi09235267-48 2024-04-26T03:39:39.810 INFO:tasks.rados.rados.0.smithi092.stdout:369: finishing rollback tid 1 to smithi09235267-48 2024-04-26T03:39:39.810 INFO:tasks.rados.rados.0.smithi092.stdout:369: finishing rollback tid 2 to smithi09235267-48 2024-04-26T03:39:39.810 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 48 v 88 (ObjNum 161 snap 0 seq_num 0) dirty exists 2024-04-26T03:39:39.810 INFO:tasks.rados.rados.0.smithi092.stdout:371: finishing write tid 1 to smithi09235267-32 2024-04-26T03:39:39.810 INFO:tasks.rados.rados.0.smithi092.stdout:371: finishing write tid 2 to smithi09235267-32 2024-04-26T03:39:39.810 INFO:tasks.rados.rados.0.smithi092.stdout:371: finishing write tid 3 to smithi09235267-32 2024-04-26T03:39:39.810 INFO:tasks.rados.rados.0.smithi092.stdout:371: oid 32 updating version 0 to 91 2024-04-26T03:39:39.810 INFO:tasks.rados.rados.0.smithi092.stdout:371: oid 32 version 91 is already newer than 90 2024-04-26T03:39:39.810 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 32 v 91 (ObjNum 161 snap 29 seq_num 161) dirty exists 2024-04-26T03:39:39.810 INFO:tasks.rados.rados.0.smithi092.stdout:371: left oid 32 (ObjNum 161 snap 29 seq_num 161) 2024-04-26T03:39:39.810 INFO:tasks.rados.rados.0.smithi092.stdout:369: done (3 left) 2024-04-26T03:39:39.810 INFO:tasks.rados.rados.0.smithi092.stdout:371: done (2 left) 2024-04-26T03:39:39.810 INFO:tasks.rados.rados.0.smithi092.stdout:375: read oid 43 snap 16 2024-04-26T03:39:39.810 INFO:tasks.rados.rados.0.smithi092.stdout:375: expect deleted 2024-04-26T03:39:39.811 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 38 current snap is 29 2024-04-26T03:39:39.811 INFO:tasks.rados.rados.0.smithi092.stdout:376: seq_num 162 ranges {0=327680} 2024-04-26T03:39:39.811 INFO:tasks.rados.rados.0.smithi092.stdout:376: writing smithi09235267-38 from 0 to 327680 tid 1 2024-04-26T03:39:39.811 INFO:tasks.rados.rados.0.smithi092.stdout:377: delete oid 15 current snap is 29 2024-04-26T03:39:39.812 INFO:tasks.rados.rados.0.smithi092.stdout:373: done (4 left) 2024-04-26T03:39:39.813 INFO:tasks.rados.rados.0.smithi092.stdout:375: done (3 left) 2024-04-26T03:39:39.813 INFO:tasks.rados.rados.0.smithi092.stdout:377: done (2 left) 2024-04-26T03:39:39.813 INFO:tasks.rados.rados.0.smithi092.stdout:378: read oid 22 snap -1 2024-04-26T03:39:39.813 INFO:tasks.rados.rados.0.smithi092.stdout:378: expect (ObjNum 136 snap 17 seq_num 136) 2024-04-26T03:39:39.813 INFO:tasks.rados.rados.0.smithi092.stdout:379: snap_create 2024-04-26T03:39:39.817 INFO:tasks.rados.rados.0.smithi092.stdout:376: finishing write tid 1 to smithi09235267-38 2024-04-26T03:39:39.817 INFO:tasks.rados.rados.0.smithi092.stdout:376: finishing write tid 2 to smithi09235267-38 2024-04-26T03:39:39.818 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-26T03:39:39.820 INFO:tasks.rados.rados.0.smithi092.stdout:376: finishing write tid 3 to smithi09235267-38 2024-04-26T03:39:39.820 INFO:tasks.rados.rados.0.smithi092.stdout:376: oid 38 updating version 0 to 86 2024-04-26T03:39:39.820 INFO:tasks.rados.rados.0.smithi092.stdout:376: oid 38 updating version 86 to 87 2024-04-26T03:39:39.820 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 38 v 87 (ObjNum 162 snap 29 seq_num 162) dirty exists 2024-04-26T03:39:39.821 INFO:tasks.rados.rados.0.smithi092.stdout:376: left oid 38 (ObjNum 162 snap 29 seq_num 162) 2024-04-26T03:39:39.824 INFO:tasks.rados.rados.0.smithi092.stdout:378: expect (ObjNum 136 snap 17 seq_num 136) 2024-04-26T03:39:39.896 INFO:tasks.ceph.osd.12.smithi028.stderr:2024-04-26T03:39:39.895+0000 7f114fe14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 31967) UID: 0 2024-04-26T03:39:39.917 INFO:tasks.rados.rados.0.smithi092.stdout:374: finishing copy_from racing read to smithi09235267-21 2024-04-26T03:39:39.917 INFO:tasks.rados.rados.0.smithi092.stdout:374: finishing copy_from to smithi09235267-21 2024-04-26T03:39:39.917 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 21 v 50 (ObjNum 90 snap 7 seq_num 90) dirty exists 2024-04-26T03:39:39.924 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:39.924 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:39.932 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0: {} 2024-04-26T03:39:39.932 INFO:teuthology.orchestra.run.smithi028.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T03:39:39.933 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-26T03:39:39.951 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1: {} 2024-04-26T03:39:39.951 INFO:teuthology.orchestra.run.smithi028.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T03:39:39.968 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2: {} 2024-04-26T03:39:39.968 INFO:teuthology.orchestra.run.smithi028.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T03:39:39.986 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3: {} 2024-04-26T03:39:39.986 INFO:teuthology.orchestra.run.smithi028.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T03:39:39.997 INFO:tasks.ceph.osd.8.smithi028.stderr:2024-04-26T03:39:39.995+0000 7f5bb75d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 31968) UID: 0 2024-04-26T03:39:40.004 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4: {} 2024-04-26T03:39:40.004 INFO:teuthology.orchestra.run.smithi028.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T03:39:40.022 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5: {} 2024-04-26T03:39:40.022 INFO:teuthology.orchestra.run.smithi028.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T03:39:40.039 INFO:teuthology.orchestra.run.smithi028.stdout:osd.6: {} 2024-04-26T03:39:40.040 INFO:teuthology.orchestra.run.smithi028.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T03:39:40.057 INFO:teuthology.orchestra.run.smithi028.stdout:osd.7: {} 2024-04-26T03:39:40.057 INFO:teuthology.orchestra.run.smithi028.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T03:39:40.076 INFO:teuthology.orchestra.run.smithi028.stdout:osd.8: {} 2024-04-26T03:39:40.076 INFO:teuthology.orchestra.run.smithi028.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T03:39:40.077 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:40.077 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-26T03:39:40.077 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-26T03:39:40.077 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-26T03:39:40.077 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:40.084 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-26T03:39:40.094 INFO:teuthology.orchestra.run.smithi028.stdout:osd.9: {} 2024-04-26T03:39:40.094 INFO:teuthology.orchestra.run.smithi028.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T03:39:40.098 INFO:tasks.ceph.osd.9.smithi042.stderr:2024-04-26T03:39:40.097+0000 7fdac63b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 31960) UID: 0 2024-04-26T03:39:40.112 INFO:teuthology.orchestra.run.smithi028.stdout:osd.10: {} 2024-04-26T03:39:40.112 INFO:teuthology.orchestra.run.smithi028.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T03:39:40.129 INFO:teuthology.orchestra.run.smithi028.stdout:osd.11: {} 2024-04-26T03:39:40.129 INFO:teuthology.orchestra.run.smithi028.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T03:39:40.146 INFO:teuthology.orchestra.run.smithi028.stdout:osd.12: {} 2024-04-26T03:39:40.146 INFO:teuthology.orchestra.run.smithi028.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T03:39:40.164 INFO:teuthology.orchestra.run.smithi028.stdout:osd.13: {} 2024-04-26T03:39:40.164 INFO:teuthology.orchestra.run.smithi028.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T03:39:40.181 INFO:teuthology.orchestra.run.smithi028.stdout:osd.14: {} 2024-04-26T03:39:40.182 INFO:teuthology.orchestra.run.smithi028.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T03:39:40.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:39:40.180+0000 7f584df71640 0 -- 172.21.15.28:0/2039498478 >> [v2:172.21.15.92:6826/3982634323,v1:172.21.15.92:6827/3982634323] conn(0x7f5848222bd0 msgr2=0x7f5848224fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-26T03:39:40.189 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:40.189 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-26T03:39:40.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-26T03:39:40.189 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-26T03:39:40.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.189 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:45 3.ds0 3:b4337c88:::smithi09235267-15:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:40.189 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.498959+0000", 2024-04-26T03:39:40.189 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 48.687463346999998, 2024-04-26T03:39:40.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.028116775, 2024-04-26T03:39:40.190 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:40.190 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:40.190 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:40.190 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:40.190 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:40.190 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:40.190 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45 2024-04-26T03:39:40.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.190 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:40.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:40.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498959+0000", 2024-04-26T03:39:40.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:40.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498959+0000", 2024-04-26T03:39:40.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:40.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498959+0000", 2024-04-26T03:39:40.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.58e-07 2024-04-26T03:39:40.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:40.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498963+0000", 2024-04-26T03:39:40.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2449999999999998e-06 2024-04-26T03:39:40.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:40.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498970+0000", 2024-04-26T03:39:40.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5780000000000001e-06 2024-04-26T03:39:40.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:40.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498976+0000", 2024-04-26T03:39:40.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6400000000000002e-06 2024-04-26T03:39:40.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.504199+0000", 2024-04-26T03:39:40.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052229190000000003 2024-04-26T03:39:40.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:40.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.504274+0000", 2024-04-26T03:39:40.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5187999999999999e-05 2024-04-26T03:39:40.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.512941+0000", 2024-04-26T03:39:40.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0086671879999999993 2024-04-26T03:39:40.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:40.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.513001+0000", 2024-04-26T03:39:40.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0204000000000002e-05 2024-04-26T03:39:40.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:40.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.527076+0000", 2024-04-26T03:39:40.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014074355 2024-04-26T03:39:40.196 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.196 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:40.196 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.196 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:52 3.ds0 3:b895fe71:::smithi09235267-18:head [append 0~294912,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:40.196 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.505560+0000", 2024-04-26T03:39:40.196 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 48.680862347999998, 2024-04-26T03:39:40.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016117464000000001, 2024-04-26T03:39:40.196 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:40.196 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:40.196 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:40.197 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:40.197 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:40.197 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:40.197 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 52 2024-04-26T03:39:40.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.197 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:40.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:40.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505560+0000", 2024-04-26T03:39:40.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:40.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505560+0000", 2024-04-26T03:39:40.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:40.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505562+0000", 2024-04-26T03:39:40.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0729999999999999e-06 2024-04-26T03:39:40.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:40.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505753+0000", 2024-04-26T03:39:40.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019100500000000001 2024-04-26T03:39:40.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:40.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505768+0000", 2024-04-26T03:39:40.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5213e-05 2024-04-26T03:39:40.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:40.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505776+0000", 2024-04-26T03:39:40.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7279999999999995e-06 2024-04-26T03:39:40.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505830+0000", 2024-04-26T03:39:40.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3749999999999999e-05 2024-04-26T03:39:40.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:40.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505998+0000", 2024-04-26T03:39:40.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000168259 2024-04-26T03:39:40.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:40.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.506585+0000", 2024-04-26T03:39:40.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00058713299999999999 2024-04-26T03:39:40.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:40.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.511525+0000", 2024-04-26T03:39:40.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004939789 2024-04-26T03:39:40.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:40.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.521601+0000", 2024-04-26T03:39:40.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010075766 2024-04-26T03:39:40.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:40.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.521677+0000", 2024-04-26T03:39:40.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6748000000000004e-05 2024-04-26T03:39:40.201 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.201 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:40.201 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.202 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:53 3.ds0 3:b895fe71:::smithi09235267-18:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:40.202 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.505792+0000", 2024-04-26T03:39:40.202 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 48.680629908999997, 2024-04-26T03:39:40.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016119725000000001, 2024-04-26T03:39:40.202 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:40.202 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:40.202 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:40.202 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:40.202 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:40.202 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:40.202 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 53 2024-04-26T03:39:40.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.202 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:40.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:40.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505792+0000", 2024-04-26T03:39:40.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:40.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505792+0000", 2024-04-26T03:39:40.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:40.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505793+0000", 2024-04-26T03:39:40.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5400000000000001e-07 2024-04-26T03:39:40.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:40.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505806+0000", 2024-04-26T03:39:40.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2733e-05 2024-04-26T03:39:40.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:40.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505814+0000", 2024-04-26T03:39:40.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0260000000000007e-06 2024-04-26T03:39:40.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:40.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505820+0000", 2024-04-26T03:39:40.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8769999999999998e-06 2024-04-26T03:39:40.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.509659+0000", 2024-04-26T03:39:40.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0038395149999999999 2024-04-26T03:39:40.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:40.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.509764+0000", 2024-04-26T03:39:40.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105159 2024-04-26T03:39:40.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:40.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.510189+0000", 2024-04-26T03:39:40.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00042430899999999999 2024-04-26T03:39:40.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:40.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.511526+0000", 2024-04-26T03:39:40.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001337934 2024-04-26T03:39:40.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:40.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.521837+0000", 2024-04-26T03:39:40.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010310503 2024-04-26T03:39:40.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:40.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.521912+0000", 2024-04-26T03:39:40.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5014999999999999e-05 2024-04-26T03:39:40.208 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.208 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:40.208 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.208 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:54 3.ds0 3:b895fe71:::smithi09235267-18:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:40.208 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.505829+0000", 2024-04-26T03:39:40.208 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 48.680593068, 2024-04-26T03:39:40.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021406122, 2024-04-26T03:39:40.209 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:40.209 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:40.209 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:40.209 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:40.209 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:40.209 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:40.209 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 54 2024-04-26T03:39:40.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.209 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:40.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:40.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505829+0000", 2024-04-26T03:39:40.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:40.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505829+0000", 2024-04-26T03:39:40.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:40.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505830+0000", 2024-04-26T03:39:40.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2e-07 2024-04-26T03:39:40.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:40.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505832+0000", 2024-04-26T03:39:40.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.509e-06 2024-04-26T03:39:40.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:40.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505839+0000", 2024-04-26T03:39:40.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9110000000000004e-06 2024-04-26T03:39:40.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:40.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505844+0000", 2024-04-26T03:39:40.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2109999999999997e-06 2024-04-26T03:39:40.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.510372+0000", 2024-04-26T03:39:40.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045275990000000002 2024-04-26T03:39:40.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:40.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.510419+0000", 2024-04-26T03:39:40.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7524000000000001e-05 2024-04-26T03:39:40.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.522258+0000", 2024-04-26T03:39:40.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011838448999999999 2024-04-26T03:39:40.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:40.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.522334+0000", 2024-04-26T03:39:40.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6134e-05 2024-04-26T03:39:40.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:40.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.527235+0000", 2024-04-26T03:39:40.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049012650000000001 2024-04-26T03:39:40.214 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.214 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:40.214 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.214 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:64 3.ds0 3:b3f958c3:::smithi09235267-22:head [append 0~360448,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:40.214 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.515665+0000", 2024-04-26T03:39:40.214 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 48.670756873000002, 2024-04-26T03:39:40.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.023538667999999999, 2024-04-26T03:39:40.214 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:40.214 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:40.215 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:40.215 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:40.215 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:40.215 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:40.215 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 64 2024-04-26T03:39:40.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.215 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:40.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:40.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515665+0000", 2024-04-26T03:39:40.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:40.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515665+0000", 2024-04-26T03:39:40.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:40.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515667+0000", 2024-04-26T03:39:40.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.652e-06 2024-04-26T03:39:40.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:40.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515857+0000", 2024-04-26T03:39:40.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000190447 2024-04-26T03:39:40.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:40.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515869+0000", 2024-04-26T03:39:40.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1450999999999999e-05 2024-04-26T03:39:40.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:40.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515876+0000", 2024-04-26T03:39:40.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2024-04-26T03:39:40.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.527263+0000", 2024-04-26T03:39:40.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011387061 2024-04-26T03:39:40.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:40.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.527389+0000", 2024-04-26T03:39:40.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000126172 2024-04-26T03:39:40.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:40.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.528593+0000", 2024-04-26T03:39:40.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00120386 2024-04-26T03:39:40.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:40.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.533773+0000", 2024-04-26T03:39:40.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005179973 2024-04-26T03:39:40.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:40.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.539131+0000", 2024-04-26T03:39:40.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053582279999999996 2024-04-26T03:39:40.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:40.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.539204+0000", 2024-04-26T03:39:40.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2809e-05 2024-04-26T03:39:40.219 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.219 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:40.219 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.219 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:65 3.ds0 3:b3f958c3:::smithi09235267-22:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:40.220 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.515889+0000", 2024-04-26T03:39:40.220 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 48.670533233, 2024-04-26T03:39:40.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.023486671000000001, 2024-04-26T03:39:40.220 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:40.220 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:40.220 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:40.220 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:40.220 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:40.220 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:40.220 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 65 2024-04-26T03:39:40.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.220 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:40.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:40.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515889+0000", 2024-04-26T03:39:40.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:40.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515889+0000", 2024-04-26T03:39:40.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:40.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515890+0000", 2024-04-26T03:39:40.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.13e-07 2024-04-26T03:39:40.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:40.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515894+0000", 2024-04-26T03:39:40.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.386e-06 2024-04-26T03:39:40.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:40.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515903+0000", 2024-04-26T03:39:40.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.166e-06 2024-04-26T03:39:40.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:40.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515909+0000", 2024-04-26T03:39:40.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.412e-06 2024-04-26T03:39:40.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.532799+0000", 2024-04-26T03:39:40.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016889990000000001 2024-04-26T03:39:40.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:40.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.532885+0000", 2024-04-26T03:39:40.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6336999999999997e-05 2024-04-26T03:39:40.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:40.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.533334+0000", 2024-04-26T03:39:40.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00044950799999999999 2024-04-26T03:39:40.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:40.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.538492+0000", 2024-04-26T03:39:40.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051571389999999998 2024-04-26T03:39:40.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:40.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.539303+0000", 2024-04-26T03:39:40.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000811325 2024-04-26T03:39:40.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:40.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.539376+0000", 2024-04-26T03:39:40.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2794999999999997e-05 2024-04-26T03:39:40.225 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.225 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:40.225 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.225 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:66 3.ds0 3:b3f958c3:::smithi09235267-22:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:40.225 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.515917+0000", 2024-04-26T03:39:40.225 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 48.670505658000003, 2024-04-26T03:39:40.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.024757049, 2024-04-26T03:39:40.225 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:40.225 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:40.225 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:40.225 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:40.226 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:40.226 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:40.226 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 66 2024-04-26T03:39:40.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.226 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:40.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:40.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515917+0000", 2024-04-26T03:39:40.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:40.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515917+0000", 2024-04-26T03:39:40.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:40.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515917+0000", 2024-04-26T03:39:40.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8699999999999995e-07 2024-04-26T03:39:40.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:40.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515920+0000", 2024-04-26T03:39:40.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.055e-06 2024-04-26T03:39:40.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:40.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515927+0000", 2024-04-26T03:39:40.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5150000000000003e-06 2024-04-26T03:39:40.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:40.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515936+0000", 2024-04-26T03:39:40.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7920000000000004e-06 2024-04-26T03:39:40.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.533517+0000", 2024-04-26T03:39:40.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017580535000000001 2024-04-26T03:39:40.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:40.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.533598+0000", 2024-04-26T03:39:40.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1354999999999996e-05 2024-04-26T03:39:40.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.539706+0000", 2024-04-26T03:39:40.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061075569999999996 2024-04-26T03:39:40.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:40.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.539758+0000", 2024-04-26T03:39:40.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2500000000000002e-05 2024-04-26T03:39:40.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:40.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.540674+0000", 2024-04-26T03:39:40.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00091525299999999999 2024-04-26T03:39:40.230 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.230 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:40.230 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.230 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:607 3.ds0 3:b4337c88:::smithi09235267-15:head [append 0~737280] snapc d=[d,c,b,a] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-26T03:39:40.230 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:11.722465+0000", 2024-04-26T03:39:40.230 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.46395708, 2024-04-26T03:39:40.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019589842999999999, 2024-04-26T03:39:40.230 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:40.231 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:40.231 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:40.231 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:40.231 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:40.231 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:40.231 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 607 2024-04-26T03:39:40.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.231 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:40.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:40.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.722465+0000", 2024-04-26T03:39:40.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:40.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.722465+0000", 2024-04-26T03:39:40.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:40.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.722467+0000", 2024-04-26T03:39:40.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.553e-06 2024-04-26T03:39:40.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:40.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.722788+0000", 2024-04-26T03:39:40.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00032146600000000002 2024-04-26T03:39:40.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:40.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.722805+0000", 2024-04-26T03:39:40.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6534000000000001e-05 2024-04-26T03:39:40.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:40.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.722815+0000", 2024-04-26T03:39:40.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0062e-05 2024-04-26T03:39:40.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.722890+0000", 2024-04-26T03:39:40.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5012000000000006e-05 2024-04-26T03:39:40.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:40.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.723013+0000", 2024-04-26T03:39:40.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000123693 2024-04-26T03:39:40.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:40.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.724487+0000", 2024-04-26T03:39:40.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001473427 2024-04-26T03:39:40.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:40.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.741651+0000", 2024-04-26T03:39:40.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017163754999999999 2024-04-26T03:39:40.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:40.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.741886+0000", 2024-04-26T03:39:40.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0002354 2024-04-26T03:39:40.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:40.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.742055+0000", 2024-04-26T03:39:40.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016894100000000001 2024-04-26T03:39:40.235 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.235 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:40.235 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.235 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:608 3.ds0 3:b4337c88:::smithi09235267-15:head [append 0~557056] snapc d=[d,c,b,a] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-26T03:39:40.236 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:11.726593+0000", 2024-04-26T03:39:40.236 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.459828845000001, 2024-04-26T03:39:40.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019055236999999999, 2024-04-26T03:39:40.236 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:40.236 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:40.236 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:40.236 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:40.236 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:40.236 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:40.236 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 608 2024-04-26T03:39:40.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.236 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:40.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:40.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.726593+0000", 2024-04-26T03:39:40.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:40.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.726593+0000", 2024-04-26T03:39:40.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:40.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.726596+0000", 2024-04-26T03:39:40.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3649999999999998e-06 2024-04-26T03:39:40.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:40.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.726927+0000", 2024-04-26T03:39:40.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00033128999999999998 2024-04-26T03:39:40.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:40.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.726950+0000", 2024-04-26T03:39:40.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.317e-05 2024-04-26T03:39:40.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:40.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.726961+0000", 2024-04-26T03:39:40.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0387999999999999e-05 2024-04-26T03:39:40.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.730805+0000", 2024-04-26T03:39:40.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003843993 2024-04-26T03:39:40.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:40.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.730917+0000", 2024-04-26T03:39:40.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011274599999999999 2024-04-26T03:39:40.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:40.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.731823+0000", 2024-04-26T03:39:40.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00090538700000000001 2024-04-26T03:39:40.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:40.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.742998+0000", 2024-04-26T03:39:40.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011175144 2024-04-26T03:39:40.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:40.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.745560+0000", 2024-04-26T03:39:40.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025625660000000001 2024-04-26T03:39:40.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:40.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.745649+0000", 2024-04-26T03:39:40.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8188000000000003e-05 2024-04-26T03:39:40.241 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.241 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:40.241 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.241 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:610 3.ds0 3:b4337c88:::smithi09235267-15:head [setxattr _header (57) in=7b] snapc d=[d,c,b,a] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-26T03:39:40.241 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:11.731469+0000", 2024-04-26T03:39:40.241 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.454953400000001, 2024-04-26T03:39:40.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015326482000000001, 2024-04-26T03:39:40.241 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:40.241 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:40.241 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:40.241 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:40.241 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:40.241 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:40.242 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 610 2024-04-26T03:39:40.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.242 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:40.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:40.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.731469+0000", 2024-04-26T03:39:40.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:40.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.731469+0000", 2024-04-26T03:39:40.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:40.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.731469+0000", 2024-04-26T03:39:40.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2800000000000002e-07 2024-04-26T03:39:40.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:40.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.731473+0000", 2024-04-26T03:39:40.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.456e-06 2024-04-26T03:39:40.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:40.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.731478+0000", 2024-04-26T03:39:40.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9860000000000002e-06 2024-04-26T03:39:40.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:40.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.731481+0000", 2024-04-26T03:39:40.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2590000000000001e-06 2024-04-26T03:39:40.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.742103+0000", 2024-04-26T03:39:40.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010621680999999999 2024-04-26T03:39:40.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:40.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.742180+0000", 2024-04-26T03:39:40.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7392999999999996e-05 2024-04-26T03:39:40.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:40.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.742694+0000", 2024-04-26T03:39:40.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000513654 2024-04-26T03:39:40.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:40.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.745660+0000", 2024-04-26T03:39:40.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002966836 2024-04-26T03:39:40.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:40.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.746708+0000", 2024-04-26T03:39:40.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010471619999999999 2024-04-26T03:39:40.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:40.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.746795+0000", 2024-04-26T03:39:40.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7627000000000007e-05 2024-04-26T03:39:40.246 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.246 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:40.246 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.246 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:611 3.ds0 3:b4337c88:::smithi09235267-15:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-04-26T03:39:40.246 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:11.731485+0000", 2024-04-26T03:39:40.246 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.454936700000001, 2024-04-26T03:39:40.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017052068, 2024-04-26T03:39:40.247 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:40.247 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:40.247 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:40.247 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:40.247 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:40.247 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:40.247 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 611 2024-04-26T03:39:40.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.247 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:40.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:40.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.731485+0000", 2024-04-26T03:39:40.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:40.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.731485+0000", 2024-04-26T03:39:40.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:40.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.731486+0000", 2024-04-26T03:39:40.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6899999999999999e-07 2024-04-26T03:39:40.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:40.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.731487+0000", 2024-04-26T03:39:40.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.44e-06 2024-04-26T03:39:40.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:40.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.731492+0000", 2024-04-26T03:39:40.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9930000000000003e-06 2024-04-26T03:39:40.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:40.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.731495+0000", 2024-04-26T03:39:40.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.022e-06 2024-04-26T03:39:40.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.743270+0000", 2024-04-26T03:39:40.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011774316 2024-04-26T03:39:40.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:40.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.743323+0000", 2024-04-26T03:39:40.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3463e-05 2024-04-26T03:39:40.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.747168+0000", 2024-04-26T03:39:40.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003844859 2024-04-26T03:39:40.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:40.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.747255+0000", 2024-04-26T03:39:40.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7052000000000006e-05 2024-04-26T03:39:40.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:40.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.748538+0000", 2024-04-26T03:39:40.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001282654 2024-04-26T03:39:40.251 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.251 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:40.251 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.251 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:690 3.ds0 3:b895fe71:::smithi09235267-18:head [append 0~720896] snapc f=[f,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-04-26T03:39:40.251 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:15.944310+0000", 2024-04-26T03:39:40.252 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.242112621, 2024-04-26T03:39:40.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016189867, 2024-04-26T03:39:40.252 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:40.252 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:40.252 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:40.252 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:40.252 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:40.252 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:40.252 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 690 2024-04-26T03:39:40.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.252 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:40.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:40.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.944310+0000", 2024-04-26T03:39:40.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:40.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.944310+0000", 2024-04-26T03:39:40.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:40.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.944311+0000", 2024-04-26T03:39:40.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2500000000000001e-06 2024-04-26T03:39:40.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:40.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.944635+0000", 2024-04-26T03:39:40.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000324271 2024-04-26T03:39:40.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:40.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.944644+0000", 2024-04-26T03:39:40.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1670000000000005e-06 2024-04-26T03:39:40.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:40.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.944651+0000", 2024-04-26T03:39:40.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4300000000000003e-06 2024-04-26T03:39:40.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.944689+0000", 2024-04-26T03:39:40.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8272000000000002e-05 2024-04-26T03:39:40.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:40.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.944788+0000", 2024-04-26T03:39:40.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8916e-05 2024-04-26T03:39:40.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:40.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.945668+0000", 2024-04-26T03:39:40.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00088055000000000002 2024-04-26T03:39:40.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:40.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.960314+0000", 2024-04-26T03:39:40.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014645467000000001 2024-04-26T03:39:40.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:40.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.960414+0000", 2024-04-26T03:39:40.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.98e-05 2024-04-26T03:39:40.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:40.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.960499+0000", 2024-04-26T03:39:40.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5743999999999998e-05 2024-04-26T03:39:40.257 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.257 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:40.257 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.257 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:694 3.ds0 3:b895fe71:::smithi09235267-18:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e60)", 2024-04-26T03:39:40.257 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:15.950694+0000", 2024-04-26T03:39:40.257 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.23572832, 2024-04-26T03:39:40.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018244219999999998, 2024-04-26T03:39:40.257 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:40.257 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:40.257 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:40.258 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:40.258 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:40.258 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:40.258 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 694 2024-04-26T03:39:40.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.258 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:40.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:40.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.950694+0000", 2024-04-26T03:39:40.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:40.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.950694+0000", 2024-04-26T03:39:40.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:40.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.950694+0000", 2024-04-26T03:39:40.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.82e-07 2024-04-26T03:39:40.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:40.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.950697+0000", 2024-04-26T03:39:40.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.559e-06 2024-04-26T03:39:40.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:40.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.950704+0000", 2024-04-26T03:39:40.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.917e-06 2024-04-26T03:39:40.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:40.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.950713+0000", 2024-04-26T03:39:40.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5419999999999994e-06 2024-04-26T03:39:40.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.962925+0000", 2024-04-26T03:39:40.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012211926999999999 2024-04-26T03:39:40.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:40.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.963013+0000", 2024-04-26T03:39:40.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7791e-05 2024-04-26T03:39:40.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.966400+0000", 2024-04-26T03:39:40.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003386816 2024-04-26T03:39:40.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:40.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.966566+0000", 2024-04-26T03:39:40.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000166525 2024-04-26T03:39:40.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:40.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.968938+0000", 2024-04-26T03:39:40.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023716610000000002 2024-04-26T03:39:40.262 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.262 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:40.262 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.263 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.10s2 60/48 ECSubWrite(tid=272, reqid=client.4556.0:703, at_version=60'20, trim_to=0'0, roll_forward_to=60'18))", 2024-04-26T03:39:40.263 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:15.958431+0000", 2024-04-26T03:39:40.263 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.227991375999999, 2024-04-26T03:39:40.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016173328000000001, 2024-04-26T03:39:40.263 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:40.263 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:40.263 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:40.263 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:40.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:40.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.958431+0000", 2024-04-26T03:39:40.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:40.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.958431+0000", 2024-04-26T03:39:40.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:40.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.958432+0000", 2024-04-26T03:39:40.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.031e-06 2024-04-26T03:39:40.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:40.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.962281+0000", 2024-04-26T03:39:40.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0038488630000000001 2024-04-26T03:39:40.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:40.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.962301+0000", 2024-04-26T03:39:40.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9828999999999999e-05 2024-04-26T03:39:40.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:40.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.962308+0000", 2024-04-26T03:39:40.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2620000000000001e-06 2024-04-26T03:39:40.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.962375+0000", 2024-04-26T03:39:40.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6729000000000004e-05 2024-04-26T03:39:40.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:40.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.962384+0000", 2024-04-26T03:39:40.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6390000000000004e-06 2024-04-26T03:39:40.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:40.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.974567+0000", 2024-04-26T03:39:40.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012182767000000001 2024-04-26T03:39:40.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:40.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.974604+0000", 2024-04-26T03:39:40.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7208e-05 2024-04-26T03:39:40.267 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.267 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:40.267 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.267 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:774 3.ds0 3:b3f958c3:::smithi09235267-22:head [append 0~737280] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:40.267 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:20.780689+0000", 2024-04-26T03:39:40.267 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.405733396999999, 2024-04-26T03:39:40.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021054691, 2024-04-26T03:39:40.267 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:40.267 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:40.267 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:40.267 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:40.267 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:40.268 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:40.268 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 774 2024-04-26T03:39:40.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.268 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:40.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:40.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.780689+0000", 2024-04-26T03:39:40.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:40.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.780689+0000", 2024-04-26T03:39:40.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:40.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.780690+0000", 2024-04-26T03:39:40.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3400000000000001e-06 2024-04-26T03:39:40.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:40.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.780841+0000", 2024-04-26T03:39:40.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000150743 2024-04-26T03:39:40.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:40.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.780853+0000", 2024-04-26T03:39:40.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1681e-05 2024-04-26T03:39:40.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:40.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.780858+0000", 2024-04-26T03:39:40.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8390000000000003e-06 2024-04-26T03:39:40.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.780915+0000", 2024-04-26T03:39:40.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6338000000000002e-05 2024-04-26T03:39:40.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:40.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.781021+0000", 2024-04-26T03:39:40.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106498 2024-04-26T03:39:40.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:40.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.782158+0000", 2024-04-26T03:39:40.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001136977 2024-04-26T03:39:40.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:40.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.795362+0000", 2024-04-26T03:39:40.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013204052000000001 2024-04-26T03:39:40.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:40.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.801661+0000", 2024-04-26T03:39:40.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0062988540000000004 2024-04-26T03:39:40.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:40.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.801743+0000", 2024-04-26T03:39:40.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2368999999999997e-05 2024-04-26T03:39:40.272 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.272 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:40.272 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.272 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:775 3.ds0 3:b3f958c3:::smithi09235267-22:head [append 0~770048] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:40.272 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:20.785167+0000", 2024-04-26T03:39:40.272 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.401255361, 2024-04-26T03:39:40.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017555833, 2024-04-26T03:39:40.272 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:40.273 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:40.273 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:40.273 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:40.273 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:40.273 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:40.273 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 775 2024-04-26T03:39:40.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.273 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:40.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:40.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.785167+0000", 2024-04-26T03:39:40.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:40.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.785167+0000", 2024-04-26T03:39:40.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:40.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.785168+0000", 2024-04-26T03:39:40.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.339e-06 2024-04-26T03:39:40.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:40.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.785488+0000", 2024-04-26T03:39:40.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031973100000000002 2024-04-26T03:39:40.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:40.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.785499+0000", 2024-04-26T03:39:40.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1409999999999999e-05 2024-04-26T03:39:40.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:40.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.785506+0000", 2024-04-26T03:39:40.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.224e-06 2024-04-26T03:39:40.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.789235+0000", 2024-04-26T03:39:40.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0037290069999999999 2024-04-26T03:39:40.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:40.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.789350+0000", 2024-04-26T03:39:40.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115327 2024-04-26T03:39:40.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:40.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.790178+0000", 2024-04-26T03:39:40.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00082851799999999998 2024-04-26T03:39:40.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:40.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.801460+0000", 2024-04-26T03:39:40.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011282003000000001 2024-04-26T03:39:40.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:40.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.802652+0000", 2024-04-26T03:39:40.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011913889999999999 2024-04-26T03:39:40.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:40.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.802723+0000", 2024-04-26T03:39:40.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0884999999999996e-05 2024-04-26T03:39:40.277 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.277 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:40.277 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.278 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:776 3.ds0 3:b3f958c3:::smithi09235267-22:head [append 0~540672] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:40.278 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:20.788324+0000", 2024-04-26T03:39:40.278 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.398098423, 2024-04-26T03:39:40.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016617261000000001, 2024-04-26T03:39:40.278 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:40.278 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:40.278 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:40.278 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:40.278 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:40.278 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:40.278 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 776 2024-04-26T03:39:40.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.278 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:40.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:40.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.788324+0000", 2024-04-26T03:39:40.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:40.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.788324+0000", 2024-04-26T03:39:40.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:40.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.788325+0000", 2024-04-26T03:39:40.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7010000000000001e-06 2024-04-26T03:39:40.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:40.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.788621+0000", 2024-04-26T03:39:40.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029502800000000002 2024-04-26T03:39:40.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:40.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.788635+0000", 2024-04-26T03:39:40.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4741e-05 2024-04-26T03:39:40.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:40.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.788643+0000", 2024-04-26T03:39:40.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9610000000000005e-06 2024-04-26T03:39:40.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.795417+0000", 2024-04-26T03:39:40.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067738520000000003 2024-04-26T03:39:40.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:40.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.795523+0000", 2024-04-26T03:39:40.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010549599999999999 2024-04-26T03:39:40.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:40.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.796141+0000", 2024-04-26T03:39:40.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00061875999999999999 2024-04-26T03:39:40.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:40.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.801534+0000", 2024-04-26T03:39:40.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053927899999999997 2024-04-26T03:39:40.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:40.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.804742+0000", 2024-04-26T03:39:40.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003207522 2024-04-26T03:39:40.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:40.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.804941+0000", 2024-04-26T03:39:40.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019940999999999999 2024-04-26T03:39:40.282 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.283 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:40.283 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.283 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:778 3.ds0 3:b3f958c3:::smithi09235267-22:head [setxattr _header (57) in=7b] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:40.283 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:20.789998+0000", 2024-04-26T03:39:40.283 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.396424195000002, 2024-04-26T03:39:40.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015643300999999998, 2024-04-26T03:39:40.283 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:40.283 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:40.283 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:40.283 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:40.283 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:40.283 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:40.284 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 778 2024-04-26T03:39:40.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.284 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:40.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:40.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.789998+0000", 2024-04-26T03:39:40.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:40.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.789998+0000", 2024-04-26T03:39:40.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:40.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.789999+0000", 2024-04-26T03:39:40.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9800000000000003e-07 2024-04-26T03:39:40.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:40.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.790002+0000", 2024-04-26T03:39:40.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.597e-06 2024-04-26T03:39:40.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:40.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.790009+0000", 2024-04-26T03:39:40.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9700000000000002e-06 2024-04-26T03:39:40.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:40.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.790015+0000", 2024-04-26T03:39:40.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6350000000000003e-06 2024-04-26T03:39:40.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.801932+0000", 2024-04-26T03:39:40.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011917669000000001 2024-04-26T03:39:40.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:40.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.801990+0000", 2024-04-26T03:39:40.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7822000000000003e-05 2024-04-26T03:39:40.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:40.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.802279+0000", 2024-04-26T03:39:40.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028822199999999998 2024-04-26T03:39:40.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:40.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.803223+0000", 2024-04-26T03:39:40.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00094406100000000005 2024-04-26T03:39:40.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:40.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.805407+0000", 2024-04-26T03:39:40.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0021844360000000001 2024-04-26T03:39:40.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:40.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.805641+0000", 2024-04-26T03:39:40.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000234291 2024-04-26T03:39:40.288 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.288 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:40.288 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.288 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:779 3.ds0 3:b3f958c3:::smithi09235267-22:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:40.288 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:20.790023+0000", 2024-04-26T03:39:40.288 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.396399296999999, 2024-04-26T03:39:40.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017774486999999999, 2024-04-26T03:39:40.288 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:40.288 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:40.289 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:40.289 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:40.289 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:40.289 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:40.289 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 779 2024-04-26T03:39:40.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.289 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:40.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:40.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.790023+0000", 2024-04-26T03:39:40.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:40.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.790023+0000", 2024-04-26T03:39:40.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:40.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.790023+0000", 2024-04-26T03:39:40.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4200000000000001e-07 2024-04-26T03:39:40.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:40.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.790026+0000", 2024-04-26T03:39:40.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1159999999999999e-06 2024-04-26T03:39:40.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:40.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.790033+0000", 2024-04-26T03:39:40.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6100000000000002e-06 2024-04-26T03:39:40.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:40.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.790038+0000", 2024-04-26T03:39:40.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9439999999999996e-06 2024-04-26T03:39:40.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.802971+0000", 2024-04-26T03:39:40.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012933445 2024-04-26T03:39:40.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:40.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.803177+0000", 2024-04-26T03:39:40.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020553499999999999 2024-04-26T03:39:40.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.806365+0000", 2024-04-26T03:39:40.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003187615 2024-04-26T03:39:40.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:40.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.806453+0000", 2024-04-26T03:39:40.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8240999999999998e-05 2024-04-26T03:39:40.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:40.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.807797+0000", 2024-04-26T03:39:40.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013445390000000001 2024-04-26T03:39:40.293 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.293 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:40.293 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.293 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.10s2 80/48 ECSubWrite(tid=322, reqid=client.4556.0:895, at_version=80'35, trim_to=0'0, roll_forward_to=75'33))", 2024-04-26T03:39:40.293 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:35.740651+0000", 2024-04-26T03:39:40.293 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.4457716810000001, 2024-04-26T03:39:40.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.028220485999999999, 2024-04-26T03:39:40.293 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:40.294 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:40.294 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:40.294 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:40.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:40.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.740651+0000", 2024-04-26T03:39:40.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:40.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.740651+0000", 2024-04-26T03:39:40.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:40.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.740651+0000", 2024-04-26T03:39:40.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4699999999999998e-07 2024-04-26T03:39:40.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:40.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.745849+0000", 2024-04-26T03:39:40.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051984409999999998 2024-04-26T03:39:40.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:40.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.745862+0000", 2024-04-26T03:39:40.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2369e-05 2024-04-26T03:39:40.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:40.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.745868+0000", 2024-04-26T03:39:40.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6930000000000001e-06 2024-04-26T03:39:40.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.745920+0000", 2024-04-26T03:39:40.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1579000000000002e-05 2024-04-26T03:39:40.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:40.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.745928+0000", 2024-04-26T03:39:40.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0369999999999995e-06 2024-04-26T03:39:40.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:40.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.768824+0000", 2024-04-26T03:39:40.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022896162000000001 2024-04-26T03:39:40.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:40.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.768871+0000", 2024-04-26T03:39:40.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6958e-05 2024-04-26T03:39:40.297 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.297 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:40.297 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.297 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.297 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:40.297 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:40.299 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-04-26T03:39:40.300 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-04-26T03:39:40.297+0000 7f745118d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 31821) UID: 0 2024-04-26T03:39:40.399 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-04-26T03:39:40.398+0000 7fee562fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31759) UID: 0 2024-04-26T03:39:40.400 INFO:teuthology.orchestra.run.smithi028.stdout:osd.15: {} 2024-04-26T03:39:40.400 INFO:teuthology.orchestra.run.smithi028.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T03:39:40.444 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:40.444 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-26T03:39:40.444 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-26T03:39:40.444 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:40.453 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-04-26T03:39:40.500 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:39:40.498+0000 7fba0d2ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31961) UID: 0 2024-04-26T03:39:40.555 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:40.556 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-26T03:39:40.556 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-26T03:39:40.556 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-26T03:39:40.556 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:40.565 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-04-26T03:39:40.601 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-04-26T03:39:40.600+0000 7fee562fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31759) UID: 0 2024-04-26T03:39:40.668 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:40.669 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-26T03:39:40.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-26T03:39:40.669 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-26T03:39:40.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.669 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:12 3.cs0 3:3946a5f1:::smithi09235267-4:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:40.669 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.481722+0000", 2024-04-26T03:39:40.669 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 49.184472036000003, 2024-04-26T03:39:40.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039281273999999998, 2024-04-26T03:39:40.669 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:40.669 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:40.669 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:40.669 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:40.669 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:40.669 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:40.670 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 12 2024-04-26T03:39:40.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.670 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:40.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:40.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.481722+0000", 2024-04-26T03:39:40.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:40.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.481722+0000", 2024-04-26T03:39:40.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:40.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.481723+0000", 2024-04-26T03:39:40.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.75e-07 2024-04-26T03:39:40.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:40.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.481725+0000", 2024-04-26T03:39:40.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6979999999999998e-06 2024-04-26T03:39:40.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:40.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.481733+0000", 2024-04-26T03:39:40.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2899999999999997e-06 2024-04-26T03:39:40.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:40.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.481743+0000", 2024-04-26T03:39:40.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0845999999999999e-05 2024-04-26T03:39:40.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.484203+0000", 2024-04-26T03:39:40.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024598889999999998 2024-04-26T03:39:40.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:40.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.484353+0000", 2024-04-26T03:39:40.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015005199999999999 2024-04-26T03:39:40.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.497880+0000", 2024-04-26T03:39:40.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013526589 2024-04-26T03:39:40.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:40.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.497957+0000", 2024-04-26T03:39:40.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7126000000000007e-05 2024-04-26T03:39:40.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:40.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.521003+0000", 2024-04-26T03:39:40.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.023046309000000001 2024-04-26T03:39:40.674 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.674 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:40.674 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.674 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:31 3.cs0 3:3edefb40:::smithi09235267-11:head [append 0~180224,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:40.674 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.494104+0000", 2024-04-26T03:39:40.674 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 49.172089732000003, 2024-04-26T03:39:40.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025578053999999999, 2024-04-26T03:39:40.674 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:40.675 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:40.675 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:40.675 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:40.675 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:40.675 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:40.675 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 31 2024-04-26T03:39:40.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.675 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:40.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:40.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.494104+0000", 2024-04-26T03:39:40.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:40.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.494104+0000", 2024-04-26T03:39:40.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:40.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.494106+0000", 2024-04-26T03:39:40.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2100000000000001e-06 2024-04-26T03:39:40.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:40.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.494218+0000", 2024-04-26T03:39:40.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000112398 2024-04-26T03:39:40.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:40.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.494230+0000", 2024-04-26T03:39:40.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1688999999999999e-05 2024-04-26T03:39:40.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:40.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.494237+0000", 2024-04-26T03:39:40.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2049999999999996e-06 2024-04-26T03:39:40.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.494745+0000", 2024-04-26T03:39:40.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00050836900000000005 2024-04-26T03:39:40.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:40.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.494857+0000", 2024-04-26T03:39:40.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000112045 2024-04-26T03:39:40.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:40.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495764+0000", 2024-04-26T03:39:40.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00090692400000000003 2024-04-26T03:39:40.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:40.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519293+0000", 2024-04-26T03:39:40.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.023528830000000001 2024-04-26T03:39:40.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:40.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519597+0000", 2024-04-26T03:39:40.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030426100000000001 2024-04-26T03:39:40.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:40.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519682+0000", 2024-04-26T03:39:40.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5123000000000005e-05 2024-04-26T03:39:40.679 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.680 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:40.680 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.680 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:32 3.cs0 3:3edefb40:::smithi09235267-11:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:40.680 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.495390+0000", 2024-04-26T03:39:40.680 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 49.170803816000003, 2024-04-26T03:39:40.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027633438, 2024-04-26T03:39:40.680 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:40.680 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:40.680 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:40.680 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:40.680 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:40.681 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:40.681 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 32 2024-04-26T03:39:40.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.681 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:40.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:40.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495390+0000", 2024-04-26T03:39:40.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:40.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495390+0000", 2024-04-26T03:39:40.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:40.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495391+0000", 2024-04-26T03:39:40.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8999999999999995e-07 2024-04-26T03:39:40.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:40.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495396+0000", 2024-04-26T03:39:40.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4360000000000001e-06 2024-04-26T03:39:40.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:40.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495410+0000", 2024-04-26T03:39:40.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3482000000000001e-05 2024-04-26T03:39:40.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:40.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495419+0000", 2024-04-26T03:39:40.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9600000000000006e-06 2024-04-26T03:39:40.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518418+0000", 2024-04-26T03:39:40.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022999116999999999 2024-04-26T03:39:40.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:40.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518517+0000", 2024-04-26T03:39:40.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8542999999999995e-05 2024-04-26T03:39:40.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:40.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519020+0000", 2024-04-26T03:39:40.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00050364099999999998 2024-04-26T03:39:40.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:40.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519837+0000", 2024-04-26T03:39:40.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00081666699999999998 2024-04-26T03:39:40.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:40.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.522936+0000", 2024-04-26T03:39:40.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030991170000000002 2024-04-26T03:39:40.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:40.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.523024+0000", 2024-04-26T03:39:40.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7684999999999998e-05 2024-04-26T03:39:40.685 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.685 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:40.685 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.685 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:33 3.cs0 3:3edefb40:::smithi09235267-11:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:40.685 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.495433+0000", 2024-04-26T03:39:40.686 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 49.170760893999997, 2024-04-26T03:39:40.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032718572000000001, 2024-04-26T03:39:40.686 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:40.686 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:40.686 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:40.686 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:40.686 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:40.686 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:40.686 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 33 2024-04-26T03:39:40.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.686 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:40.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:40.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495433+0000", 2024-04-26T03:39:40.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:40.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495433+0000", 2024-04-26T03:39:40.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:40.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495434+0000", 2024-04-26T03:39:40.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2300000000000001e-07 2024-04-26T03:39:40.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:40.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495437+0000", 2024-04-26T03:39:40.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2399999999999999e-06 2024-04-26T03:39:40.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:40.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495444+0000", 2024-04-26T03:39:40.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3869999999999997e-06 2024-04-26T03:39:40.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:40.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495450+0000", 2024-04-26T03:39:40.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.412e-06 2024-04-26T03:39:40.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.520715+0000", 2024-04-26T03:39:40.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.02526488 2024-04-26T03:39:40.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:40.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.520783+0000", 2024-04-26T03:39:40.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8553000000000006e-05 2024-04-26T03:39:40.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.523315+0000", 2024-04-26T03:39:40.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025314299999999999 2024-04-26T03:39:40.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:40.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.523384+0000", 2024-04-26T03:39:40.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9503000000000002e-05 2024-04-26T03:39:40.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:40.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.528152+0000", 2024-04-26T03:39:40.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047676439999999997 2024-04-26T03:39:40.690 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.690 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:40.691 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.691 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.cs0 35/32 ECSubReadReply(tid=5, attrs_read=0))", 2024-04-26T03:39:40.691 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.498822+0000", 2024-04-26T03:39:40.691 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 49.16737225, 2024-04-26T03:39:40.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021016639, 2024-04-26T03:39:40.691 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:40.691 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:40.691 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:40.691 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:40.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:40.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498822+0000", 2024-04-26T03:39:40.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:40.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498822+0000", 2024-04-26T03:39:40.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:40.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498822+0000", 2024-04-26T03:39:40.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3099999999999997e-07 2024-04-26T03:39:40.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:40.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498834+0000", 2024-04-26T03:39:40.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1331e-05 2024-04-26T03:39:40.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:40.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498878+0000", 2024-04-26T03:39:40.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4092999999999999e-05 2024-04-26T03:39:40.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:40.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498888+0000", 2024-04-26T03:39:40.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0049e-05 2024-04-26T03:39:40.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519785+0000", 2024-04-26T03:39:40.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020896656 2024-04-26T03:39:40.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:40.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519838+0000", 2024-04-26T03:39:40.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3878999999999997e-05 2024-04-26T03:39:40.694 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.694 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:40.694 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.695 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 35/32 ECSubWriteReply(tid=4, last_complete=35'3, committed=1, applied=1))", 2024-04-26T03:39:40.695 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.499205+0000", 2024-04-26T03:39:40.695 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 49.166989243000003, 2024-04-26T03:39:40.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020089540999999999, 2024-04-26T03:39:40.695 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:40.695 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:40.695 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:40.695 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:40.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:40.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499205+0000", 2024-04-26T03:39:40.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.696 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:40.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499205+0000", 2024-04-26T03:39:40.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.696 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:40.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499205+0000", 2024-04-26T03:39:40.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5199999999999996e-07 2024-04-26T03:39:40.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.696 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:40.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499210+0000", 2024-04-26T03:39:40.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0690000000000003e-06 2024-04-26T03:39:40.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:40.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499223+0000", 2024-04-26T03:39:40.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3783e-05 2024-04-26T03:39:40.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:40.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499230+0000", 2024-04-26T03:39:40.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7970000000000003e-06 2024-04-26T03:39:40.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519273+0000", 2024-04-26T03:39:40.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020043068000000001 2024-04-26T03:39:40.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:40.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519294+0000", 2024-04-26T03:39:40.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1172e-05 2024-04-26T03:39:40.698 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.698 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:40.698 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.698 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.cs0 35/32 ECSubReadReply(tid=5, attrs_read=0))", 2024-04-26T03:39:40.698 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.500849+0000", 2024-04-26T03:39:40.699 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 49.165345393999999, 2024-04-26T03:39:40.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019810352, 2024-04-26T03:39:40.699 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:40.699 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:40.699 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:40.699 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:40.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:40.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500849+0000", 2024-04-26T03:39:40.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:40.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500849+0000", 2024-04-26T03:39:40.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.700 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:40.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500849+0000", 2024-04-26T03:39:40.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9999999999999999e-07 2024-04-26T03:39:40.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.700 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:40.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500860+0000", 2024-04-26T03:39:40.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0811e-05 2024-04-26T03:39:40.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.700 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:40.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500867+0000", 2024-04-26T03:39:40.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9670000000000004e-06 2024-04-26T03:39:40.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.701 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:40.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500871+0000", 2024-04-26T03:39:40.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3109999999999999e-06 2024-04-26T03:39:40.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.701 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.520619+0000", 2024-04-26T03:39:40.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019747602 2024-04-26T03:39:40.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:40.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.520659+0000", 2024-04-26T03:39:40.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0361000000000002e-05 2024-04-26T03:39:40.702 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.702 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:40.702 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.702 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:604 3.13s0 3:cad73b03:::smithi09235267-37:head [copy-from ver 50] snapc d=[d,c,b,a] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-26T03:39:40.702 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:11.710211+0000", 2024-04-26T03:39:40.702 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.955983338999999, 2024-04-26T03:39:40.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.082214682999999997, 2024-04-26T03:39:40.702 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:40.702 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:40.702 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:40.703 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:40.703 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:40.703 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:40.703 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 604 2024-04-26T03:39:40.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.703 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:40.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:40.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.710211+0000", 2024-04-26T03:39:40.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:40.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.710211+0000", 2024-04-26T03:39:40.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:40.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.710211+0000", 2024-04-26T03:39:40.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9100000000000004e-07 2024-04-26T03:39:40.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:40.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.710218+0000", 2024-04-26T03:39:40.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6880000000000002e-06 2024-04-26T03:39:40.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:40.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.710225+0000", 2024-04-26T03:39:40.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2019999999999998e-06 2024-04-26T03:39:40.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.705 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:40.705 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.710232+0000", 2024-04-26T03:39:40.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8059999999999997e-06 2024-04-26T03:39:40.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.705 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.705 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.712246+0000", 2024-04-26T03:39:40.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020136189999999999 2024-04-26T03:39:40.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.705 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-26T03:39:40.705 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.712248+0000", 2024-04-26T03:39:40.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1179999999999999e-06 2024-04-26T03:39:40.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.706 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.706 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.743111+0000", 2024-04-26T03:39:40.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.030862865999999999 2024-04-26T03:39:40.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.706 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for missing object", 2024-04-26T03:39:40.706 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.743951+0000", 2024-04-26T03:39:40.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00084010900000000002 2024-04-26T03:39:40.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.706 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.759857+0000", 2024-04-26T03:39:40.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015906699 2024-04-26T03:39:40.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:40.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.760063+0000", 2024-04-26T03:39:40.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020573299999999999 2024-04-26T03:39:40.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:40.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.774271+0000", 2024-04-26T03:39:40.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014208346 2024-04-26T03:39:40.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:40.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.790920+0000", 2024-04-26T03:39:40.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016649048 2024-04-26T03:39:40.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:40.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.791986+0000", 2024-04-26T03:39:40.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010652299999999999 2024-04-26T03:39:40.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:40.709 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.792425+0000", 2024-04-26T03:39:40.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00043972800000000002 2024-04-26T03:39:40.709 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.709 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:40.709 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.709 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:605 3.13s0 3:cad73b03:::smithi09235267-37:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-04-26T03:39:40.709 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:11.710239+0000", 2024-04-26T03:39:40.709 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.955954684000002, 2024-04-26T03:39:40.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080074244000000003, 2024-04-26T03:39:40.709 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:40.709 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:40.709 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:40.710 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:40.710 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:40.710 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:40.710 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 605 2024-04-26T03:39:40.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.710 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:40.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:40.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.710239+0000", 2024-04-26T03:39:40.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:40.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.710239+0000", 2024-04-26T03:39:40.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:40.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.710240+0000", 2024-04-26T03:39:40.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.32e-07 2024-04-26T03:39:40.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:40.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.710243+0000", 2024-04-26T03:39:40.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5319999999999998e-06 2024-04-26T03:39:40.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:40.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.710250+0000", 2024-04-26T03:39:40.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5889999999999999e-06 2024-04-26T03:39:40.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:40.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.710256+0000", 2024-04-26T03:39:40.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5670000000000002e-06 2024-04-26T03:39:40.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.712312+0000", 2024-04-26T03:39:40.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020565190000000001 2024-04-26T03:39:40.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-26T03:39:40.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.712314+0000", 2024-04-26T03:39:40.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1880000000000001e-06 2024-04-26T03:39:40.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.746793+0000", 2024-04-26T03:39:40.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034479224000000003 2024-04-26T03:39:40.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for missing object", 2024-04-26T03:39:40.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.746907+0000", 2024-04-26T03:39:40.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113739 2024-04-26T03:39:40.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.762064+0000", 2024-04-26T03:39:40.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015157096 2024-04-26T03:39:40.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.714 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for blocked object", 2024-04-26T03:39:40.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.762162+0000", 2024-04-26T03:39:40.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8133000000000004e-05 2024-04-26T03:39:40.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.714 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.789483+0000", 2024-04-26T03:39:40.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027321055 2024-04-26T03:39:40.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:40.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.789695+0000", 2024-04-26T03:39:40.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021135500000000001 2024-04-26T03:39:40.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:40.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.790314+0000", 2024-04-26T03:39:40.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00061881499999999999 2024-04-26T03:39:40.715 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.715 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:40.715 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.715 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:619 3.13s0 3:c944ba3d:::smithi09235267-6:a [read 0~4669440,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-04-26T03:39:40.716 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:11.732770+0000", 2024-04-26T03:39:40.716 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.933424334000001, 2024-04-26T03:39:40.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.12874891899999999, 2024-04-26T03:39:40.716 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:40.716 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:40.716 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:40.716 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:40.716 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:40.716 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:40.716 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 619 2024-04-26T03:39:40.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.716 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:40.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:40.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732770+0000", 2024-04-26T03:39:40.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:40.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732770+0000", 2024-04-26T03:39:40.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:40.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732772+0000", 2024-04-26T03:39:40.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.266e-06 2024-04-26T03:39:40.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:40.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732781+0000", 2024-04-26T03:39:40.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0350000000000007e-06 2024-04-26T03:39:40.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:40.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732803+0000", 2024-04-26T03:39:40.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2140000000000001e-05 2024-04-26T03:39:40.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:40.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732815+0000", 2024-04-26T03:39:40.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1637000000000001e-05 2024-04-26T03:39:40.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732928+0000", 2024-04-26T03:39:40.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000112717 2024-04-26T03:39:40.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-26T03:39:40.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732936+0000", 2024-04-26T03:39:40.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5439999999999999e-06 2024-04-26T03:39:40.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.746980+0000", 2024-04-26T03:39:40.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014043959999999999 2024-04-26T03:39:40.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for missing object", 2024-04-26T03:39:40.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.748272+0000", 2024-04-26T03:39:40.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0012919139999999999 2024-04-26T03:39:40.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.852799+0000", 2024-04-26T03:39:40.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.104527289 2024-04-26T03:39:40.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:40.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.852915+0000", 2024-04-26T03:39:40.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115395 2024-04-26T03:39:40.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:40.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.861519+0000", 2024-04-26T03:39:40.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0086040219999999994 2024-04-26T03:39:40.721 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.721 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:40.721 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.721 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:620 3.13s0 3:c944ba3d:::smithi09235267-6:a [read 0~4669440,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-04-26T03:39:40.721 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:11.732848+0000", 2024-04-26T03:39:40.721 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.933345924000001, 2024-04-26T03:39:40.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.13189979199999999, 2024-04-26T03:39:40.721 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:40.721 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:40.722 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:40.722 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:40.722 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:40.722 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:40.722 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 620 2024-04-26T03:39:40.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.722 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:40.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:40.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732848+0000", 2024-04-26T03:39:40.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:40.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732848+0000", 2024-04-26T03:39:40.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:40.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732849+0000", 2024-04-26T03:39:40.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9800000000000003e-07 2024-04-26T03:39:40.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:40.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732860+0000", 2024-04-26T03:39:40.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0942e-05 2024-04-26T03:39:40.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:40.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732869+0000", 2024-04-26T03:39:40.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1800000000000002e-06 2024-04-26T03:39:40.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:40.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732880+0000", 2024-04-26T03:39:40.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1021e-05 2024-04-26T03:39:40.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732987+0000", 2024-04-26T03:39:40.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107095 2024-04-26T03:39:40.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-26T03:39:40.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732989+0000", 2024-04-26T03:39:40.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1919999999999999e-06 2024-04-26T03:39:40.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.748328+0000", 2024-04-26T03:39:40.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015338312 2024-04-26T03:39:40.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for missing object", 2024-04-26T03:39:40.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.748430+0000", 2024-04-26T03:39:40.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102669 2024-04-26T03:39:40.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.853182+0000", 2024-04-26T03:39:40.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10475184799999999 2024-04-26T03:39:40.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:40.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.853309+0000", 2024-04-26T03:39:40.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012699099999999999 2024-04-26T03:39:40.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:40.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.864748+0000", 2024-04-26T03:39:40.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011438844 2024-04-26T03:39:40.726 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.727 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:40.727 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.727 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:621 3.13s0 3:c944ba3d:::smithi09235267-6:a [read 0~4669440,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-04-26T03:39:40.727 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:11.732920+0000", 2024-04-26T03:39:40.727 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.933273887999999, 2024-04-26T03:39:40.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.134463005, 2024-04-26T03:39:40.727 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:40.727 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:40.727 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:40.727 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:40.727 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:40.727 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:40.728 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 621 2024-04-26T03:39:40.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.728 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:40.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:40.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732920+0000", 2024-04-26T03:39:40.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:40.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732920+0000", 2024-04-26T03:39:40.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:40.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732921+0000", 2024-04-26T03:39:40.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5400000000000001e-07 2024-04-26T03:39:40.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:40.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732925+0000", 2024-04-26T03:39:40.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0180000000000003e-06 2024-04-26T03:39:40.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:40.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732938+0000", 2024-04-26T03:39:40.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2928999999999999e-05 2024-04-26T03:39:40.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:40.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.733001+0000", 2024-04-26T03:39:40.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3621999999999995e-05 2024-04-26T03:39:40.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.733061+0000", 2024-04-26T03:39:40.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9905000000000003e-05 2024-04-26T03:39:40.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-26T03:39:40.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.733063+0000", 2024-04-26T03:39:40.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3990000000000001e-06 2024-04-26T03:39:40.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.749893+0000", 2024-04-26T03:39:40.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016830254999999999 2024-04-26T03:39:40.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for missing object", 2024-04-26T03:39:40.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.750023+0000", 2024-04-26T03:39:40.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000130264 2024-04-26T03:39:40.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.853768+0000", 2024-04-26T03:39:40.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.103745171 2024-04-26T03:39:40.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:40.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.853937+0000", 2024-04-26T03:39:40.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016834 2024-04-26T03:39:40.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:40.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.867383+0000", 2024-04-26T03:39:40.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013446448 2024-04-26T03:39:40.732 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.732 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:40.732 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.732 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDPGPushReply(3.13s0 56/55 [PushReplyOp(3:c944ba3d:::smithi09235267-6:head)])", 2024-04-26T03:39:40.733 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:11.768451+0000", 2024-04-26T03:39:40.733 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.897743547000001, 2024-04-26T03:39:40.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.023073083000000001, 2024-04-26T03:39:40.733 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:40.733 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:40.733 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:40.733 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:40.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:40.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.768451+0000", 2024-04-26T03:39:40.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:40.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.768451+0000", 2024-04-26T03:39:40.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:40.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.768451+0000", 2024-04-26T03:39:40.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7400000000000002e-07 2024-04-26T03:39:40.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:40.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.768460+0000", 2024-04-26T03:39:40.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0450000000000006e-06 2024-04-26T03:39:40.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:40.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.768471+0000", 2024-04-26T03:39:40.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0776e-05 2024-04-26T03:39:40.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:40.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.768482+0000", 2024-04-26T03:39:40.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1131999999999999e-05 2024-04-26T03:39:40.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.791177+0000", 2024-04-26T03:39:40.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022694269 2024-04-26T03:39:40.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:40.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.791524+0000", 2024-04-26T03:39:40.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00034708699999999999 2024-04-26T03:39:40.736 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.736 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:40.736 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.736 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDPGPushReply(3.13s0 56/55 [PushReplyOp(3:c944ba3d:::smithi09235267-6:head)])", 2024-04-26T03:39:40.736 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:11.768915+0000", 2024-04-26T03:39:40.736 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.897279451999999, 2024-04-26T03:39:40.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022008744, 2024-04-26T03:39:40.736 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:40.736 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:40.736 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:40.736 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:40.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:40.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.768915+0000", 2024-04-26T03:39:40.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:40.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.768915+0000", 2024-04-26T03:39:40.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:40.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.768915+0000", 2024-04-26T03:39:40.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0600000000000001e-07 2024-04-26T03:39:40.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:40.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.768920+0000", 2024-04-26T03:39:40.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9740000000000001e-06 2024-04-26T03:39:40.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:40.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.768928+0000", 2024-04-26T03:39:40.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2609999999999993e-06 2024-04-26T03:39:40.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:40.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.768941+0000", 2024-04-26T03:39:40.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2480000000000001e-05 2024-04-26T03:39:40.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.790794+0000", 2024-04-26T03:39:40.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021853289000000001 2024-04-26T03:39:40.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:40.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.790923+0000", 2024-04-26T03:39:40.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012933399999999999 2024-04-26T03:39:40.739 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.739 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:40.740 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.740 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDPGPushReply(3.13s0 56/55 [PushReplyOp(3:c944ba3d:::smithi09235267-6:head)])", 2024-04-26T03:39:40.740 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:11.769116+0000", 2024-04-26T03:39:40.740 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.897078475000001, 2024-04-26T03:39:40.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021803037000000001, 2024-04-26T03:39:40.740 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:40.740 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:40.740 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:40.740 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:40.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.740 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:40.740 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769116+0000", 2024-04-26T03:39:40.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:40.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769116+0000", 2024-04-26T03:39:40.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:40.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769116+0000", 2024-04-26T03:39:40.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9400000000000005e-07 2024-04-26T03:39:40.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:40.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769120+0000", 2024-04-26T03:39:40.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0740000000000003e-06 2024-04-26T03:39:40.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:40.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769136+0000", 2024-04-26T03:39:40.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5532e-05 2024-04-26T03:39:40.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:40.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769144+0000", 2024-04-26T03:39:40.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7549999999999994e-06 2024-04-26T03:39:40.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.790869+0000", 2024-04-26T03:39:40.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021725661 2024-04-26T03:39:40.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:40.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.790919+0000", 2024-04-26T03:39:40.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9320999999999998e-05 2024-04-26T03:39:40.743 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.743 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:40.743 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.744 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDPGPushReply(3.13s0 56/55 [PushReplyOp(3:c944ba3d:::smithi09235267-6:head)])", 2024-04-26T03:39:40.744 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:11.769260+0000", 2024-04-26T03:39:40.744 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.896934139999999, 2024-04-26T03:39:40.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021509337999999999, 2024-04-26T03:39:40.744 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:40.744 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:40.744 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:40.744 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:40.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:40.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769260+0000", 2024-04-26T03:39:40.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:40.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769260+0000", 2024-04-26T03:39:40.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:40.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769260+0000", 2024-04-26T03:39:40.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9200000000000002e-07 2024-04-26T03:39:40.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:40.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769266+0000", 2024-04-26T03:39:40.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2024-04-26T03:39:40.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:40.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769274+0000", 2024-04-26T03:39:40.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3289999999999996e-06 2024-04-26T03:39:40.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:40.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769279+0000", 2024-04-26T03:39:40.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3600000000000004e-06 2024-04-26T03:39:40.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.790713+0000", 2024-04-26T03:39:40.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021433612000000001 2024-04-26T03:39:40.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:40.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.790769+0000", 2024-04-26T03:39:40.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6536999999999998e-05 2024-04-26T03:39:40.747 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.747 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:40.747 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.747 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDPGPushReply(3.13s0 56/55 [PushReplyOp(3:c944ba3d:::smithi09235267-6:head)])", 2024-04-26T03:39:40.747 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:11.769267+0000", 2024-04-26T03:39:40.747 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.896927333000001, 2024-04-26T03:39:40.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021747525, 2024-04-26T03:39:40.748 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:40.748 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:40.748 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:40.748 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:40.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:40.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769267+0000", 2024-04-26T03:39:40.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:40.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769267+0000", 2024-04-26T03:39:40.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:40.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769268+0000", 2024-04-26T03:39:40.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9400000000000004e-07 2024-04-26T03:39:40.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:40.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769271+0000", 2024-04-26T03:39:40.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5090000000000002e-06 2024-04-26T03:39:40.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:40.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769285+0000", 2024-04-26T03:39:40.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3566e-05 2024-04-26T03:39:40.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:40.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769290+0000", 2024-04-26T03:39:40.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8420000000000001e-06 2024-04-26T03:39:40.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.790968+0000", 2024-04-26T03:39:40.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021677923000000002 2024-04-26T03:39:40.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:40.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.791014+0000", 2024-04-26T03:39:40.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5890999999999998e-05 2024-04-26T03:39:40.751 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.751 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:40.751 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.751 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:710 3.13s0 3:c8283ddb:::smithi09235267-32:head [append 0~573440] snapc f=[f,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-04-26T03:39:40.751 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:15.970859+0000", 2024-04-26T03:39:40.751 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.695335552, 2024-04-26T03:39:40.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019548756, 2024-04-26T03:39:40.751 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:40.751 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:40.751 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:40.752 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:40.752 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:40.752 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:40.752 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 710 2024-04-26T03:39:40.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.752 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:40.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:40.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.970859+0000", 2024-04-26T03:39:40.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:40.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.970859+0000", 2024-04-26T03:39:40.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:40.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.970860+0000", 2024-04-26T03:39:40.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9649999999999998e-06 2024-04-26T03:39:40.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:40.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.971054+0000", 2024-04-26T03:39:40.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000193413 2024-04-26T03:39:40.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:40.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.971071+0000", 2024-04-26T03:39:40.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.694e-05 2024-04-26T03:39:40.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:40.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.971080+0000", 2024-04-26T03:39:40.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.414e-06 2024-04-26T03:39:40.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.971155+0000", 2024-04-26T03:39:40.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4482999999999998e-05 2024-04-26T03:39:40.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:40.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.971280+0000", 2024-04-26T03:39:40.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012498899999999999 2024-04-26T03:39:40.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:40.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.971925+0000", 2024-04-26T03:39:40.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000645003 2024-04-26T03:39:40.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:40.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.982227+0000", 2024-04-26T03:39:40.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01030181 2024-04-26T03:39:40.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:40.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.990344+0000", 2024-04-26T03:39:40.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0081176319999999996 2024-04-26T03:39:40.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:40.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.990407+0000", 2024-04-26T03:39:40.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3107000000000004e-05 2024-04-26T03:39:40.756 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.756 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:40.756 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.756 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:782 3.fs0 3:f81f96ba:::smithi09235267-46:head [append 0~540672] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:40.756 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:20.821620+0000", 2024-04-26T03:39:40.757 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.844573656000001, 2024-04-26T03:39:40.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020779720000000002, 2024-04-26T03:39:40.757 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:40.757 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:40.757 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:40.757 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:40.757 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:40.757 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:40.757 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 782 2024-04-26T03:39:40.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.757 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:40.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:40.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.821620+0000", 2024-04-26T03:39:40.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:40.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.821620+0000", 2024-04-26T03:39:40.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:40.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.821621+0000", 2024-04-26T03:39:40.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6200000000000006e-07 2024-04-26T03:39:40.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:40.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.821861+0000", 2024-04-26T03:39:40.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000240063 2024-04-26T03:39:40.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:40.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.821873+0000", 2024-04-26T03:39:40.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1525999999999999e-05 2024-04-26T03:39:40.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:40.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.821882+0000", 2024-04-26T03:39:40.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9290000000000004e-06 2024-04-26T03:39:40.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.828072+0000", 2024-04-26T03:39:40.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061898659999999996 2024-04-26T03:39:40.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:40.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.828171+0000", 2024-04-26T03:39:40.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9437000000000004e-05 2024-04-26T03:39:40.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:40.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.828799+0000", 2024-04-26T03:39:40.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00062785900000000003 2024-04-26T03:39:40.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:40.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.841267+0000", 2024-04-26T03:39:40.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012467914 2024-04-26T03:39:40.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:40.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.842350+0000", 2024-04-26T03:39:40.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010830099999999999 2024-04-26T03:39:40.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:40.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.842400+0000", 2024-04-26T03:39:40.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0154000000000001e-05 2024-04-26T03:39:40.762 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.762 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:40.762 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.762 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.10s6 80/48 ECSubWrite(tid=322, reqid=client.4556.0:895, at_version=80'35, trim_to=0'0, roll_forward_to=75'33))", 2024-04-26T03:39:40.762 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:35.741160+0000", 2024-04-26T03:39:40.762 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.925033666, 2024-04-26T03:39:40.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029831673999999999, 2024-04-26T03:39:40.762 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:40.762 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:40.762 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:40.762 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:40.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:40.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.741160+0000", 2024-04-26T03:39:40.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:40.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.741160+0000", 2024-04-26T03:39:40.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:40.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:40.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.741161+0000", 2024-04-26T03:39:40.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4999999999999998e-07 2024-04-26T03:39:40.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:40.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.745641+0000", 2024-04-26T03:39:40.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00447978 2024-04-26T03:39:40.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:40.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.745652+0000", 2024-04-26T03:39:40.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1596999999999999e-05 2024-04-26T03:39:40.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:40.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.745675+0000", 2024-04-26T03:39:40.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2960999999999999e-05 2024-04-26T03:39:40.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:40.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.745736+0000", 2024-04-26T03:39:40.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1106999999999996e-05 2024-04-26T03:39:40.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:40.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.745747+0000", 2024-04-26T03:39:40.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0190000000000001e-05 2024-04-26T03:39:40.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:40.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.770911+0000", 2024-04-26T03:39:40.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025164907 2024-04-26T03:39:40.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:40.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:40.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:40.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.770992+0000", 2024-04-26T03:39:40.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0681999999999998e-05 2024-04-26T03:39:40.766 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.766 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:40.766 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.766 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:40.766 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:40.766 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:40.767 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-26T03:39:40.768 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:39:40.767+0000 7f60d3c4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31757) UID: 0 2024-04-26T03:39:40.769 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-04-26T03:39:40.804 INFO:teuthology.orchestra.run.smithi028.stderr:reweighted osd.12 to 1 (10000) 2024-04-26T03:39:40.807 INFO:tasks.rados.rados.0.smithi092.stdout:374: done (3 left) 2024-04-26T03:39:40.807 INFO:tasks.rados.rados.0.smithi092.stdout:376: done (2 left) 2024-04-26T03:39:40.807 INFO:tasks.rados.rados.0.smithi092.stdout:378: done (1 left) 2024-04-26T03:39:40.807 INFO:tasks.rados.rados.0.smithi092.stdout:379: done (0 left) 2024-04-26T03:39:40.807 INFO:tasks.rados.rados.0.smithi092.stdout:380: delete oid 6 current snap is 30 2024-04-26T03:39:40.815 INFO:tasks.rados.rados.0.smithi092.stdout:380: done (0 left) 2024-04-26T03:39:40.816 INFO:tasks.rados.rados.0.smithi092.stdout:381: read oid 12 snap -1 2024-04-26T03:39:40.816 INFO:tasks.rados.rados.0.smithi092.stdout:381: expect (ObjNum 133 snap 17 seq_num 133) 2024-04-26T03:39:40.816 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 13 current snap is 30 2024-04-26T03:39:40.816 INFO:tasks.rados.rados.0.smithi092.stdout:382: seq_num 165 ranges {6356992=376832} 2024-04-26T03:39:40.820 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 13 1 2024-04-26T03:39:40.852 INFO:tasks.rados.rados.0.smithi092.stdout:382: writing smithi09235267-13 from 6356992 to 6733824 tid 1 2024-04-26T03:39:40.852 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 34 current snap is 30 2024-04-26T03:39:40.852 INFO:tasks.rados.rados.0.smithi092.stdout:383: seq_num 166 ranges {0=393216} 2024-04-26T03:39:40.854 INFO:tasks.rados.rados.0.smithi092.stdout:383: writing smithi09235267-34 from 0 to 393216 tid 1 2024-04-26T03:39:40.854 INFO:tasks.rados.rados.0.smithi092.stdout:384: copy_from oid 36 from oid 8 current snap is 30 2024-04-26T03:39:40.854 INFO:tasks.rados.rados.0.smithi092.stdout:385: rollback oid 21 current snap is 30 2024-04-26T03:39:40.854 INFO:tasks.rados.rados.0.smithi092.stdout:rollback oid 21 to 24 2024-04-26T03:39:40.854 INFO:tasks.rados.rados.0.smithi092.stdout:381: expect (ObjNum 133 snap 17 seq_num 133) 2024-04-26T03:39:40.868 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:39:40.867+0000 7fb78c544640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31956) UID: 0 2024-04-26T03:39:40.880 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:40.880 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-26T03:39:40.880 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-26T03:39:40.880 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:40.889 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-04-26T03:39:40.928 INFO:tasks.rados.rados.0.smithi092.stdout:382: finishing write tid 1 to smithi09235267-13 2024-04-26T03:39:40.928 INFO:tasks.rados.rados.0.smithi092.stdout:382: finishing write tid 2 to smithi09235267-13 2024-04-26T03:39:40.928 INFO:tasks.rados.rados.0.smithi092.stdout:382: finishing write tid 3 to smithi09235267-13 2024-04-26T03:39:40.928 INFO:tasks.rados.rados.0.smithi092.stdout:382: oid 13 updating version 0 to 126 2024-04-26T03:39:40.929 INFO:tasks.rados.rados.0.smithi092.stdout:382: oid 13 version 126 is already newer than 125 2024-04-26T03:39:40.929 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 13 v 126 (ObjNum 165 snap 30 seq_num 165) dirty exists 2024-04-26T03:39:40.929 INFO:tasks.rados.rados.0.smithi092.stdout:382: left oid 13 (ObjNum 165 snap 30 seq_num 165) 2024-04-26T03:39:40.929 INFO:tasks.rados.rados.0.smithi092.stdout:385: finishing rollback tid 0 to smithi09235267-21 2024-04-26T03:39:40.929 INFO:tasks.rados.rados.0.smithi092.stdout:385: finishing rollback tid 1 to smithi09235267-21 2024-04-26T03:39:40.929 INFO:tasks.rados.rados.0.smithi092.stdout:385: finishing rollback tid 2 to smithi09235267-21 2024-04-26T03:39:40.929 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 21 v 130 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-04-26T03:39:40.929 INFO:tasks.rados.rados.0.smithi092.stdout:384: finishing copy_from racing read to smithi09235267-36 2024-04-26T03:39:40.929 INFO:tasks.rados.rados.0.smithi092.stdout:383: finishing write tid 1 to smithi09235267-34 2024-04-26T03:39:40.929 INFO:tasks.rados.rados.0.smithi092.stdout:381: done (4 left) 2024-04-26T03:39:40.930 INFO:tasks.rados.rados.0.smithi092.stdout:382: done (3 left) 2024-04-26T03:39:40.930 INFO:tasks.rados.rados.0.smithi092.stdout:385: done (2 left) 2024-04-26T03:39:40.930 INFO:tasks.rados.rados.0.smithi092.stdout:386: delete oid 1 current snap is 30 2024-04-26T03:39:40.930 INFO:tasks.rados.rados.0.smithi092.stdout:383: finishing write tid 2 to smithi09235267-34 2024-04-26T03:39:40.930 INFO:tasks.rados.rados.0.smithi092.stdout:384: finishing copy_from to smithi09235267-36 2024-04-26T03:39:40.930 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 36 v 6 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-04-26T03:39:40.930 INFO:tasks.rados.rados.0.smithi092.stdout:383: finishing write tid 3 to smithi09235267-34 2024-04-26T03:39:40.930 INFO:tasks.rados.rados.0.smithi092.stdout:383: oid 34 updating version 0 to 81 2024-04-26T03:39:40.930 INFO:tasks.rados.rados.0.smithi092.stdout:383: oid 34 updating version 81 to 82 2024-04-26T03:39:40.930 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 34 v 82 (ObjNum 166 snap 30 seq_num 166) dirty exists 2024-04-26T03:39:40.930 INFO:tasks.rados.rados.0.smithi092.stdout:383: left oid 34 (ObjNum 166 snap 30 seq_num 166) 2024-04-26T03:39:40.973 INFO:tasks.ceph.osd.15.smithi092.stderr:2024-04-26T03:39:40.972+0000 7f6d12dec640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 31824) UID: 0 2024-04-26T03:39:41.032 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:41.033 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-26T03:39:41.033 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-26T03:39:41.033 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-26T03:39:41.033 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:41.041 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-04-26T03:39:41.070 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-04-26T03:39:41.069+0000 7fe9223c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31954) UID: 0 2024-04-26T03:39:41.170 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-26T03:39:41.169+0000 7f610b3a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31816) UID: 0 2024-04-26T03:39:41.185 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:41.185 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-26T03:39:41.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-26T03:39:41.185 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-26T03:39:41.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.186 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:30 3.5s0 3:a64abe59:::smithi09235267-10:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:41.186 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.491045+0000", 2024-04-26T03:39:41.186 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 49.692046056999999, 2024-04-26T03:39:41.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.024868352, 2024-04-26T03:39:41.186 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:41.186 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:41.187 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:41.187 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:41.187 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:41.187 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:41.187 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 30 2024-04-26T03:39:41.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.187 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:41.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:41.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.491045+0000", 2024-04-26T03:39:41.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:41.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:41.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.491045+0000", 2024-04-26T03:39:41.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:41.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:41.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.491045+0000", 2024-04-26T03:39:41.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3799999999999997e-07 2024-04-26T03:39:41.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:41.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.491049+0000", 2024-04-26T03:39:41.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3359999999999999e-06 2024-04-26T03:39:41.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:41.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.491056+0000", 2024-04-26T03:39:41.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7549999999999994e-06 2024-04-26T03:39:41.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:41.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.491062+0000", 2024-04-26T03:39:41.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3730000000000001e-06 2024-04-26T03:39:41.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:41.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.493431+0000", 2024-04-26T03:39:41.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023688490000000001 2024-04-26T03:39:41.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:41.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.493515+0000", 2024-04-26T03:39:41.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4342e-05 2024-04-26T03:39:41.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:41.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499490+0000", 2024-04-26T03:39:41.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059748809999999996 2024-04-26T03:39:41.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:41.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499562+0000", 2024-04-26T03:39:41.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1942999999999999e-05 2024-04-26T03:39:41.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:41.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515913+0000", 2024-04-26T03:39:41.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016351335000000002 2024-04-26T03:39:41.191 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:41.191 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:41.191 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:41.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.192 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.5s0 35/32 ECSubReadReply(tid=3, attrs_read=0))", 2024-04-26T03:39:41.192 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.500548+0000", 2024-04-26T03:39:41.192 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 49.682542849000001, 2024-04-26T03:39:41.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014243212999999999, 2024-04-26T03:39:41.192 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:41.192 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:41.192 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:41.192 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:41.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:41.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500548+0000", 2024-04-26T03:39:41.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:41.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:41.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500548+0000", 2024-04-26T03:39:41.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:41.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:41.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500549+0000", 2024-04-26T03:39:41.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3300000000000002e-07 2024-04-26T03:39:41.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:41.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500560+0000", 2024-04-26T03:39:41.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1316e-05 2024-04-26T03:39:41.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:41.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500573+0000", 2024-04-26T03:39:41.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2853999999999999e-05 2024-04-26T03:39:41.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:41.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500579+0000", 2024-04-26T03:39:41.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3090000000000001e-06 2024-04-26T03:39:41.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:41.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.514618+0000", 2024-04-26T03:39:41.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014038861 2024-04-26T03:39:41.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:41.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.514791+0000", 2024-04-26T03:39:41.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017323999999999999 2024-04-26T03:39:41.195 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:41.195 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:41.196 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:41.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.196 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.5s0 35/32 ECSubReadReply(tid=3, attrs_read=0))", 2024-04-26T03:39:41.196 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.500609+0000", 2024-04-26T03:39:41.196 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 49.682481697999997, 2024-04-26T03:39:41.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014706643, 2024-04-26T03:39:41.196 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:41.196 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:41.196 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:41.196 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:41.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:41.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500609+0000", 2024-04-26T03:39:41.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:41.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:41.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500609+0000", 2024-04-26T03:39:41.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:41.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:41.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500610+0000", 2024-04-26T03:39:41.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1900000000000002e-07 2024-04-26T03:39:41.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:41.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500621+0000", 2024-04-26T03:39:41.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1102e-05 2024-04-26T03:39:41.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:41.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500631+0000", 2024-04-26T03:39:41.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0197000000000001e-05 2024-04-26T03:39:41.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:41.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500638+0000", 2024-04-26T03:39:41.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3770000000000001e-06 2024-04-26T03:39:41.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:41.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515285+0000", 2024-04-26T03:39:41.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014647004 2024-04-26T03:39:41.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:41.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515316+0000", 2024-04-26T03:39:41.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1244000000000001e-05 2024-04-26T03:39:41.200 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:41.200 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:41.200 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:41.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.200 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 35/32 ECSubWriteReply(tid=2, last_complete=34'2, committed=1, applied=1))", 2024-04-26T03:39:41.200 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.501067+0000", 2024-04-26T03:39:41.200 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 49.682024249999998, 2024-04-26T03:39:41.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013961199000000001, 2024-04-26T03:39:41.200 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:41.200 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:41.200 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:41.200 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:41.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:41.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501067+0000", 2024-04-26T03:39:41.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:41.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:41.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501067+0000", 2024-04-26T03:39:41.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:41.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:41.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501067+0000", 2024-04-26T03:39:41.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7599999999999996e-07 2024-04-26T03:39:41.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:41.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501072+0000", 2024-04-26T03:39:41.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7700000000000001e-06 2024-04-26T03:39:41.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:41.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501083+0000", 2024-04-26T03:39:41.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0652000000000001e-05 2024-04-26T03:39:41.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:41.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501090+0000", 2024-04-26T03:39:41.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5299999999999999e-06 2024-04-26T03:39:41.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:41.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515013+0000", 2024-04-26T03:39:41.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013922436999999999 2024-04-26T03:39:41.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:41.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515028+0000", 2024-04-26T03:39:41.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5034e-05 2024-04-26T03:39:41.203 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:41.203 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:41.203 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:41.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.204 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.5s0 35/32 ECSubReadReply(tid=3, attrs_read=0))", 2024-04-26T03:39:41.204 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.501907+0000", 2024-04-26T03:39:41.204 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 49.681183433000001, 2024-04-26T03:39:41.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013462257, 2024-04-26T03:39:41.204 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:41.204 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:41.204 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:41.204 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:41.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:41.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501907+0000", 2024-04-26T03:39:41.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:41.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:41.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501907+0000", 2024-04-26T03:39:41.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:41.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:41.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501908+0000", 2024-04-26T03:39:41.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2200000000000003e-07 2024-04-26T03:39:41.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:41.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501918+0000", 2024-04-26T03:39:41.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0329999999999999e-05 2024-04-26T03:39:41.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:41.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501945+0000", 2024-04-26T03:39:41.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6047999999999999e-05 2024-04-26T03:39:41.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:41.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501951+0000", 2024-04-26T03:39:41.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7560000000000002e-06 2024-04-26T03:39:41.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:41.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515339+0000", 2024-04-26T03:39:41.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013387731 2024-04-26T03:39:41.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:41.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515370+0000", 2024-04-26T03:39:41.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0670000000000003e-05 2024-04-26T03:39:41.207 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:41.208 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:41.208 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:41.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.208 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:67 3.5s0 3:a79a63ac:::smithi09235267-23:head [append 0~376832,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:41.208 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.518499+0000", 2024-04-26T03:39:41.208 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 49.664591555999998, 2024-04-26T03:39:41.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01578007, 2024-04-26T03:39:41.208 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:41.208 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:41.208 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:41.208 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:41.208 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:41.209 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:41.209 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 67 2024-04-26T03:39:41.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.209 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:41.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:41.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518499+0000", 2024-04-26T03:39:41.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:41.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:41.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518499+0000", 2024-04-26T03:39:41.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:41.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:41.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518501+0000", 2024-04-26T03:39:41.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9010000000000001e-06 2024-04-26T03:39:41.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:41.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518725+0000", 2024-04-26T03:39:41.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000224183 2024-04-26T03:39:41.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:41.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518740+0000", 2024-04-26T03:39:41.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4966e-05 2024-04-26T03:39:41.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:41.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518748+0000", 2024-04-26T03:39:41.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6399999999999997e-06 2024-04-26T03:39:41.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:41.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518798+0000", 2024-04-26T03:39:41.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0263000000000003e-05 2024-04-26T03:39:41.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:41.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518927+0000", 2024-04-26T03:39:41.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012898900000000001 2024-04-26T03:39:41.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:41.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519605+0000", 2024-04-26T03:39:41.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00067774899999999995 2024-04-26T03:39:41.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:41.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.524447+0000", 2024-04-26T03:39:41.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0048424319999999998 2024-04-26T03:39:41.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:41.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.534195+0000", 2024-04-26T03:39:41.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097473149999999995 2024-04-26T03:39:41.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:41.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.534279+0000", 2024-04-26T03:39:41.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4631999999999999e-05 2024-04-26T03:39:41.214 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:41.214 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:41.214 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:41.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.214 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:68 3.5s0 3:a79a63ac:::smithi09235267-23:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:41.214 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.518766+0000", 2024-04-26T03:39:41.214 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 49.664324635, 2024-04-26T03:39:41.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015709772, 2024-04-26T03:39:41.214 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:41.214 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:41.215 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:41.215 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:41.215 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:41.215 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:41.215 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 68 2024-04-26T03:39:41.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.215 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:41.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:41.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518766+0000", 2024-04-26T03:39:41.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:41.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:41.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518766+0000", 2024-04-26T03:39:41.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:41.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:41.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518767+0000", 2024-04-26T03:39:41.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6899999999999997e-07 2024-04-26T03:39:41.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:41.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518771+0000", 2024-04-26T03:39:41.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-06 2024-04-26T03:39:41.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:41.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518777+0000", 2024-04-26T03:39:41.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3600000000000004e-06 2024-04-26T03:39:41.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:41.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518783+0000", 2024-04-26T03:39:41.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9490000000000001e-06 2024-04-26T03:39:41.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:41.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.522888+0000", 2024-04-26T03:39:41.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004105577 2024-04-26T03:39:41.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:41.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.522976+0000", 2024-04-26T03:39:41.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400999999999999e-05 2024-04-26T03:39:41.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:41.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.523380+0000", 2024-04-26T03:39:41.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00040403599999999998 2024-04-26T03:39:41.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:41.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.524449+0000", 2024-04-26T03:39:41.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00106897 2024-04-26T03:39:41.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:41.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.534390+0000", 2024-04-26T03:39:41.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099415249999999997 2024-04-26T03:39:41.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:41.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.534476+0000", 2024-04-26T03:39:41.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5884999999999995e-05 2024-04-26T03:39:41.220 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:41.220 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:41.220 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:41.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.220 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:69 3.5s0 3:a79a63ac:::smithi09235267-23:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:41.220 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.518790+0000", 2024-04-26T03:39:41.220 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 49.664300402999999, 2024-04-26T03:39:41.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018453847999999998, 2024-04-26T03:39:41.221 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:41.221 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:41.221 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:41.221 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:41.221 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:41.221 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:41.221 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 69 2024-04-26T03:39:41.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.221 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:41.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:41.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518790+0000", 2024-04-26T03:39:41.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:41.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:41.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518790+0000", 2024-04-26T03:39:41.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:41.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:41.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518791+0000", 2024-04-26T03:39:41.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9999999999999998e-07 2024-04-26T03:39:41.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:41.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518793+0000", 2024-04-26T03:39:41.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0949999999999998e-06 2024-04-26T03:39:41.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:41.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518799+0000", 2024-04-26T03:39:41.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.117e-06 2024-04-26T03:39:41.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:41.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518804+0000", 2024-04-26T03:39:41.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8670000000000002e-06 2024-04-26T03:39:41.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:41.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.523575+0000", 2024-04-26T03:39:41.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004771155 2024-04-26T03:39:41.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:41.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.523646+0000", 2024-04-26T03:39:41.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0992000000000001e-05 2024-04-26T03:39:41.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:41.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.534812+0000", 2024-04-26T03:39:41.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011166099000000001 2024-04-26T03:39:41.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:41.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.534887+0000", 2024-04-26T03:39:41.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5094999999999998e-05 2024-04-26T03:39:41.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:41.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.537244+0000", 2024-04-26T03:39:41.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023569279999999999 2024-04-26T03:39:41.226 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:41.226 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:41.226 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:41.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.226 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:256 3.5s0 3:a64abe59:::smithi09235267-10:head [append 0~0] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T03:39:41.226 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:56.508641+0000", 2024-04-26T03:39:41.226 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 44.674449932000002, 2024-04-26T03:39:41.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013570576000000001, 2024-04-26T03:39:41.226 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:41.226 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:41.226 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:41.226 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:41.227 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:41.227 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:41.227 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 256 2024-04-26T03:39:41.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.227 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:41.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:41.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508641+0000", 2024-04-26T03:39:41.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:41.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:41.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508641+0000", 2024-04-26T03:39:41.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:41.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:41.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508644+0000", 2024-04-26T03:39:41.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7760000000000002e-06 2024-04-26T03:39:41.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:41.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508655+0000", 2024-04-26T03:39:41.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1049e-05 2024-04-26T03:39:41.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:41.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508727+0000", 2024-04-26T03:39:41.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1907000000000001e-05 2024-04-26T03:39:41.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:41.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508737+0000", 2024-04-26T03:39:41.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0691e-05 2024-04-26T03:39:41.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:41.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508889+0000", 2024-04-26T03:39:41.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015123099999999999 2024-04-26T03:39:41.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-04-26T03:39:41.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508905+0000", 2024-04-26T03:39:41.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6042999999999999e-05 2024-04-26T03:39:41.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:41.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.511938+0000", 2024-04-26T03:39:41.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030333119999999998 2024-04-26T03:39:41.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:41.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.512054+0000", 2024-04-26T03:39:41.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115742 2024-04-26T03:39:41.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:41.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.512811+0000", 2024-04-26T03:39:41.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00075763100000000004 2024-04-26T03:39:41.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:41.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.521838+0000", 2024-04-26T03:39:41.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0090266489999999994 2024-04-26T03:39:41.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:41.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.522144+0000", 2024-04-26T03:39:41.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030574600000000002 2024-04-26T03:39:41.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:41.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.522211+0000", 2024-04-26T03:39:41.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7799000000000006e-05 2024-04-26T03:39:41.232 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:41.232 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:41.232 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:41.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.233 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:257 3.5s0 3:a64abe59:::smithi09235267-10:head [rollback 2] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T03:39:41.233 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:56.508783+0000", 2024-04-26T03:39:41.233 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 44.674307912000003, 2024-04-26T03:39:41.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014341474999999999, 2024-04-26T03:39:41.233 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:41.233 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:41.233 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:41.233 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:41.233 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:41.233 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:41.233 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 257 2024-04-26T03:39:41.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.233 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:41.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:41.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508783+0000", 2024-04-26T03:39:41.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:41.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:41.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508783+0000", 2024-04-26T03:39:41.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:41.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:41.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508784+0000", 2024-04-26T03:39:41.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9999999999999995e-07 2024-04-26T03:39:41.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:41.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508789+0000", 2024-04-26T03:39:41.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4280000000000004e-06 2024-04-26T03:39:41.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:41.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508800+0000", 2024-04-26T03:39:41.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.076e-05 2024-04-26T03:39:41.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:41.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508808+0000", 2024-04-26T03:39:41.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.593e-06 2024-04-26T03:39:41.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:41.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508993+0000", 2024-04-26T03:39:41.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018527999999999999 2024-04-26T03:39:41.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-04-26T03:39:41.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.509001+0000", 2024-04-26T03:39:41.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6529999999999994e-06 2024-04-26T03:39:41.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:41.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.513192+0000", 2024-04-26T03:39:41.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041913009999999997 2024-04-26T03:39:41.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:41.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.513348+0000", 2024-04-26T03:39:41.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000155654 2024-04-26T03:39:41.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:41.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.514509+0000", 2024-04-26T03:39:41.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011615169999999999 2024-04-26T03:39:41.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:41.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.521845+0000", 2024-04-26T03:39:41.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0073361140000000003 2024-04-26T03:39:41.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:41.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.523044+0000", 2024-04-26T03:39:41.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001198939 2024-04-26T03:39:41.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:41.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.523124+0000", 2024-04-26T03:39:41.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0235999999999995e-05 2024-04-26T03:39:41.239 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:41.239 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:41.239 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:41.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.239 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:258 3.5s0 3:a64abe59:::smithi09235267-10:head [append 0~0] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T03:39:41.239 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:56.508917+0000", 2024-04-26T03:39:41.240 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 44.674174067999999, 2024-04-26T03:39:41.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014562304999999999, 2024-04-26T03:39:41.240 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:41.240 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:41.240 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:41.240 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:41.240 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:41.240 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:41.240 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 258 2024-04-26T03:39:41.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.240 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:41.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:41.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508917+0000", 2024-04-26T03:39:41.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:41.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:41.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508917+0000", 2024-04-26T03:39:41.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:41.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:41.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508918+0000", 2024-04-26T03:39:41.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5600000000000004e-07 2024-04-26T03:39:41.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:41.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508922+0000", 2024-04-26T03:39:41.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4730000000000002e-06 2024-04-26T03:39:41.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:41.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508937+0000", 2024-04-26T03:39:41.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5114e-05 2024-04-26T03:39:41.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:41.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508948+0000", 2024-04-26T03:39:41.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1222e-05 2024-04-26T03:39:41.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:41.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.509035+0000", 2024-04-26T03:39:41.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7032000000000003e-05 2024-04-26T03:39:41.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-04-26T03:39:41.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.509040+0000", 2024-04-26T03:39:41.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6299999999999997e-06 2024-04-26T03:39:41.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:41.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.515348+0000", 2024-04-26T03:39:41.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0063082959999999997 2024-04-26T03:39:41.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:41.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.515613+0000", 2024-04-26T03:39:41.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026497100000000002 2024-04-26T03:39:41.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:41.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.516295+0000", 2024-04-26T03:39:41.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00068196499999999998 2024-04-26T03:39:41.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:41.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.521977+0000", 2024-04-26T03:39:41.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056815759999999998 2024-04-26T03:39:41.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:41.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.523407+0000", 2024-04-26T03:39:41.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014299409999999999 2024-04-26T03:39:41.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:41.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.523479+0000", 2024-04-26T03:39:41.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2229000000000003e-05 2024-04-26T03:39:41.246 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:41.246 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:41.246 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:41.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.246 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:318 3.5s0 3:a79a63ac:::smithi09235267-23:head [delete] snapc 7=[7,6,5] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e44)", 2024-04-26T03:39:41.246 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:00.528087+0000", 2024-04-26T03:39:41.246 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.655003379, 2024-04-26T03:39:41.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.020004624, 2024-04-26T03:39:41.246 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:41.246 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:41.246 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:41.246 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:41.247 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:41.247 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:41.247 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 318 2024-04-26T03:39:41.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.247 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:41.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:41.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:00.528087+0000", 2024-04-26T03:39:41.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:41.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:41.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:00.528087+0000", 2024-04-26T03:39:41.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:41.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:41.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:00.528089+0000", 2024-04-26T03:39:41.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9120000000000001e-06 2024-04-26T03:39:41.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:41.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:00.528096+0000", 2024-04-26T03:39:41.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0829999999999998e-06 2024-04-26T03:39:41.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:41.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:00.528112+0000", 2024-04-26T03:39:41.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5619000000000001e-05 2024-04-26T03:39:41.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:41.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:00.528121+0000", 2024-04-26T03:39:41.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6789999999999994e-06 2024-04-26T03:39:41.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:41.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:00.528185+0000", 2024-04-26T03:39:41.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4156e-05 2024-04-26T03:39:41.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-04-26T03:39:41.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:00.528190+0000", 2024-04-26T03:39:41.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9010000000000002e-06 2024-04-26T03:39:41.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:41.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:00.534892+0000", 2024-04-26T03:39:41.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067026239999999999 2024-04-26T03:39:41.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-26T03:39:41.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:00.534897+0000", 2024-04-26T03:39:41.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9389999999999996e-06 2024-04-26T03:39:41.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:41.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.540918+0000", 2024-04-26T03:39:41.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0060203679999999 2024-04-26T03:39:41.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:41.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.541113+0000", 2024-04-26T03:39:41.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019529200000000001 2024-04-26T03:39:41.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:41.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.541689+0000", 2024-04-26T03:39:41.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00057578299999999996 2024-04-26T03:39:41.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:41.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.544502+0000", 2024-04-26T03:39:41.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028131950000000001 2024-04-26T03:39:41.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:41.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.547870+0000", 2024-04-26T03:39:41.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0033683039999999999 2024-04-26T03:39:41.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:41.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.548092+0000", 2024-04-26T03:39:41.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000221769 2024-04-26T03:39:41.253 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:41.253 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:41.253 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:41.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.253 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.13s3 50/44 ECSubWrite(tid=74, reqid=client.4556.0:446, at_version=50'37, trim_to=0'0, roll_forward_to=50'36))", 2024-04-26T03:39:41.253 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:05.702348+0000", 2024-04-26T03:39:41.253 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.480742782999997, 2024-04-26T03:39:41.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013733293000000001, 2024-04-26T03:39:41.253 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:41.253 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:41.253 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:41.253 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:41.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:41.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.702348+0000", 2024-04-26T03:39:41.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:41.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:41.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.702348+0000", 2024-04-26T03:39:41.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:41.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:41.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.702349+0000", 2024-04-26T03:39:41.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5400000000000001e-07 2024-04-26T03:39:41.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:41.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.702837+0000", 2024-04-26T03:39:41.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00048857299999999996 2024-04-26T03:39:41.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:41.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.702851+0000", 2024-04-26T03:39:41.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3723999999999999e-05 2024-04-26T03:39:41.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:41.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.702862+0000", 2024-04-26T03:39:41.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1571e-05 2024-04-26T03:39:41.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:41.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.702925+0000", 2024-04-26T03:39:41.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2890000000000003e-05 2024-04-26T03:39:41.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:41.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.702942+0000", 2024-04-26T03:39:41.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6857000000000001e-05 2024-04-26T03:39:41.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:41.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716045+0000", 2024-04-26T03:39:41.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013102744 2024-04-26T03:39:41.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:41.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716081+0000", 2024-04-26T03:39:41.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6380000000000001e-05 2024-04-26T03:39:41.257 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:41.257 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:41.257 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:41.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.257 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:462 3.5s0 3:a64abe59:::smithi09235267-10:head [append 0~753664] snapc 8=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:41.258 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:05.749661+0000", 2024-04-26T03:39:41.258 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.433429873999998, 2024-04-26T03:39:41.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014030855, 2024-04-26T03:39:41.258 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:41.258 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:41.258 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:41.258 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:41.258 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:41.258 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:41.258 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 462 2024-04-26T03:39:41.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.258 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:41.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:41.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.749661+0000", 2024-04-26T03:39:41.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:41.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:41.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.749661+0000", 2024-04-26T03:39:41.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:41.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:41.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.749663+0000", 2024-04-26T03:39:41.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.722e-06 2024-04-26T03:39:41.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:41.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.749971+0000", 2024-04-26T03:39:41.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000308475 2024-04-26T03:39:41.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:41.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.749985+0000", 2024-04-26T03:39:41.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4191e-05 2024-04-26T03:39:41.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:41.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.749995+0000", 2024-04-26T03:39:41.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0020000000000001e-05 2024-04-26T03:39:41.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:41.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.750050+0000", 2024-04-26T03:39:41.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5000000000000002e-05 2024-04-26T03:39:41.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:41.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.750121+0000", 2024-04-26T03:39:41.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0669999999999999e-05 2024-04-26T03:39:41.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:41.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.750769+0000", 2024-04-26T03:39:41.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00064844000000000002 2024-04-26T03:39:41.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:41.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.763254+0000", 2024-04-26T03:39:41.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012484215999999999 2024-04-26T03:39:41.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:41.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.763622+0000", 2024-04-26T03:39:41.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00036791699999999999 2024-04-26T03:39:41.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:41.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.763692+0000", 2024-04-26T03:39:41.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0203999999999994e-05 2024-04-26T03:39:41.263 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:41.263 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:41.263 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:41.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.263 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:466 3.5s0 3:a64abe59:::smithi09235267-10:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:41.263 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:05.755663+0000", 2024-04-26T03:39:41.263 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.427427377000001, 2024-04-26T03:39:41.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021406392999999999, 2024-04-26T03:39:41.263 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:41.263 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:41.263 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:41.264 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:41.264 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:41.264 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:41.264 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 466 2024-04-26T03:39:41.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.264 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:41.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:41.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755663+0000", 2024-04-26T03:39:41.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:41.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:41.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755663+0000", 2024-04-26T03:39:41.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:41.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:41.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755664+0000", 2024-04-26T03:39:41.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6600000000000003e-07 2024-04-26T03:39:41.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:41.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755665+0000", 2024-04-26T03:39:41.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.389e-06 2024-04-26T03:39:41.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:41.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755669+0000", 2024-04-26T03:39:41.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2024-04-26T03:39:41.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:41.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755672+0000", 2024-04-26T03:39:41.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7190000000000001e-06 2024-04-26T03:39:41.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:41.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.764511+0000", 2024-04-26T03:39:41.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0088397730000000004 2024-04-26T03:39:41.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:41.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.764598+0000", 2024-04-26T03:39:41.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6438000000000001e-05 2024-04-26T03:39:41.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:41.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.766650+0000", 2024-04-26T03:39:41.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020521430000000002 2024-04-26T03:39:41.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:41.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.766698+0000", 2024-04-26T03:39:41.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8133000000000001e-05 2024-04-26T03:39:41.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:41.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.777070+0000", 2024-04-26T03:39:41.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010371672 2024-04-26T03:39:41.268 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:41.268 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:41.268 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:41.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.269 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:467 3.5s0 3:a79a63ac:::smithi09235267-23:head [stat,delete] snapc 8=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:41.269 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:05.755675+0000", 2024-04-26T03:39:41.269 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.427415621999998, 2024-04-26T03:39:41.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020820686000000001, 2024-04-26T03:39:41.269 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:41.269 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:41.269 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:41.269 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:41.269 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:41.269 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:41.269 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 467 2024-04-26T03:39:41.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.269 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:41.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:41.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755675+0000", 2024-04-26T03:39:41.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:41.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:41.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755675+0000", 2024-04-26T03:39:41.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:41.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:41.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755675+0000", 2024-04-26T03:39:41.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0900000000000001e-07 2024-04-26T03:39:41.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:41.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755677+0000", 2024-04-26T03:39:41.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.406e-06 2024-04-26T03:39:41.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:41.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755680+0000", 2024-04-26T03:39:41.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2329999999999998e-06 2024-04-26T03:39:41.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:41.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755684+0000", 2024-04-26T03:39:41.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.472e-06 2024-04-26T03:39:41.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:41.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.764647+0000", 2024-04-26T03:39:41.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0089629900000000005 2024-04-26T03:39:41.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:41.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.764702+0000", 2024-04-26T03:39:41.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5748000000000002e-05 2024-04-26T03:39:41.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:41.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.765004+0000", 2024-04-26T03:39:41.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030170000000000002 2024-04-26T03:39:41.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:41.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.776390+0000", 2024-04-26T03:39:41.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011386454000000001 2024-04-26T03:39:41.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:41.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.776455+0000", 2024-04-26T03:39:41.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4304e-05 2024-04-26T03:39:41.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:41.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.776496+0000", 2024-04-26T03:39:41.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1170000000000001e-05 2024-04-26T03:39:41.274 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:41.274 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:41.274 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:41.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.274 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:615 3.5s0 3:a79a63ac:::smithi09235267-23:head [create,setxattr Z]ca\\SkkXhsYi]s_mecUZQfPbbjaqacZrSObr_`a (734) in=40b,setxattr i (1998) in=1b,setxattr OQosgYUVOPebZdfit`Xtcc]XX_pdNqdkSof_lrck (1822) in=40b,setxattr mrQaYTllqfj^q]UYaoR`drdgY]bVfYQ\\SgOe_gca (1789) in=40b,setxattr ZaeONfchPlXaOYt[dQNtaR[NNiaSangpfQNVhVY[ (1646) in=40b,setxattr epskr[spqr^j^XqcosqPQ (1998) in=21b,setxattr WarqdipUlkSQhTbQR[NhrYRrda[QmOgj^VqWWero (313) in=40b,setxattr f\\fnrpSa`gWXj`Yo`dQY]Vdr[Z^`gkVjb^enmUig (1645) in=40b,setxattr bjWaN_RafX`the (1998) in=14b,setxattr nmdNljVWb`aR_`Oeb[l_OntR_YgidWjRabRf[]Ye (10596) in=40b,setxattr [nhisfmkVd_n_YV\\efebsW_pQaiWle_]o]rhc[bO (14456) in=40b,setxattr []hmkY]V]hkrrd]gr\\c\\hPq_smnTfakPZRNg^bY[ (1649) in=40b,setxattr ZZ\\\\jZnU`jtY\\[k]hnm`r_OYQeWTg\\rnQdX^rbhg (807) in=40b,setxattr QT_d^gX[dNZPP\\VapoetR^[gpVcZQXfhp^`l_Vh[ (66) in=40b,setxattr ee[ (1998) in=3b,setxattr RY\\sp]UifT[Zt\\[VenNWarqiiNSXsdnktf^QhT[U (487) in=40b,setxattr hUnl\\fnak]U``QN^rs`a^kfaQ`NoTVNnPXYbdmre (13680) in=40b,setxattr gha[S`afeloOcnfbeOkUV]T]ZTYWaOeaS]bbVa\\Q (1226) in=40b,setxattr NaraXbinjWQraPh`Pp]qnPWgWfZf[WaUb]oOY]^r (1611) in=40b,setxattr tpfZiddi]a^RVWpgOZormUeqYosdY_UsSpiQQRgj (1225) in=40b] snapc d=[d,c,b,a] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-26T03:39:41.274 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:11.732685+0000", 2024-04-26T03:39:41.274 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 29.450405514, 2024-04-26T03:39:41.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013378048999999999, 2024-04-26T03:39:41.275 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:41.275 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:41.275 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:41.275 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:41.275 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:41.275 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:41.275 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 615 2024-04-26T03:39:41.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.275 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:41.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:41.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732685+0000", 2024-04-26T03:39:41.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:41.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:41.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732685+0000", 2024-04-26T03:39:41.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:41.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:41.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732688+0000", 2024-04-26T03:39:41.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8320000000000002e-06 2024-04-26T03:39:41.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:41.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732753+0000", 2024-04-26T03:39:41.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4678999999999995e-05 2024-04-26T03:39:41.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:41.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732774+0000", 2024-04-26T03:39:41.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1637999999999998e-05 2024-04-26T03:39:41.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:41.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732784+0000", 2024-04-26T03:39:41.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5410000000000006e-06 2024-04-26T03:39:41.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:41.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732856+0000", 2024-04-26T03:39:41.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2020000000000005e-05 2024-04-26T03:39:41.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:41.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.733002+0000", 2024-04-26T03:39:41.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000145697 2024-04-26T03:39:41.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:41.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.733773+0000", 2024-04-26T03:39:41.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000770774 2024-04-26T03:39:41.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:41.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.735028+0000", 2024-04-26T03:39:41.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0012551089999999999 2024-04-26T03:39:41.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:41.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.745960+0000", 2024-04-26T03:39:41.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010932737999999999 2024-04-26T03:39:41.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:41.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.746063+0000", 2024-04-26T03:39:41.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103021 2024-04-26T03:39:41.279 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:41.279 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:41.280 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:41.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.280 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:818 3.5s0 3:a64abe59:::smithi09235267-10:head [read 0~2605056,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-04-26T03:39:41.280 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:22.692385+0000", 2024-04-26T03:39:41.280 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.490706216, 2024-04-26T03:39:41.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018880436, 2024-04-26T03:39:41.280 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:41.280 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:41.280 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:41.280 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:41.280 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:41.280 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:41.280 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 818 2024-04-26T03:39:41.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.281 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:41.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:41.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692385+0000", 2024-04-26T03:39:41.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:41.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:41.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692385+0000", 2024-04-26T03:39:41.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:41.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:41.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692385+0000", 2024-04-26T03:39:41.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2900000000000003e-07 2024-04-26T03:39:41.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:41.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692390+0000", 2024-04-26T03:39:41.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7380000000000002e-06 2024-04-26T03:39:41.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:41.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692400+0000", 2024-04-26T03:39:41.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5880000000000004e-06 2024-04-26T03:39:41.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:41.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692406+0000", 2024-04-26T03:39:41.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5529999999999998e-06 2024-04-26T03:39:41.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:41.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692508+0000", 2024-04-26T03:39:41.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101698 2024-04-26T03:39:41.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-04-26T03:39:41.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692515+0000", 2024-04-26T03:39:41.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1629999999999999e-06 2024-04-26T03:39:41.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:41.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.697431+0000", 2024-04-26T03:39:41.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049163280000000002 2024-04-26T03:39:41.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:41.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.697602+0000", 2024-04-26T03:39:41.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017039999999999999 2024-04-26T03:39:41.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:41.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.711265+0000", 2024-04-26T03:39:41.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013663339 2024-04-26T03:39:41.284 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:41.284 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:41.285 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:41.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.285 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:819 3.5s0 3:a64abe59:::smithi09235267-10:head [read 0~2605056,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-04-26T03:39:41.285 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:22.692428+0000", 2024-04-26T03:39:41.285 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.490663175000002, 2024-04-26T03:39:41.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020633177999999999, 2024-04-26T03:39:41.285 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:41.285 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:41.285 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:41.285 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:41.285 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:41.285 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:41.285 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 819 2024-04-26T03:39:41.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.286 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:41.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:41.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692428+0000", 2024-04-26T03:39:41.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:41.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:41.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692428+0000", 2024-04-26T03:39:41.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:41.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:41.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692428+0000", 2024-04-26T03:39:41.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2024-04-26T03:39:41.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:41.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692432+0000", 2024-04-26T03:39:41.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6320000000000001e-06 2024-04-26T03:39:41.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:41.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692437+0000", 2024-04-26T03:39:41.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.164e-06 2024-04-26T03:39:41.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:41.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692441+0000", 2024-04-26T03:39:41.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7459999999999998e-06 2024-04-26T03:39:41.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:41.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692537+0000", 2024-04-26T03:39:41.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6175000000000007e-05 2024-04-26T03:39:41.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-04-26T03:39:41.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692541+0000", 2024-04-26T03:39:41.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6279999999999998e-06 2024-04-26T03:39:41.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:41.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.699526+0000", 2024-04-26T03:39:41.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0069851940000000001 2024-04-26T03:39:41.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:41.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.699637+0000", 2024-04-26T03:39:41.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000111437 2024-04-26T03:39:41.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:41.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.713061+0000", 2024-04-26T03:39:41.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013423554000000001 2024-04-26T03:39:41.290 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:41.290 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:41.290 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:41.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.290 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.8s5 80/63 ECSubWrite(tid=321, reqid=client.4556.0:897, at_version=80'111, trim_to=0'0, roll_forward_to=80'110))", 2024-04-26T03:39:41.290 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:35.717514+0000", 2024-04-26T03:39:41.290 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.4655765199999999, 2024-04-26T03:39:41.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018440938, 2024-04-26T03:39:41.290 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:41.290 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:41.290 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:41.291 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:41.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:41.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.717514+0000", 2024-04-26T03:39:41.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:41.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:41.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.717514+0000", 2024-04-26T03:39:41.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:41.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:41.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.717515+0000", 2024-04-26T03:39:41.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1900000000000002e-07 2024-04-26T03:39:41.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:41.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.719503+0000", 2024-04-26T03:39:41.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019876220000000001 2024-04-26T03:39:41.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:41.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.719513+0000", 2024-04-26T03:39:41.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0638e-05 2024-04-26T03:39:41.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:41.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.719519+0000", 2024-04-26T03:39:41.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9190000000000004e-06 2024-04-26T03:39:41.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:41.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.719572+0000", 2024-04-26T03:39:41.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3158999999999997e-05 2024-04-26T03:39:41.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:41.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.719579+0000", 2024-04-26T03:39:41.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4420000000000003e-06 2024-04-26T03:39:41.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:41.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.735911+0000", 2024-04-26T03:39:41.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016332583000000001 2024-04-26T03:39:41.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:41.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:41.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:41.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.735955+0000", 2024-04-26T03:39:41.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3856000000000002e-05 2024-04-26T03:39:41.294 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:41.294 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:41.294 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:41.294 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:41.294 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:41.294 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:41.296 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-26T03:39:41.297 INFO:tasks.ceph.osd.14.smithi066.stderr:2024-04-26T03:39:41.295+0000 7f5395f08640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 31765) UID: 0 2024-04-26T03:39:41.396 INFO:tasks.ceph.osd.10.smithi066.stderr:2024-04-26T03:39:41.395+0000 7f130fcc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 31763) UID: 0 2024-04-26T03:39:41.404 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:39:41.405 INFO:teuthology.orchestra.run.smithi028.stdout: "ops": [], 2024-04-26T03:39:41.405 INFO:teuthology.orchestra.run.smithi028.stdout: "num_ops": 0 2024-04-26T03:39:41.405 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:39:41.413 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-26T03:39:41.497 INFO:tasks.ceph.osd.9.smithi042.stderr:2024-04-26T03:39:41.496+0000 7fdac63b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 31960) UID: 0 2024-04-26T03:39:41.520 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:39:41.520 INFO:teuthology.orchestra.run.smithi028.stdout: "ops": [], 2024-04-26T03:39:41.520 INFO:teuthology.orchestra.run.smithi028.stdout: "complaint_time": 30, 2024-04-26T03:39:41.520 INFO:teuthology.orchestra.run.smithi028.stdout: "num_blocked_ops": 0 2024-04-26T03:39:41.520 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:39:41.530 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-26T03:39:41.598 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:39:41.596+0000 7fba0d2ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31961) UID: 0 2024-04-26T03:39:41.638 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:39:41.639 INFO:teuthology.orchestra.run.smithi028.stdout: "size": 20, 2024-04-26T03:39:41.639 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 600, 2024-04-26T03:39:41.639 INFO:teuthology.orchestra.run.smithi028.stdout: "ops": [ 2024-04-26T03:39:41.639 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.639 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(osd.3.24:1 3.6s0 3:64627763:::smithi09235267-49:head [assert-version v3,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-04-26T03:39:41.639 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:59.539567+0000", 2024-04-26T03:39:41.639 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 42.096523634, 2024-04-26T03:39:41.639 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0073782170000000003, 2024-04-26T03:39:41.639 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:41.639 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:41.639 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "started", 2024-04-26T03:39:41.639 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:41.639 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.639 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:41.640 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.539567+0000", 2024-04-26T03:39:41.640 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:41.640 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.640 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.640 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:41.640 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.539567+0000", 2024-04-26T03:39:41.640 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:41.640 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.640 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.640 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:41.640 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.539569+0000", 2024-04-26T03:39:41.640 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.9240000000000001e-06 2024-04-26T03:39:41.640 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.640 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.641 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:41.641 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.539575+0000", 2024-04-26T03:39:41.641 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.4949999999999999e-06 2024-04-26T03:39:41.641 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.641 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.641 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:41.641 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.539585+0000", 2024-04-26T03:39:41.641 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0084e-05 2024-04-26T03:39:41.641 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.641 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.641 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:41.641 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.539592+0000", 2024-04-26T03:39:41.641 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.3350000000000001e-06 2024-04-26T03:39:41.642 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.642 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.642 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:41.642 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.539651+0000", 2024-04-26T03:39:41.642 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.8789000000000002e-05 2024-04-26T03:39:41.642 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.642 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.642 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:41.642 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.539738+0000", 2024-04-26T03:39:41.642 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.6734e-05 2024-04-26T03:39:41.642 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.642 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.642 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:41.643 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:59.546946+0000", 2024-04-26T03:39:41.643 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0072078560000000003 2024-04-26T03:39:41.643 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:41.643 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:41.643 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:41.643 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.643 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.643 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.bs1 45/32 ECSubWrite(tid=68, reqid=client.4556.0:361, at_version=45'31, trim_to=0'0, roll_forward_to=43'30))", 2024-04-26T03:39:41.643 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:01.715207+0000", 2024-04-26T03:39:41.643 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 39.920884037, 2024-04-26T03:39:41.643 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0081499150000000006, 2024-04-26T03:39:41.643 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:41.643 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:41.644 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:41.644 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:41.644 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.644 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:41.644 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.715207+0000", 2024-04-26T03:39:41.644 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:41.644 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.644 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.644 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:41.644 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.715207+0000", 2024-04-26T03:39:41.644 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:41.644 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.644 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.645 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:41.645 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.715208+0000", 2024-04-26T03:39:41.645 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.3e-06 2024-04-26T03:39:41.645 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.645 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.645 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:41.645 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.715385+0000", 2024-04-26T03:39:41.645 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00017650000000000001 2024-04-26T03:39:41.645 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.645 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.645 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:41.645 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.715407+0000", 2024-04-26T03:39:41.645 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.2334999999999999e-05 2024-04-26T03:39:41.646 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.646 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.646 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:41.646 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.715418+0000", 2024-04-26T03:39:41.646 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1134999999999999e-05 2024-04-26T03:39:41.646 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.646 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.646 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:41.646 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.715534+0000", 2024-04-26T03:39:41.646 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000115322 2024-04-26T03:39:41.646 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.646 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.646 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:41.646 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.715544+0000", 2024-04-26T03:39:41.647 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0815000000000001e-05 2024-04-26T03:39:41.647 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.647 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.647 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:41.647 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.723316+0000", 2024-04-26T03:39:41.647 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0077715550000000003 2024-04-26T03:39:41.647 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.647 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.647 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:41.647 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:01.723357+0000", 2024-04-26T03:39:41.647 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.0952999999999999e-05 2024-04-26T03:39:41.647 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:41.647 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:41.647 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:41.648 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.648 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.648 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:430 3.6s0 3:64627763:::smithi09235267-49:head [append 0~0] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-04-26T03:39:41.648 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:04.683807+0000", 2024-04-26T03:39:41.648 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 36.952284249999998, 2024-04-26T03:39:41.648 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.017357337, 2024-04-26T03:39:41.648 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:41.648 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:41.648 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:41.648 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:41.648 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:41.648 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:41.648 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 430 2024-04-26T03:39:41.648 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.649 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:41.649 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.649 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:41.649 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.683807+0000", 2024-04-26T03:39:41.649 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:41.649 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.649 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.649 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:41.649 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.683807+0000", 2024-04-26T03:39:41.649 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:41.649 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.649 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.649 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:41.649 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.683809+0000", 2024-04-26T03:39:41.649 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.8610000000000001e-06 2024-04-26T03:39:41.650 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.650 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.650 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:41.650 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.683818+0000", 2024-04-26T03:39:41.650 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.4250000000000004e-06 2024-04-26T03:39:41.650 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.650 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.650 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:41.650 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.683834+0000", 2024-04-26T03:39:41.650 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.6228000000000001e-05 2024-04-26T03:39:41.650 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.650 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.650 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:41.651 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.683843+0000", 2024-04-26T03:39:41.651 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.2910000000000007e-06 2024-04-26T03:39:41.651 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.651 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.651 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:41.651 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.683899+0000", 2024-04-26T03:39:41.651 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.6681999999999997e-05 2024-04-26T03:39:41.651 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.651 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.651 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "op must wait for map", 2024-04-26T03:39:41.651 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.683904+0000", 2024-04-26T03:39:41.651 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.0719999999999999e-06 2024-04-26T03:39:41.651 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.651 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.651 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:41.652 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.688041+0000", 2024-04-26T03:39:41.652 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0041362339999999999 2024-04-26T03:39:41.652 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.652 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.652 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:41.652 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.688204+0000", 2024-04-26T03:39:41.652 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00016366800000000001 2024-04-26T03:39:41.652 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.652 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.652 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:41.652 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.689177+0000", 2024-04-26T03:39:41.652 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00097255499999999995 2024-04-26T03:39:41.652 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.652 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.653 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:41.653 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.692572+0000", 2024-04-26T03:39:41.653 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0033949879999999998 2024-04-26T03:39:41.653 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.653 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.653 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:41.653 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.701106+0000", 2024-04-26T03:39:41.653 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0085337069999999998 2024-04-26T03:39:41.653 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.653 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.653 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:41.653 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.701164+0000", 2024-04-26T03:39:41.653 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.8625999999999997e-05 2024-04-26T03:39:41.653 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:41.654 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:41.654 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:41.654 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.654 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.654 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:431 3.6s0 3:64627763:::smithi09235267-49:head [rollback 8] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-04-26T03:39:41.654 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:04.683867+0000", 2024-04-26T03:39:41.654 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 36.952223840999999, 2024-04-26T03:39:41.654 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.017754329999999999, 2024-04-26T03:39:41.654 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:41.654 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:41.654 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:41.654 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:41.654 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:41.654 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:41.655 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 431 2024-04-26T03:39:41.655 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.655 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:41.655 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.655 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:41.655 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.683867+0000", 2024-04-26T03:39:41.655 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:41.655 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.655 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.655 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:41.655 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.683867+0000", 2024-04-26T03:39:41.655 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:41.655 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.656 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.656 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:41.656 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.683868+0000", 2024-04-26T03:39:41.656 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.8599999999999999e-07 2024-04-26T03:39:41.656 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.656 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.656 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:41.656 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.683870+0000", 2024-04-26T03:39:41.656 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.2910000000000002e-06 2024-04-26T03:39:41.656 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.657 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.657 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:41.657 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.683875+0000", 2024-04-26T03:39:41.657 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.6829999999999999e-06 2024-04-26T03:39:41.657 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.657 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.657 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:41.657 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.683878+0000", 2024-04-26T03:39:41.657 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.4690000000000002e-06 2024-04-26T03:39:41.657 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.657 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.657 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:41.657 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.683965+0000", 2024-04-26T03:39:41.657 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.7071999999999996e-05 2024-04-26T03:39:41.658 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.658 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.658 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "waiting_for_map not empty", 2024-04-26T03:39:41.658 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.683974+0000", 2024-04-26T03:39:41.658 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.7159999999999992e-06 2024-04-26T03:39:41.658 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.658 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.658 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:41.658 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.689846+0000", 2024-04-26T03:39:41.658 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0058718399999999997 2024-04-26T03:39:41.658 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.658 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.658 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:41.659 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.689941+0000", 2024-04-26T03:39:41.659 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.5580000000000003e-05 2024-04-26T03:39:41.659 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.659 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.659 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:41.659 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.690497+0000", 2024-04-26T03:39:41.659 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00055555299999999997 2024-04-26T03:39:41.659 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.659 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.659 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:41.659 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.693675+0000", 2024-04-26T03:39:41.659 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0031778269999999998 2024-04-26T03:39:41.659 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.660 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.660 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:41.660 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.701555+0000", 2024-04-26T03:39:41.660 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0078804089999999997 2024-04-26T03:39:41.660 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.660 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.660 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:41.660 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.701622+0000", 2024-04-26T03:39:41.660 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.6503999999999999e-05 2024-04-26T03:39:41.660 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:41.660 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:41.660 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:41.660 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.660 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.661 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:432 3.6s0 3:64627763:::smithi09235267-49:head [append 0~0] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-04-26T03:39:41.661 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:04.683885+0000", 2024-04-26T03:39:41.661 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 36.952205896999999, 2024-04-26T03:39:41.661 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.017911924999999999, 2024-04-26T03:39:41.661 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:41.661 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:41.661 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:41.661 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:41.661 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:41.661 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:41.661 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 432 2024-04-26T03:39:41.661 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.661 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:41.661 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.661 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:41.662 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.683885+0000", 2024-04-26T03:39:41.662 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:41.662 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.662 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.662 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:41.662 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.683885+0000", 2024-04-26T03:39:41.662 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:41.662 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.662 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.662 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:41.662 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.683885+0000", 2024-04-26T03:39:41.662 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.1e-07 2024-04-26T03:39:41.662 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.663 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.663 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:41.663 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.683887+0000", 2024-04-26T03:39:41.663 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.863e-06 2024-04-26T03:39:41.663 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.663 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.663 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:41.663 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.683891+0000", 2024-04-26T03:39:41.663 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.3289999999999998e-06 2024-04-26T03:39:41.663 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.663 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.663 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:41.663 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.683893+0000", 2024-04-26T03:39:41.664 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.8169999999999999e-06 2024-04-26T03:39:41.664 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.664 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.664 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:41.664 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.683999+0000", 2024-04-26T03:39:41.664 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00010592200000000001 2024-04-26T03:39:41.664 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.664 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.664 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "waiting_for_map not empty", 2024-04-26T03:39:41.664 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.684007+0000", 2024-04-26T03:39:41.664 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.3119999999999996e-06 2024-04-26T03:39:41.664 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.664 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.665 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:41.665 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.690818+0000", 2024-04-26T03:39:41.665 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0068111589999999998 2024-04-26T03:39:41.665 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.665 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.665 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:41.665 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.690910+0000", 2024-04-26T03:39:41.665 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.1916999999999994e-05 2024-04-26T03:39:41.665 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.665 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.665 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:41.665 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.691360+0000", 2024-04-26T03:39:41.665 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00045033600000000002 2024-04-26T03:39:41.665 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.666 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.666 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:41.666 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.694888+0000", 2024-04-26T03:39:41.666 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0035276880000000002 2024-04-26T03:39:41.666 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.666 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.666 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:41.666 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.701741+0000", 2024-04-26T03:39:41.666 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0068528529999999999 2024-04-26T03:39:41.666 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.666 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.666 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:41.666 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.701797+0000", 2024-04-26T03:39:41.667 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.6518999999999999e-05 2024-04-26T03:39:41.667 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:41.667 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:41.667 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:41.667 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.667 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.667 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.18s1 49/44 ECSubWrite(tid=206, reqid=client.4556.0:443, at_version=49'36, trim_to=0'0, roll_forward_to=49'34))", 2024-04-26T03:39:41.667 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:04.734356+0000", 2024-04-26T03:39:41.667 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 36.901735432000002, 2024-04-26T03:39:41.667 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0072519239999999999, 2024-04-26T03:39:41.667 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:41.667 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:41.667 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:41.667 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:41.668 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.668 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:41.668 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.734356+0000", 2024-04-26T03:39:41.668 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:41.668 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.668 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.668 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:41.668 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.734356+0000", 2024-04-26T03:39:41.668 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:41.668 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.668 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.668 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:41.668 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.734356+0000", 2024-04-26T03:39:41.669 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.2900000000000003e-07 2024-04-26T03:39:41.669 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.669 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.669 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:41.669 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.734606+0000", 2024-04-26T03:39:41.669 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00024913799999999999 2024-04-26T03:39:41.669 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.669 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.669 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:41.669 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.734622+0000", 2024-04-26T03:39:41.669 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.6453999999999999e-05 2024-04-26T03:39:41.669 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.669 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.670 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:41.670 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.734631+0000", 2024-04-26T03:39:41.670 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.6929999999999996e-06 2024-04-26T03:39:41.670 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.670 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.670 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:41.670 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.734713+0000", 2024-04-26T03:39:41.670 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.2408000000000001e-05 2024-04-26T03:39:41.670 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.670 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.670 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:41.670 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.734745+0000", 2024-04-26T03:39:41.670 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.1866999999999998e-05 2024-04-26T03:39:41.671 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.671 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.671 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:41.671 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.741565+0000", 2024-04-26T03:39:41.671 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0068200869999999999 2024-04-26T03:39:41.671 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.671 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.671 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:41.671 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:04.741608+0000", 2024-04-26T03:39:41.671 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.2548e-05 2024-04-26T03:39:41.671 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:41.672 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:41.672 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:41.672 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.672 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.672 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.13s4 56/55 ECSubWrite(tid=104, reqid=client.4556.0:604, at_version=56'47, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:39:41.672 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:11.774415+0000", 2024-04-26T03:39:41.672 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 29.861676451000001, 2024-04-26T03:39:41.672 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.016022932, 2024-04-26T03:39:41.672 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:41.672 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:41.672 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:41.672 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:41.672 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.672 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:41.673 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.774415+0000", 2024-04-26T03:39:41.673 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:41.673 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.673 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.673 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:41.673 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.774415+0000", 2024-04-26T03:39:41.673 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:41.673 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.673 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.673 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:41.673 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.774416+0000", 2024-04-26T03:39:41.673 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.497e-06 2024-04-26T03:39:41.673 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.673 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.674 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:41.674 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.775996+0000", 2024-04-26T03:39:41.674 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.001579558 2024-04-26T03:39:41.674 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.674 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.674 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:41.674 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.776007+0000", 2024-04-26T03:39:41.674 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1355000000000001e-05 2024-04-26T03:39:41.674 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.674 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.675 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:41.675 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.776013+0000", 2024-04-26T03:39:41.675 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.7579999999999998e-06 2024-04-26T03:39:41.675 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.675 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.675 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:41.675 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.776070+0000", 2024-04-26T03:39:41.675 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.6818999999999999e-05 2024-04-26T03:39:41.675 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.675 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.675 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:41.675 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.776076+0000", 2024-04-26T03:39:41.675 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.6460000000000004e-06 2024-04-26T03:39:41.675 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.676 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.676 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:41.676 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.790407+0000", 2024-04-26T03:39:41.676 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.014330489 2024-04-26T03:39:41.676 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.676 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.676 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:41.676 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.790438+0000", 2024-04-26T03:39:41.676 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.0809999999999998e-05 2024-04-26T03:39:41.676 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:41.676 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:41.676 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:41.676 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.676 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.677 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDPGPush(3.13s4 56/55 [PushOp(3:c8a9e28b:::smithi09235267-30:head, version: 49'32, data_included: [0~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 4, recovery_info: ObjectRecoveryInfo(3:c8a9e28b:::smithi09235267-30:head@49'32, size: 2768896, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-04-26T03:39:41.677 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:11.811344+0000", 2024-04-26T03:39:41.677 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 29.824747414000001, 2024-04-26T03:39:41.677 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.010863571000000001, 2024-04-26T03:39:41.677 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:41.677 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:41.677 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:41.677 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:41.677 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.677 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:41.677 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.811344+0000", 2024-04-26T03:39:41.677 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:41.677 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.678 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.678 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:41.678 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.811344+0000", 2024-04-26T03:39:41.678 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:41.678 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.678 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.678 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:41.678 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.811344+0000", 2024-04-26T03:39:41.678 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.4899999999999995e-07 2024-04-26T03:39:41.678 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.678 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.678 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:41.678 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.811943+0000", 2024-04-26T03:39:41.679 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00059826900000000001 2024-04-26T03:39:41.679 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.679 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.679 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:41.679 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.811957+0000", 2024-04-26T03:39:41.679 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.4831000000000001e-05 2024-04-26T03:39:41.679 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.679 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.679 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:41.679 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.811966+0000", 2024-04-26T03:39:41.679 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.0280000000000006e-06 2024-04-26T03:39:41.679 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.679 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.679 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:41.679 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.816605+0000", 2024-04-26T03:39:41.680 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0046387889999999999 2024-04-26T03:39:41.680 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.680 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.680 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:41.680 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.822207+0000", 2024-04-26T03:39:41.680 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0056020050000000002 2024-04-26T03:39:41.680 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:41.680 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:41.680 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:41.680 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.680 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.680 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDPGPush(3.13s4 56/55 [PushOp(3:c8a9e28b:::smithi09235267-30:head, version: 49'32, data_included: [524288~167936], data_size: 167936, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:c8a9e28b:::smithi09235267-30:head@49'32, size: 2768896, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2768896, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-04-26T03:39:41.680 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:11.842440+0000", 2024-04-26T03:39:41.681 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 29.793650951, 2024-04-26T03:39:41.681 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0079054569999999994, 2024-04-26T03:39:41.681 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:41.681 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:41.681 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:41.681 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:41.681 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.681 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:41.681 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.842440+0000", 2024-04-26T03:39:41.681 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:41.681 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.681 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.681 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:41.681 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.842440+0000", 2024-04-26T03:39:41.682 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:41.682 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.682 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.682 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:41.682 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.842441+0000", 2024-04-26T03:39:41.682 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.75e-07 2024-04-26T03:39:41.682 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.682 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.682 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:41.682 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.842876+0000", 2024-04-26T03:39:41.682 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000435757 2024-04-26T03:39:41.682 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.682 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.683 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:41.683 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.842896+0000", 2024-04-26T03:39:41.683 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.9202e-05 2024-04-26T03:39:41.683 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.683 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.683 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:41.683 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.842905+0000", 2024-04-26T03:39:41.683 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.4080000000000004e-06 2024-04-26T03:39:41.683 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.683 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.683 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:41.683 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.844849+0000", 2024-04-26T03:39:41.683 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.001944028 2024-04-26T03:39:41.684 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.684 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.684 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:41.684 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.850346+0000", 2024-04-26T03:39:41.684 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0054965869999999998 2024-04-26T03:39:41.684 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:41.684 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:41.684 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:41.684 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.684 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.684 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.10s5 60/48 ECSubWrite(tid=272, reqid=client.4556.0:703, at_version=60'20, trim_to=0'0, roll_forward_to=60'18))", 2024-04-26T03:39:41.684 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:15.958104+0000", 2024-04-26T03:39:41.684 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 25.677987085000002, 2024-04-26T03:39:41.684 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0092449460000000004, 2024-04-26T03:39:41.685 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:41.685 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:41.685 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:41.685 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:41.685 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.685 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:41.685 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.958104+0000", 2024-04-26T03:39:41.685 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:41.685 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.685 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.685 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:41.685 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.958104+0000", 2024-04-26T03:39:41.685 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:41.685 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.686 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.686 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:41.686 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.958105+0000", 2024-04-26T03:39:41.686 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.2500000000000005e-07 2024-04-26T03:39:41.686 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.686 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.686 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:41.686 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.958846+0000", 2024-04-26T03:39:41.686 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00074102 2024-04-26T03:39:41.686 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.686 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.686 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:41.686 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.958857+0000", 2024-04-26T03:39:41.687 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1601e-05 2024-04-26T03:39:41.687 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.687 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.687 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:41.687 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.958865+0000", 2024-04-26T03:39:41.687 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.0260000000000007e-06 2024-04-26T03:39:41.687 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.687 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.687 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:41.687 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.958947+0000", 2024-04-26T03:39:41.687 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.1675999999999995e-05 2024-04-26T03:39:41.687 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.687 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.688 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:41.688 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.958956+0000", 2024-04-26T03:39:41.688 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.0710000000000001e-06 2024-04-26T03:39:41.688 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.688 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.688 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:41.688 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.967309+0000", 2024-04-26T03:39:41.688 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0083527810000000001 2024-04-26T03:39:41.688 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.688 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.688 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:41.688 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.967349+0000", 2024-04-26T03:39:41.688 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.0145999999999998e-05 2024-04-26T03:39:41.689 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:41.689 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:41.689 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:41.689 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.689 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.689 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.10s5 60/48 ECSubWrite(tid=273, reqid=client.4556.0:704, at_version=60'21, trim_to=0'0, roll_forward_to=60'18))", 2024-04-26T03:39:41.689 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:15.962785+0000", 2024-04-26T03:39:41.689 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 25.673305766999999, 2024-04-26T03:39:41.689 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0092490620000000006, 2024-04-26T03:39:41.689 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:41.689 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:41.689 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:41.689 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:41.689 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.690 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:41.690 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.962785+0000", 2024-04-26T03:39:41.690 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:41.690 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.690 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.690 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:41.690 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.962785+0000", 2024-04-26T03:39:41.690 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:41.690 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.690 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.690 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:41.690 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.962786+0000", 2024-04-26T03:39:41.691 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.8999999999999995e-07 2024-04-26T03:39:41.691 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.691 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.691 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:41.691 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.962919+0000", 2024-04-26T03:39:41.691 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000132538 2024-04-26T03:39:41.691 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.691 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.691 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:41.691 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.962938+0000", 2024-04-26T03:39:41.691 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.9210000000000001e-05 2024-04-26T03:39:41.691 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.691 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.691 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:41.692 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.962946+0000", 2024-04-26T03:39:41.692 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.4570000000000005e-06 2024-04-26T03:39:41.692 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.692 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.692 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:41.692 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.962998+0000", 2024-04-26T03:39:41.692 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.1811000000000003e-05 2024-04-26T03:39:41.692 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.692 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.692 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:41.692 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.963007+0000", 2024-04-26T03:39:41.692 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.8550000000000008e-06 2024-04-26T03:39:41.692 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.692 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.693 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:41.693 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.972000+0000", 2024-04-26T03:39:41.693 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0089924989999999993 2024-04-26T03:39:41.693 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.693 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.693 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:41.693 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.972034+0000", 2024-04-26T03:39:41.693 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.4802000000000003e-05 2024-04-26T03:39:41.693 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:41.693 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:41.693 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:41.693 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.693 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.694 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.10s5 60/48 ECSubWrite(tid=274, reqid=client.4556.0:705, at_version=60'22, trim_to=0'0, roll_forward_to=60'18))", 2024-04-26T03:39:41.694 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:15.966729+0000", 2024-04-26T03:39:41.694 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 25.669362203999999, 2024-04-26T03:39:41.694 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0095163020000000008, 2024-04-26T03:39:41.694 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:41.694 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:41.694 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:41.694 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:41.694 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.694 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:41.694 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.966729+0000", 2024-04-26T03:39:41.694 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:41.694 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.695 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.695 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:41.695 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.966729+0000", 2024-04-26T03:39:41.695 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:41.695 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.695 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.695 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:41.695 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.966730+0000", 2024-04-26T03:39:41.695 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1510000000000001e-06 2024-04-26T03:39:41.695 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.695 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.695 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:41.695 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.966955+0000", 2024-04-26T03:39:41.696 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000224897 2024-04-26T03:39:41.696 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.696 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.696 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:41.696 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.966981+0000", 2024-04-26T03:39:41.696 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.6290000000000001e-05 2024-04-26T03:39:41.696 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.696 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.696 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:41.696 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.966992+0000", 2024-04-26T03:39:41.696 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0740999999999999e-05 2024-04-26T03:39:41.696 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.696 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.696 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:41.697 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.967384+0000", 2024-04-26T03:39:41.697 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00039248700000000001 2024-04-26T03:39:41.697 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.697 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.697 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:41.697 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.967395+0000", 2024-04-26T03:39:41.697 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0168e-05 2024-04-26T03:39:41.697 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.697 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.697 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:41.697 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.976218+0000", 2024-04-26T03:39:41.697 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0088229169999999996 2024-04-26T03:39:41.697 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.698 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.698 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:41.698 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.976245+0000", 2024-04-26T03:39:41.698 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.7651e-05 2024-04-26T03:39:41.698 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:41.698 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:41.698 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:41.698 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.698 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.698 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.10s5 60/48 ECSubWrite(tid=275, reqid=client.4556.0:706, at_version=60'23, trim_to=0'0, roll_forward_to=60'18))", 2024-04-26T03:39:41.698 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:15.970750+0000", 2024-04-26T03:39:41.698 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 25.665340623999999, 2024-04-26T03:39:41.699 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0072075180000000004, 2024-04-26T03:39:41.699 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:41.699 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:41.699 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:41.699 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:41.699 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.699 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:41.699 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.970750+0000", 2024-04-26T03:39:41.699 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:41.699 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.699 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.699 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:41.699 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.970750+0000", 2024-04-26T03:39:41.699 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:41.699 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.700 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.700 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:41.700 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.970751+0000", 2024-04-26T03:39:41.700 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.7699999999999992e-07 2024-04-26T03:39:41.700 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.700 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.700 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:41.700 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.970952+0000", 2024-04-26T03:39:41.700 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00020024499999999999 2024-04-26T03:39:41.700 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.700 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.700 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:41.700 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.970965+0000", 2024-04-26T03:39:41.700 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.3179999999999999e-05 2024-04-26T03:39:41.700 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.701 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.701 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:41.701 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.970974+0000", 2024-04-26T03:39:41.701 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.5570000000000001e-06 2024-04-26T03:39:41.701 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.701 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.701 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:41.701 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.972106+0000", 2024-04-26T03:39:41.701 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0011319279999999999 2024-04-26T03:39:41.701 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.701 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.701 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:41.701 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.972117+0000", 2024-04-26T03:39:41.701 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0181e-05 2024-04-26T03:39:41.701 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.702 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.702 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:41.702 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.977907+0000", 2024-04-26T03:39:41.702 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0057901180000000003 2024-04-26T03:39:41.702 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.702 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.702 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:41.702 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.977958+0000", 2024-04-26T03:39:41.702 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.1332000000000003e-05 2024-04-26T03:39:41.702 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:41.702 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:41.702 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:41.702 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.703 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.703 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.13s4 60/55 ECSubWrite(tid=141, reqid=client.4556.0:711, at_version=60'53, trim_to=0'0, roll_forward_to=60'51))", 2024-04-26T03:39:41.703 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:15.977943+0000", 2024-04-26T03:39:41.703 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 25.658147725999999, 2024-04-26T03:39:41.703 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.010870046, 2024-04-26T03:39:41.703 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:41.703 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:41.703 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:41.703 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:41.703 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.703 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:41.703 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.977943+0000", 2024-04-26T03:39:41.704 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:41.704 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.704 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.704 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:41.704 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.977943+0000", 2024-04-26T03:39:41.704 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:41.704 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.704 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.704 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:41.704 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.977944+0000", 2024-04-26T03:39:41.704 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0499999999999999e-06 2024-04-26T03:39:41.704 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.704 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.704 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:41.705 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.978162+0000", 2024-04-26T03:39:41.705 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000217238 2024-04-26T03:39:41.705 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.705 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.705 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:41.705 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.978177+0000", 2024-04-26T03:39:41.705 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.4895999999999999e-05 2024-04-26T03:39:41.705 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.705 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.705 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:41.705 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.978186+0000", 2024-04-26T03:39:41.705 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.9129999999999998e-06 2024-04-26T03:39:41.705 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.705 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.706 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:41.706 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.978255+0000", 2024-04-26T03:39:41.706 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.8915e-05 2024-04-26T03:39:41.706 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.706 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.706 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:41.706 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.978261+0000", 2024-04-26T03:39:41.706 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.0680000000000002e-06 2024-04-26T03:39:41.706 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.706 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.706 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:41.706 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.988781+0000", 2024-04-26T03:39:41.706 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.010519153 2024-04-26T03:39:41.707 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.707 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.707 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:41.707 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.988813+0000", 2024-04-26T03:39:41.707 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.2812999999999999e-05 2024-04-26T03:39:41.707 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:41.707 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:41.707 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:41.707 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.707 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.707 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.13s4 60/55 ECSubWrite(tid=142, reqid=client.4556.0:712, at_version=60'54, trim_to=0'0, roll_forward_to=60'51))", 2024-04-26T03:39:41.707 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:15.983346+0000", 2024-04-26T03:39:41.707 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 25.652744890000001, 2024-04-26T03:39:41.707 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0075977960000000004, 2024-04-26T03:39:41.708 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:41.708 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:41.708 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:41.708 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:41.708 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.708 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:41.708 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.983346+0000", 2024-04-26T03:39:41.708 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:41.708 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.708 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.708 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:41.708 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.983346+0000", 2024-04-26T03:39:41.708 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:41.708 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.709 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.709 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:41.709 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.983347+0000", 2024-04-26T03:39:41.709 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.1900000000000002e-07 2024-04-26T03:39:41.709 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.709 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.709 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:41.709 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.983538+0000", 2024-04-26T03:39:41.709 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000190807 2024-04-26T03:39:41.709 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.709 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.709 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:41.709 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.983548+0000", 2024-04-26T03:39:41.709 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0566e-05 2024-04-26T03:39:41.710 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.710 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.710 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:41.710 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.983557+0000", 2024-04-26T03:39:41.710 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.9700000000000005e-06 2024-04-26T03:39:41.710 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.710 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.710 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:41.710 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.983621+0000", 2024-04-26T03:39:41.710 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.3316000000000002e-05 2024-04-26T03:39:41.710 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.710 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.711 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:41.711 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.983630+0000", 2024-04-26T03:39:41.711 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.9119999999999993e-06 2024-04-26T03:39:41.711 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.711 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.711 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:41.711 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.990910+0000", 2024-04-26T03:39:41.711 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00727986 2024-04-26T03:39:41.711 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.711 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.711 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:41.711 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.990944+0000", 2024-04-26T03:39:41.711 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.3646000000000003e-05 2024-04-26T03:39:41.711 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:41.712 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:41.712 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:41.712 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.712 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.712 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.fs6 65/62 ECSubWrite(tid=179, reqid=client.4556.0:783, at_version=65'61, trim_to=0'0, roll_forward_to=65'59))", 2024-04-26T03:39:41.712 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:20.833582+0000", 2024-04-26T03:39:41.712 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 20.802509075, 2024-04-26T03:39:41.712 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0068806600000000002, 2024-04-26T03:39:41.712 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:41.712 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:41.712 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:41.712 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:41.712 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.712 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:41.712 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.833582+0000", 2024-04-26T03:39:41.712 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:41.713 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.713 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.713 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:41.713 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.833582+0000", 2024-04-26T03:39:41.713 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:41.713 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.713 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.713 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:41.713 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.833583+0000", 2024-04-26T03:39:41.713 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.6499999999999998e-07 2024-04-26T03:39:41.713 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.713 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.713 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:41.713 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.834537+0000", 2024-04-26T03:39:41.714 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00095386600000000005 2024-04-26T03:39:41.714 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.714 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.714 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:41.714 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.834552+0000", 2024-04-26T03:39:41.714 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.517e-05 2024-04-26T03:39:41.714 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.714 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.714 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:41.714 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.834566+0000", 2024-04-26T03:39:41.714 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.4087000000000001e-05 2024-04-26T03:39:41.714 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.714 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.714 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:41.715 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.834621+0000", 2024-04-26T03:39:41.715 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.4849999999999998e-05 2024-04-26T03:39:41.715 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.715 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.715 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:41.715 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.834629+0000", 2024-04-26T03:39:41.715 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.8050000000000007e-06 2024-04-26T03:39:41.715 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.715 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.715 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:41.715 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.840429+0000", 2024-04-26T03:39:41.715 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0058003569999999999 2024-04-26T03:39:41.715 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.715 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.716 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:41.716 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.840463+0000", 2024-04-26T03:39:41.716 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.3760000000000002e-05 2024-04-26T03:39:41.716 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:41.716 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:41.716 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:41.716 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.716 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.716 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.fs6 75/62 ECSubWrite(tid=222, reqid=osd.11.0:222, at_version=75'83, trim_to=0'0, roll_forward_to=74'79))", 2024-04-26T03:39:41.716 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:30.692763+0000", 2024-04-26T03:39:41.716 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 10.943327673000001, 2024-04-26T03:39:41.716 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0090881069999999998, 2024-04-26T03:39:41.716 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:41.717 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:41.717 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:41.717 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:41.717 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.717 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:41.717 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:30.692763+0000", 2024-04-26T03:39:41.717 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:41.717 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.717 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.717 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:41.717 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:30.692763+0000", 2024-04-26T03:39:41.717 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:41.717 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.717 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.717 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:41.718 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:30.692764+0000", 2024-04-26T03:39:41.718 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.6400000000000001e-07 2024-04-26T03:39:41.718 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.718 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.718 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:41.718 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:30.692800+0000", 2024-04-26T03:39:41.718 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.5636000000000002e-05 2024-04-26T03:39:41.718 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.718 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.718 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:41.718 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:30.692813+0000", 2024-04-26T03:39:41.718 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.3234999999999999e-05 2024-04-26T03:39:41.718 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.718 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.719 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:41.719 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:30.692822+0000", 2024-04-26T03:39:41.719 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.2590000000000006e-06 2024-04-26T03:39:41.719 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.719 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.719 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:41.719 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:30.692888+0000", 2024-04-26T03:39:41.719 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.5592000000000005e-05 2024-04-26T03:39:41.719 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.719 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.719 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:41.719 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:30.692896+0000", 2024-04-26T03:39:41.719 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.2959999999999999e-06 2024-04-26T03:39:41.720 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.720 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.720 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:41.720 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:30.701811+0000", 2024-04-26T03:39:41.720 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0089148219999999993 2024-04-26T03:39:41.720 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.720 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.720 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:41.720 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:30.701852+0000", 2024-04-26T03:39:41.720 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.0402999999999999e-05 2024-04-26T03:39:41.720 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:41.720 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:41.720 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:41.721 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.721 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.721 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.8s1 80/63 ECSubWrite(tid=321, reqid=client.4556.0:897, at_version=80'111, trim_to=0'0, roll_forward_to=80'110))", 2024-04-26T03:39:41.721 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:35.716512+0000", 2024-04-26T03:39:41.721 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 5.9195791409999998, 2024-04-26T03:39:41.721 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.019561482000000002, 2024-04-26T03:39:41.721 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:41.721 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:41.721 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:41.721 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:41.721 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.721 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:41.722 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:35.716512+0000", 2024-04-26T03:39:41.722 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:41.722 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.722 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.722 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:41.722 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:35.716512+0000", 2024-04-26T03:39:41.722 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:41.722 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.722 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.722 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:41.722 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:35.716513+0000", 2024-04-26T03:39:41.723 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.26e-06 2024-04-26T03:39:41.723 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.723 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.723 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:41.723 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:35.719489+0000", 2024-04-26T03:39:41.723 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0029761750000000002 2024-04-26T03:39:41.723 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.723 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.723 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:41.723 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:35.719507+0000", 2024-04-26T03:39:41.723 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.7368999999999999e-05 2024-04-26T03:39:41.723 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.723 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.724 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:41.724 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:35.719516+0000", 2024-04-26T03:39:41.724 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.3729999999999999e-06 2024-04-26T03:39:41.724 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.724 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.724 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:41.724 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:35.719590+0000", 2024-04-26T03:39:41.724 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.3705e-05 2024-04-26T03:39:41.724 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.724 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.724 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:41.724 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:35.719600+0000", 2024-04-26T03:39:41.724 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0227000000000001e-05 2024-04-26T03:39:41.725 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.725 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.725 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:41.725 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:35.736040+0000", 2024-04-26T03:39:41.725 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.01644004 2024-04-26T03:39:41.725 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.725 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.725 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:41.725 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:35.736073+0000", 2024-04-26T03:39:41.725 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.3333000000000001e-05 2024-04-26T03:39:41.725 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:41.725 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:41.725 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:41.726 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.726 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.726 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.10s5 80/48 ECSubWrite(tid=322, reqid=client.4556.0:895, at_version=80'35, trim_to=0'0, roll_forward_to=75'33))", 2024-04-26T03:39:41.726 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:35.739904+0000", 2024-04-26T03:39:41.726 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 5.8961873909999998, 2024-04-26T03:39:41.726 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.035200074999999997, 2024-04-26T03:39:41.726 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:41.726 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:41.726 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:41.726 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:41.726 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.726 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:41.726 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:35.739904+0000", 2024-04-26T03:39:41.726 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:41.727 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.727 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.727 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:41.727 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:35.739904+0000", 2024-04-26T03:39:41.727 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:41.727 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.727 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.727 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:41.727 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:35.739904+0000", 2024-04-26T03:39:41.727 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.4600000000000004e-07 2024-04-26T03:39:41.727 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.727 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.727 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:41.728 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:35.742220+0000", 2024-04-26T03:39:41.728 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0023153890000000002 2024-04-26T03:39:41.728 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.728 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.728 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:41.728 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:35.742232+0000", 2024-04-26T03:39:41.728 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.2529e-05 2024-04-26T03:39:41.728 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.728 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.728 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:41.728 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:35.742240+0000", 2024-04-26T03:39:41.728 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.3039999999999999e-06 2024-04-26T03:39:41.728 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.728 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.728 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:41.729 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:35.742302+0000", 2024-04-26T03:39:41.729 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.1934999999999996e-05 2024-04-26T03:39:41.729 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.729 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.729 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:41.729 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:35.742310+0000", 2024-04-26T03:39:41.729 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.9590000000000001e-06 2024-04-26T03:39:41.729 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.729 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.729 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:41.729 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:35.775052+0000", 2024-04-26T03:39:41.729 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.032741127000000002 2024-04-26T03:39:41.729 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.729 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.730 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:41.730 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:35.775104+0000", 2024-04-26T03:39:41.730 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.2185999999999998e-05 2024-04-26T03:39:41.730 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:41.730 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:41.730 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:41.730 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.730 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.730 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:905 3.6s0 3:64627763:::smithi09235267-49:head [delete] snapc 1c=[1c,1b,1a,19,18,17,15,14,12] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e81)", 2024-04-26T03:39:41.730 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:36.694795+0000", 2024-04-26T03:39:41.730 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 4.9412958790000001, 2024-04-26T03:39:41.730 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0087604020000000005, 2024-04-26T03:39:41.730 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:41.731 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:41.731 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "started", 2024-04-26T03:39:41.731 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:41.731 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:41.731 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:41.731 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 905 2024-04-26T03:39:41.731 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.731 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:41.731 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.731 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:41.731 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:36.694795+0000", 2024-04-26T03:39:41.731 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:41.731 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.732 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.732 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:41.732 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:36.694795+0000", 2024-04-26T03:39:41.732 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:41.732 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.732 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.732 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:41.732 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:36.694798+0000", 2024-04-26T03:39:41.732 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.1839999999999999e-06 2024-04-26T03:39:41.732 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.732 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.732 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:41.732 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:36.694815+0000", 2024-04-26T03:39:41.732 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.6348e-05 2024-04-26T03:39:41.733 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.733 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.733 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:41.733 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:36.694847+0000", 2024-04-26T03:39:41.733 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.2453000000000003e-05 2024-04-26T03:39:41.733 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.733 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.733 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:41.733 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:36.694863+0000", 2024-04-26T03:39:41.733 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.5756999999999998e-05 2024-04-26T03:39:41.733 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.733 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.733 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:41.733 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:36.694971+0000", 2024-04-26T03:39:41.733 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000107601 2024-04-26T03:39:41.734 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.734 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.734 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "op must wait for map", 2024-04-26T03:39:41.734 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:36.694978+0000", 2024-04-26T03:39:41.734 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.1130000000000003e-06 2024-04-26T03:39:41.734 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.734 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.734 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:41.734 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:36.700343+0000", 2024-04-26T03:39:41.734 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0053652919999999998 2024-04-26T03:39:41.734 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.734 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.734 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:41.735 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:36.700415+0000", 2024-04-26T03:39:41.735 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.2516999999999997e-05 2024-04-26T03:39:41.735 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:41.735 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:41.735 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:41.735 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:36.703556+0000", 2024-04-26T03:39:41.735 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0031401369999999999 2024-04-26T03:39:41.735 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:41.735 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:41.735 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:41.735 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:41.735 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:41.736 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:39:41.737 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-26T03:39:41.738 INFO:tasks.ceph.osd.15.smithi092.stderr:2024-04-26T03:39:41.735+0000 7f6d12dec640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 31824) UID: 0 2024-04-26T03:39:41.805 INFO:teuthology.orchestra.run.smithi028.stderr:noscrub is unset 2024-04-26T03:39:41.806 INFO:teuthology.orchestra.run.smithi028.stderr:reweighted osd.13 to 1 (10000) 2024-04-26T03:39:41.822 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 14 1 2024-04-26T03:39:41.864 INFO:tasks.ceph.osd.8.smithi028.stderr:2024-04-26T03:39:41.862+0000 7f5bb75d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 31968) UID: 0 2024-04-26T03:39:41.890 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:39:41.890 INFO:teuthology.orchestra.run.smithi028.stdout: "ops": [], 2024-04-26T03:39:41.890 INFO:teuthology.orchestra.run.smithi028.stdout: "num_ops": 0 2024-04-26T03:39:41.891 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:39:41.900 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-26T03:39:41.939 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-04-26T03:39:41.937+0000 7fe9223c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31954) UID: 0 2024-04-26T03:39:42.041 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:39:42.040+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:39:42.050 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:39:42.050 INFO:teuthology.orchestra.run.smithi028.stdout: "ops": [], 2024-04-26T03:39:42.050 INFO:teuthology.orchestra.run.smithi028.stdout: "complaint_time": 30, 2024-04-26T03:39:42.050 INFO:teuthology.orchestra.run.smithi028.stdout: "num_blocked_ops": 0 2024-04-26T03:39:42.050 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:39:42.059 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-26T03:39:42.141 INFO:tasks.ceph.osd.8.smithi028.stderr:2024-04-26T03:39:42.140+0000 7f5bb75d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 31968) UID: 0 2024-04-26T03:39:42.169 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:39:42.170 INFO:teuthology.orchestra.run.smithi028.stdout: "size": 20, 2024-04-26T03:39:42.170 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 600, 2024-04-26T03:39:42.170 INFO:teuthology.orchestra.run.smithi028.stdout: "ops": [ 2024-04-26T03:39:42.170 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.170 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:9 3.bs0 3:dbf634d0:::smithi09235267-3:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:42.170 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.481597+0000", 2024-04-26T03:39:42.170 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 50.685338244, 2024-04-26T03:39:42.171 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.024373156999999999, 2024-04-26T03:39:42.171 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:42.171 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:42.171 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "started", 2024-04-26T03:39:42.171 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:42.171 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:42.171 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:42.171 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 9 2024-04-26T03:39:42.171 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.171 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:42.171 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.171 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:42.171 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.481597+0000", 2024-04-26T03:39:42.172 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:42.172 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.172 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.172 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:42.172 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.481597+0000", 2024-04-26T03:39:42.172 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:42.172 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.172 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.172 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:42.172 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.481597+0000", 2024-04-26T03:39:42.173 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.5599999999999995e-07 2024-04-26T03:39:42.173 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.173 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.173 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:42.173 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.481610+0000", 2024-04-26T03:39:42.173 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.2330000000000001e-05 2024-04-26T03:39:42.173 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.173 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.174 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:42.174 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.481618+0000", 2024-04-26T03:39:42.174 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.7159999999999992e-06 2024-04-26T03:39:42.174 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.174 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.174 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:42.174 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.481624+0000", 2024-04-26T03:39:42.174 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.8989999999999998e-06 2024-04-26T03:39:42.175 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.175 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.175 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:42.175 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.484627+0000", 2024-04-26T03:39:42.175 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00300257 2024-04-26T03:39:42.175 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.175 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.175 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:42.175 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.484712+0000", 2024-04-26T03:39:42.175 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.4753000000000007e-05 2024-04-26T03:39:42.175 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.175 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.176 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:42.176 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.494632+0000", 2024-04-26T03:39:42.176 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0099199100000000005 2024-04-26T03:39:42.176 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.176 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.176 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:42.176 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.494899+0000", 2024-04-26T03:39:42.176 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00026711399999999999 2024-04-26T03:39:42.176 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.176 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.176 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:42.176 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.505970+0000", 2024-04-26T03:39:42.177 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.011071309 2024-04-26T03:39:42.177 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:42.177 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:42.177 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:42.177 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.177 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.177 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:24 3.7s0 3:e644bba7:::smithi09235267-8:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:42.177 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.485110+0000", 2024-04-26T03:39:42.177 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 50.681825197000002, 2024-04-26T03:39:42.177 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.015789734, 2024-04-26T03:39:42.177 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:42.178 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:42.178 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "started", 2024-04-26T03:39:42.178 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:42.178 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:42.178 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:42.178 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 24 2024-04-26T03:39:42.178 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.178 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:42.178 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.178 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:42.178 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.485110+0000", 2024-04-26T03:39:42.178 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:42.179 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.179 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.179 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:42.179 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.485110+0000", 2024-04-26T03:39:42.179 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:42.179 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.179 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.179 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:42.179 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.485110+0000", 2024-04-26T03:39:42.179 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.4099999999999999e-07 2024-04-26T03:39:42.179 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.179 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.179 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:42.179 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.485114+0000", 2024-04-26T03:39:42.180 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.2430000000000001e-06 2024-04-26T03:39:42.180 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.180 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.180 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:42.180 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.485120+0000", 2024-04-26T03:39:42.180 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.5810000000000002e-06 2024-04-26T03:39:42.180 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.180 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.180 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:42.180 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.485125+0000", 2024-04-26T03:39:42.180 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.0270000000000003e-06 2024-04-26T03:39:42.180 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.180 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.180 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:42.180 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.488721+0000", 2024-04-26T03:39:42.181 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.003595844 2024-04-26T03:39:42.181 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.181 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.181 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:42.181 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.488795+0000", 2024-04-26T03:39:42.181 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.3850000000000006e-05 2024-04-26T03:39:42.181 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.181 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.181 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:42.181 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.496954+0000", 2024-04-26T03:39:42.181 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0081585890000000008 2024-04-26T03:39:42.181 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.181 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.181 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:42.181 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.497038+0000", 2024-04-26T03:39:42.182 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.3934e-05 2024-04-26T03:39:42.182 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.182 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.182 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:42.182 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.500900+0000", 2024-04-26T03:39:42.182 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0038621250000000001 2024-04-26T03:39:42.182 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:42.182 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:42.182 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:42.182 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.182 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.182 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:37 3.bs0 3:d42e9299:::smithi09235267-13:head [append 0~212992,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:42.182 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.492880+0000", 2024-04-26T03:39:42.182 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 50.674055162000002, 2024-04-26T03:39:42.183 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.019386614999999999, 2024-04-26T03:39:42.183 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:42.183 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:42.183 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:42.183 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:42.183 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:42.183 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:42.183 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 37 2024-04-26T03:39:42.183 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.183 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:42.183 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.183 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:42.183 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.492880+0000", 2024-04-26T03:39:42.183 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:42.184 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.184 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.184 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:42.184 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.492880+0000", 2024-04-26T03:39:42.184 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:42.184 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.184 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.184 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:42.184 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.492898+0000", 2024-04-26T03:39:42.184 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.7759000000000001e-05 2024-04-26T03:39:42.184 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.184 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.184 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:42.184 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493142+0000", 2024-04-26T03:39:42.184 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00024467400000000001 2024-04-26T03:39:42.185 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.185 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.185 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:42.185 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493155+0000", 2024-04-26T03:39:42.185 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.2862e-05 2024-04-26T03:39:42.185 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.185 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.185 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:42.185 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493197+0000", 2024-04-26T03:39:42.185 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.2239e-05 2024-04-26T03:39:42.185 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.185 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.185 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:42.185 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495213+0000", 2024-04-26T03:39:42.186 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0020159090000000002 2024-04-26T03:39:42.186 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.186 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.186 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:42.186 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.495342+0000", 2024-04-26T03:39:42.186 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000128214 2024-04-26T03:39:42.186 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.186 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.186 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:42.186 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.501605+0000", 2024-04-26T03:39:42.186 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0062634530000000004 2024-04-26T03:39:42.186 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.186 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.186 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:42.187 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.505099+0000", 2024-04-26T03:39:42.187 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0034938859999999999 2024-04-26T03:39:42.187 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.187 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.187 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:42.187 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.512085+0000", 2024-04-26T03:39:42.187 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0069863369999999996 2024-04-26T03:39:42.187 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.187 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.187 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:42.187 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.512267+0000", 2024-04-26T03:39:42.187 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000181282 2024-04-26T03:39:42.187 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:42.187 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:42.187 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:42.188 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.188 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.188 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:38 3.bs0 3:d42e9299:::smithi09235267-13:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:42.188 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.493215+0000", 2024-04-26T03:39:42.188 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 50.673719949000002, 2024-04-26T03:39:42.188 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.025985364, 2024-04-26T03:39:42.188 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:42.188 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:42.188 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:42.188 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:42.188 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:42.188 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:42.188 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 38 2024-04-26T03:39:42.188 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.188 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:42.189 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.189 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:42.189 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493215+0000", 2024-04-26T03:39:42.189 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:42.189 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.189 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.189 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:42.189 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493215+0000", 2024-04-26T03:39:42.189 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:42.189 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.189 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.189 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:42.189 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493216+0000", 2024-04-26T03:39:42.189 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.4700000000000001e-07 2024-04-26T03:39:42.189 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.190 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.190 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:42.190 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493261+0000", 2024-04-26T03:39:42.190 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.4604000000000002e-05 2024-04-26T03:39:42.190 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.190 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.190 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:42.190 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493271+0000", 2024-04-26T03:39:42.190 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0550999999999999e-05 2024-04-26T03:39:42.190 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.190 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.190 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:42.190 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493279+0000", 2024-04-26T03:39:42.190 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.3790000000000006e-06 2024-04-26T03:39:42.191 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.191 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.191 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:42.191 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.506027+0000", 2024-04-26T03:39:42.191 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.012747058 2024-04-26T03:39:42.191 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.191 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.191 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:42.191 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.506166+0000", 2024-04-26T03:39:42.191 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000139357 2024-04-26T03:39:42.191 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.191 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.191 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:42.191 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.506670+0000", 2024-04-26T03:39:42.191 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00050440100000000002 2024-04-26T03:39:42.192 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.192 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.192 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:42.192 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.509149+0000", 2024-04-26T03:39:42.192 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.002478839 2024-04-26T03:39:42.192 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.192 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.192 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:42.192 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.519099+0000", 2024-04-26T03:39:42.192 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0099501880000000004 2024-04-26T03:39:42.192 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.192 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.192 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:42.192 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.519201+0000", 2024-04-26T03:39:42.192 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00010124 2024-04-26T03:39:42.193 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:42.193 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:42.193 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:42.193 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.193 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.193 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:39 3.bs0 3:d42e9299:::smithi09235267-13:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:42.193 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.493292+0000", 2024-04-26T03:39:42.193 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 50.673643574000003, 2024-04-26T03:39:42.193 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.029045519999999998, 2024-04-26T03:39:42.193 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:42.193 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:42.193 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "started", 2024-04-26T03:39:42.193 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:42.193 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:42.194 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:42.194 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 39 2024-04-26T03:39:42.194 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.194 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:42.194 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.194 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:42.194 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493292+0000", 2024-04-26T03:39:42.194 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:42.194 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.194 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.194 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:42.194 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493292+0000", 2024-04-26T03:39:42.194 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:42.194 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.194 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.195 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:42.195 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493292+0000", 2024-04-26T03:39:42.195 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.3099999999999997e-07 2024-04-26T03:39:42.195 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.195 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.195 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:42.195 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493296+0000", 2024-04-26T03:39:42.195 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.0820000000000001e-06 2024-04-26T03:39:42.195 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.195 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.195 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:42.195 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493304+0000", 2024-04-26T03:39:42.195 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.0269999999999995e-06 2024-04-26T03:39:42.195 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.195 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.196 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:42.196 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.493311+0000", 2024-04-26T03:39:42.196 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.1140000000000002e-06 2024-04-26T03:39:42.196 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.196 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.196 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:42.196 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.506997+0000", 2024-04-26T03:39:42.196 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.013686218999999999 2024-04-26T03:39:42.197 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.197 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.197 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:42.197 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.507073+0000", 2024-04-26T03:39:42.197 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.6766000000000003e-05 2024-04-26T03:39:42.197 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.197 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.197 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:42.197 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.519285+0000", 2024-04-26T03:39:42.198 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.012211324000000001 2024-04-26T03:39:42.198 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.198 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.198 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:42.198 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.519498+0000", 2024-04-26T03:39:42.198 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000213348 2024-04-26T03:39:42.198 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.198 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.198 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:42.198 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.522337+0000", 2024-04-26T03:39:42.198 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0028389090000000001 2024-04-26T03:39:42.198 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:42.198 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:42.198 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:42.199 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.199 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.199 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:128 3.bs0 3:d339032c:::smithi09235267-43:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:42.199 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.600125+0000", 2024-04-26T03:39:42.199 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 50.566810011000001, 2024-04-26T03:39:42.199 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.018721534000000001, 2024-04-26T03:39:42.199 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:42.199 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:42.199 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:42.199 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:42.199 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:42.199 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:42.199 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 128 2024-04-26T03:39:42.199 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.200 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:42.200 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.200 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:42.200 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.600125+0000", 2024-04-26T03:39:42.200 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:42.200 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.200 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.200 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:42.200 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.600125+0000", 2024-04-26T03:39:42.200 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:42.200 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.200 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.200 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:42.200 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.600126+0000", 2024-04-26T03:39:42.201 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.2100000000000002e-07 2024-04-26T03:39:42.201 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.201 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.201 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:42.201 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.600128+0000", 2024-04-26T03:39:42.201 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.5150000000000001e-06 2024-04-26T03:39:42.201 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.201 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.201 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:42.201 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.600133+0000", 2024-04-26T03:39:42.201 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.4240000000000001e-06 2024-04-26T03:39:42.201 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.201 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.201 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:42.201 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.600137+0000", 2024-04-26T03:39:42.202 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.4300000000000002e-06 2024-04-26T03:39:42.202 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.202 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.202 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:42.202 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.607830+0000", 2024-04-26T03:39:42.202 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0076931029999999997 2024-04-26T03:39:42.202 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.202 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.202 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:42.202 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.607888+0000", 2024-04-26T03:39:42.202 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.8260999999999997e-05 2024-04-26T03:39:42.202 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.202 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.202 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:42.203 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.608325+0000", 2024-04-26T03:39:42.203 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00043624700000000001 2024-04-26T03:39:42.203 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.203 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.203 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:42.203 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.608965+0000", 2024-04-26T03:39:42.203 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00064069400000000003 2024-04-26T03:39:42.203 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.203 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.203 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:42.203 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.618751+0000", 2024-04-26T03:39:42.203 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0097859209999999995 2024-04-26T03:39:42.203 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.203 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.203 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:42.204 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.618847+0000", 2024-04-26T03:39:42.204 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.5518000000000003e-05 2024-04-26T03:39:42.204 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:42.204 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:42.204 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:42.204 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.204 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.204 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:129 3.bs0 3:d339032c:::smithi09235267-43:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:42.204 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:38:51.600159+0000", 2024-04-26T03:39:42.204 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 50.566776011000002, 2024-04-26T03:39:42.204 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.020601199000000001, 2024-04-26T03:39:42.204 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:42.204 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:42.204 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "started", 2024-04-26T03:39:42.205 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:42.205 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:42.205 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:42.205 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 129 2024-04-26T03:39:42.205 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.205 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:42.205 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.205 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:42.205 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.600159+0000", 2024-04-26T03:39:42.205 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:42.205 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.205 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.205 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:42.205 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.600159+0000", 2024-04-26T03:39:42.205 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:42.206 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.206 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.206 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:42.206 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.600159+0000", 2024-04-26T03:39:42.206 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.2000000000000001e-07 2024-04-26T03:39:42.206 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.206 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.206 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:42.206 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.600162+0000", 2024-04-26T03:39:42.206 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.3070000000000001e-06 2024-04-26T03:39:42.206 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.206 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.206 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:42.206 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.600170+0000", 2024-04-26T03:39:42.207 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.568e-06 2024-04-26T03:39:42.207 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.207 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.207 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:42.207 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.600174+0000", 2024-04-26T03:39:42.207 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.241e-06 2024-04-26T03:39:42.207 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.207 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.207 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:42.207 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.608991+0000", 2024-04-26T03:39:42.207 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0088175800000000002 2024-04-26T03:39:42.207 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.207 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.207 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:42.207 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.609072+0000", 2024-04-26T03:39:42.208 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.1075000000000006e-05 2024-04-26T03:39:42.208 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.208 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.208 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:42.208 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.618897+0000", 2024-04-26T03:39:42.208 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0098250460000000005 2024-04-26T03:39:42.208 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.208 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.208 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:42.208 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.618974+0000", 2024-04-26T03:39:42.208 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.7075000000000003e-05 2024-04-26T03:39:42.208 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.208 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.208 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:42.208 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:38:51.620760+0000", 2024-04-26T03:39:42.209 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0017859869999999999 2024-04-26T03:39:42.209 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:42.209 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:42.209 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:42.209 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.209 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.209 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.13s5 56/55 ECSubWrite(tid=104, reqid=client.4556.0:604, at_version=56'47, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:39:42.209 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:11.774442+0000", 2024-04-26T03:39:42.209 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 30.392492953000001, 2024-04-26T03:39:42.209 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.016284751, 2024-04-26T03:39:42.209 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:42.209 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:42.209 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:42.209 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:42.210 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.210 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:42.210 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.774442+0000", 2024-04-26T03:39:42.210 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:42.210 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.210 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.210 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:42.210 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.774442+0000", 2024-04-26T03:39:42.210 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:42.210 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.210 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.210 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:42.210 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.774444+0000", 2024-04-26T03:39:42.210 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.406e-06 2024-04-26T03:39:42.210 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.211 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.211 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:42.211 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.776475+0000", 2024-04-26T03:39:42.211 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.002030933 2024-04-26T03:39:42.211 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.211 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.211 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:42.211 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.776486+0000", 2024-04-26T03:39:42.211 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.1221e-05 2024-04-26T03:39:42.211 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.211 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.211 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:42.211 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.776492+0000", 2024-04-26T03:39:42.211 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.9499999999999998e-06 2024-04-26T03:39:42.211 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.212 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.212 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:42.212 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.776534+0000", 2024-04-26T03:39:42.212 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.2404000000000003e-05 2024-04-26T03:39:42.212 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.212 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.212 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:42.212 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.776540+0000", 2024-04-26T03:39:42.212 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.1249999999999998e-06 2024-04-26T03:39:42.212 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.212 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.212 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:42.212 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.790691+0000", 2024-04-26T03:39:42.212 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.014150805000000001 2024-04-26T03:39:42.212 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.213 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.213 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:42.213 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:11.790727+0000", 2024-04-26T03:39:42.213 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.5907e-05 2024-04-26T03:39:42.213 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:42.213 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:42.213 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:42.213 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.213 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.213 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:684 3.bs0 3:d42e9299:::smithi09235267-13:head [append 0~573440] snapc f=[f,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-04-26T03:39:42.213 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:15.930550+0000", 2024-04-26T03:39:42.213 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 26.236384744999999, 2024-04-26T03:39:42.213 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.020486684000000002, 2024-04-26T03:39:42.213 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:42.214 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:42.214 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:42.214 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:42.214 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:42.214 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:42.214 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 684 2024-04-26T03:39:42.214 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.214 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:42.214 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.214 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:42.214 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.930550+0000", 2024-04-26T03:39:42.214 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:42.214 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.214 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.215 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:42.215 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.930550+0000", 2024-04-26T03:39:42.215 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:42.215 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.215 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.215 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:42.215 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.930552+0000", 2024-04-26T03:39:42.215 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.4169999999999999e-06 2024-04-26T03:39:42.215 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.215 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.215 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:42.215 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.931191+0000", 2024-04-26T03:39:42.215 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00063884100000000002 2024-04-26T03:39:42.215 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.216 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.216 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:42.216 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.931205+0000", 2024-04-26T03:39:42.216 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.4521999999999999e-05 2024-04-26T03:39:42.216 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.216 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.216 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:42.216 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.931213+0000", 2024-04-26T03:39:42.216 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.8769999999999993e-06 2024-04-26T03:39:42.216 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.216 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.216 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:42.216 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.934870+0000", 2024-04-26T03:39:42.216 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0036572810000000001 2024-04-26T03:39:42.216 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.217 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.217 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:42.217 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.934956+0000", 2024-04-26T03:39:42.217 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.5513999999999995e-05 2024-04-26T03:39:42.217 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.217 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.217 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:42.217 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.935556+0000", 2024-04-26T03:39:42.217 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00060064099999999996 2024-04-26T03:39:42.217 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.217 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.217 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:42.217 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.950858+0000", 2024-04-26T03:39:42.217 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.015301726999999999 2024-04-26T03:39:42.217 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.218 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.218 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:42.218 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.950956+0000", 2024-04-26T03:39:42.218 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.7608000000000005e-05 2024-04-26T03:39:42.218 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.218 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.218 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:42.218 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.951037+0000", 2024-04-26T03:39:42.218 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.1255999999999996e-05 2024-04-26T03:39:42.218 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:42.218 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:42.218 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:42.218 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.218 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.219 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:685 3.bs0 3:d42e9299:::smithi09235267-13:head [append 0~688128] snapc f=[f,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-04-26T03:39:42.219 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:15.934342+0000", 2024-04-26T03:39:42.219 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 26.232593488999999, 2024-04-26T03:39:42.219 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.017104579000000002, 2024-04-26T03:39:42.219 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:42.219 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:42.219 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:42.219 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:42.219 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:42.219 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:42.219 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 685 2024-04-26T03:39:42.219 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.220 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:42.220 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.220 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:42.220 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.934342+0000", 2024-04-26T03:39:42.220 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:42.220 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.220 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.220 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:42.220 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.934342+0000", 2024-04-26T03:39:42.220 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:42.220 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.220 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.220 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:42.220 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.934343+0000", 2024-04-26T03:39:42.220 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.0800000000000003e-07 2024-04-26T03:39:42.221 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.221 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.221 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:42.221 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.934992+0000", 2024-04-26T03:39:42.221 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00064962500000000003 2024-04-26T03:39:42.221 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.221 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.221 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:42.221 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.935001+0000", 2024-04-26T03:39:42.221 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.0059999999999998e-06 2024-04-26T03:39:42.221 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.221 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.221 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:42.221 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.935006+0000", 2024-04-26T03:39:42.222 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.9740000000000001e-06 2024-04-26T03:39:42.222 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.222 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.222 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:42.222 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.939492+0000", 2024-04-26T03:39:42.222 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0044860309999999997 2024-04-26T03:39:42.222 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.222 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.222 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:42.222 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.939600+0000", 2024-04-26T03:39:42.222 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000108044 2024-04-26T03:39:42.222 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.222 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.222 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:42.223 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.940601+0000", 2024-04-26T03:39:42.223 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0010011180000000001 2024-04-26T03:39:42.223 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.223 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.223 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:42.223 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.951051+0000", 2024-04-26T03:39:42.223 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.010449738 2024-04-26T03:39:42.223 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.223 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.223 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:42.223 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.951390+0000", 2024-04-26T03:39:42.223 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00033857100000000002 2024-04-26T03:39:42.223 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.223 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.223 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:42.224 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.951446+0000", 2024-04-26T03:39:42.224 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.6564000000000003e-05 2024-04-26T03:39:42.224 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:42.224 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:42.224 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:42.224 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.224 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.224 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:686 3.bs0 3:d42e9299:::smithi09235267-13:head [append 0~294912] snapc f=[f,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-04-26T03:39:42.224 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:15.935994+0000", 2024-04-26T03:39:42.224 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 26.230941116, 2024-04-26T03:39:42.224 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.016832585000000001, 2024-04-26T03:39:42.224 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:42.224 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:42.224 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:42.224 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:42.225 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:42.225 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:42.225 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 686 2024-04-26T03:39:42.225 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.225 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:42.225 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.225 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:42.225 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.935994+0000", 2024-04-26T03:39:42.225 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:42.225 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.225 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.225 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:42.225 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.935994+0000", 2024-04-26T03:39:42.225 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:42.225 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.226 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.226 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:42.226 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.935995+0000", 2024-04-26T03:39:42.226 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.4130000000000001e-06 2024-04-26T03:39:42.226 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.226 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.226 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:42.226 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.936361+0000", 2024-04-26T03:39:42.226 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00036540299999999998 2024-04-26T03:39:42.226 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.226 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.226 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:42.226 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.936376+0000", 2024-04-26T03:39:42.227 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.4983e-05 2024-04-26T03:39:42.227 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.227 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.227 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:42.227 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.936384+0000", 2024-04-26T03:39:42.227 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.2530000000000004e-06 2024-04-26T03:39:42.227 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.227 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.227 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:42.227 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.947040+0000", 2024-04-26T03:39:42.227 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.010655825000000001 2024-04-26T03:39:42.227 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.227 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.227 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:42.227 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.947149+0000", 2024-04-26T03:39:42.228 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00010880299999999999 2024-04-26T03:39:42.228 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.228 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.228 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:42.228 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.947876+0000", 2024-04-26T03:39:42.228 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00072771100000000003 2024-04-26T03:39:42.228 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.228 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.228 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:42.228 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.952293+0000", 2024-04-26T03:39:42.228 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0044169999999999999 2024-04-26T03:39:42.228 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.228 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.228 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:42.228 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.952760+0000", 2024-04-26T03:39:42.229 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00046631800000000002 2024-04-26T03:39:42.229 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.229 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.229 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:42.229 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.952827+0000", 2024-04-26T03:39:42.229 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.6876000000000001e-05 2024-04-26T03:39:42.229 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:42.229 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:42.229 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:42.229 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.229 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.229 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:687 3.bs0 3:d42e9299:::smithi09235267-13:head [setxattr _header (57) in=7b] snapc f=[f,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-04-26T03:39:42.229 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:15.936397+0000", 2024-04-26T03:39:42.229 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 26.230537656999999, 2024-04-26T03:39:42.229 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.018657155000000002, 2024-04-26T03:39:42.230 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:42.230 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:42.230 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:42.230 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:42.230 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:42.230 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:42.230 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 687 2024-04-26T03:39:42.230 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.230 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:42.230 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.230 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:42.230 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.936397+0000", 2024-04-26T03:39:42.230 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:42.230 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.231 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.231 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:42.231 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.936397+0000", 2024-04-26T03:39:42.231 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:42.231 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.231 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.231 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:42.231 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.936398+0000", 2024-04-26T03:39:42.231 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.5300000000000004e-07 2024-04-26T03:39:42.231 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.231 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.231 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:42.231 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.936402+0000", 2024-04-26T03:39:42.231 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.8580000000000002e-06 2024-04-26T03:39:42.232 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.232 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.232 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:42.232 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.936412+0000", 2024-04-26T03:39:42.232 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.5340000000000005e-06 2024-04-26T03:39:42.232 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.232 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.232 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:42.232 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.936418+0000", 2024-04-26T03:39:42.232 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.5350000000000001e-06 2024-04-26T03:39:42.232 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.232 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.232 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:42.232 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.951554+0000", 2024-04-26T03:39:42.232 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.015135964 2024-04-26T03:39:42.233 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.233 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.233 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:42.233 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.951659+0000", 2024-04-26T03:39:42.233 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000105376 2024-04-26T03:39:42.233 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.233 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.233 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:42.233 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.952057+0000", 2024-04-26T03:39:42.233 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000397258 2024-04-26T03:39:42.233 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.233 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.233 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:42.233 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.952942+0000", 2024-04-26T03:39:42.233 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00088508900000000004 2024-04-26T03:39:42.234 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.234 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.234 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:42.234 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.954966+0000", 2024-04-26T03:39:42.234 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0020244669999999998 2024-04-26T03:39:42.234 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.234 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.234 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:42.234 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.955055+0000", 2024-04-26T03:39:42.234 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.8420999999999999e-05 2024-04-26T03:39:42.234 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:42.234 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:42.234 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:42.234 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.235 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.235 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:688 3.bs0 3:d42e9299:::smithi09235267-13:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e60)", 2024-04-26T03:39:42.235 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:15.936428+0000", 2024-04-26T03:39:42.235 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 26.230507126999999, 2024-04-26T03:39:42.235 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.020098925, 2024-04-26T03:39:42.235 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:42.235 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:42.235 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "started", 2024-04-26T03:39:42.235 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:42.235 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:42.235 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:42.236 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 688 2024-04-26T03:39:42.236 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.236 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:42.236 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.236 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:42.236 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.936428+0000", 2024-04-26T03:39:42.236 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:42.236 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.236 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.236 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:42.236 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.936428+0000", 2024-04-26T03:39:42.236 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:42.236 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.236 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.236 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:42.237 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.936429+0000", 2024-04-26T03:39:42.237 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.44e-07 2024-04-26T03:39:42.237 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.237 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.237 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:42.237 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.936431+0000", 2024-04-26T03:39:42.237 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.621e-06 2024-04-26T03:39:42.237 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.237 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.237 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:42.237 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.936440+0000", 2024-04-26T03:39:42.237 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.1379999999999996e-06 2024-04-26T03:39:42.237 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.237 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.237 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:42.238 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.936446+0000", 2024-04-26T03:39:42.238 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.0990000000000004e-06 2024-04-26T03:39:42.238 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.238 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.238 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:42.238 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.952325+0000", 2024-04-26T03:39:42.238 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.015878541999999999 2024-04-26T03:39:42.238 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.238 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.238 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:42.238 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.952405+0000", 2024-04-26T03:39:42.238 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.0042000000000004e-05 2024-04-26T03:39:42.238 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.239 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.239 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:42.239 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.955424+0000", 2024-04-26T03:39:42.239 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0030187619999999999 2024-04-26T03:39:42.239 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.239 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.239 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:42.239 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.955503+0000", 2024-04-26T03:39:42.239 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.8850000000000006e-05 2024-04-26T03:39:42.239 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.239 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.239 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:42.239 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.956527+0000", 2024-04-26T03:39:42.239 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.001024327 2024-04-26T03:39:42.240 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:42.240 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:42.240 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:42.240 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.240 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.240 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.10s3 60/48 ECSubWrite(tid=272, reqid=client.4556.0:703, at_version=60'20, trim_to=0'0, roll_forward_to=60'18))", 2024-04-26T03:39:42.240 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:15.958315+0000", 2024-04-26T03:39:42.240 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 26.208620357000001, 2024-04-26T03:39:42.240 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.014611541, 2024-04-26T03:39:42.240 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:42.240 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:42.240 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:42.240 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:42.240 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.241 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:42.241 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.958315+0000", 2024-04-26T03:39:42.241 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:42.241 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.241 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.241 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:42.241 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.958315+0000", 2024-04-26T03:39:42.242 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:42.242 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.242 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.242 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:42.242 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.958315+0000", 2024-04-26T03:39:42.242 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.3099999999999997e-07 2024-04-26T03:39:42.242 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.242 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.242 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:42.242 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.959300+0000", 2024-04-26T03:39:42.242 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00098483399999999997 2024-04-26T03:39:42.242 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.243 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.243 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:42.243 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.959313+0000", 2024-04-26T03:39:42.243 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.2662e-05 2024-04-26T03:39:42.243 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.243 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.243 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:42.243 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.959321+0000", 2024-04-26T03:39:42.243 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.1799999999999996e-06 2024-04-26T03:39:42.243 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.243 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.243 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:42.243 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.959385+0000", 2024-04-26T03:39:42.243 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.4199e-05 2024-04-26T03:39:42.243 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.244 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.244 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:42.244 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.959396+0000", 2024-04-26T03:39:42.244 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0277e-05 2024-04-26T03:39:42.244 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.244 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.244 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:42.244 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.972883+0000", 2024-04-26T03:39:42.244 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.013487025 2024-04-26T03:39:42.244 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.244 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.244 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:42.244 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:15.972926+0000", 2024-04-26T03:39:42.244 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.3733000000000003e-05 2024-04-26T03:39:42.245 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:42.245 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:42.245 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:42.245 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.245 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.245 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:746 3.bs0 3:d42e9299:::smithi09235267-13:head [append 0~573440] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:42.245 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:20.717432+0000", 2024-04-26T03:39:42.245 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 21.449502988999999, 2024-04-26T03:39:42.245 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.020989714, 2024-04-26T03:39:42.245 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:42.245 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:42.245 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:42.245 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:42.245 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:42.245 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:42.246 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 746 2024-04-26T03:39:42.246 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.246 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:42.246 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.246 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:42.246 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.717432+0000", 2024-04-26T03:39:42.246 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:42.246 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.246 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.246 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:42.246 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.717432+0000", 2024-04-26T03:39:42.246 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:42.246 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.246 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.246 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:42.247 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.717434+0000", 2024-04-26T03:39:42.247 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.6050000000000001e-06 2024-04-26T03:39:42.247 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.247 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.247 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:42.247 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.717989+0000", 2024-04-26T03:39:42.247 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00055509299999999999 2024-04-26T03:39:42.247 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.247 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.247 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:42.247 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.718007+0000", 2024-04-26T03:39:42.247 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.8349999999999999e-05 2024-04-26T03:39:42.247 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.247 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.248 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:42.248 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.718018+0000", 2024-04-26T03:39:42.248 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.082e-05 2024-04-26T03:39:42.248 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.248 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.248 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:42.248 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.722909+0000", 2024-04-26T03:39:42.248 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0048909770000000003 2024-04-26T03:39:42.248 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.248 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.248 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:42.248 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.723028+0000", 2024-04-26T03:39:42.248 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00011944599999999999 2024-04-26T03:39:42.248 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.248 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.249 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:42.249 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.723782+0000", 2024-04-26T03:39:42.249 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00075392899999999997 2024-04-26T03:39:42.249 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.249 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.249 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:42.249 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.737780+0000", 2024-04-26T03:39:42.249 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.013997921999999999 2024-04-26T03:39:42.249 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.249 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.249 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:42.249 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.738324+0000", 2024-04-26T03:39:42.249 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00054407299999999995 2024-04-26T03:39:42.249 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.249 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.250 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:42.250 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.738422+0000", 2024-04-26T03:39:42.250 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.7498999999999996e-05 2024-04-26T03:39:42.250 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:42.250 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:42.250 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:42.250 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.250 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.250 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:747 3.bs0 3:d42e9299:::smithi09235267-13:head [append 0~753664] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:42.250 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:20.721726+0000", 2024-04-26T03:39:42.250 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 21.445209513999998, 2024-04-26T03:39:42.250 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.018181914, 2024-04-26T03:39:42.250 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:42.250 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:42.251 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:42.251 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:42.251 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:42.251 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:42.251 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 747 2024-04-26T03:39:42.251 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.251 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:42.251 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.251 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:42.251 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.721726+0000", 2024-04-26T03:39:42.251 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:42.251 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.251 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.251 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:42.251 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.721726+0000", 2024-04-26T03:39:42.252 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:42.252 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.252 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.252 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:42.252 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.721727+0000", 2024-04-26T03:39:42.252 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.762e-06 2024-04-26T03:39:42.252 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.252 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.252 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:42.252 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.722450+0000", 2024-04-26T03:39:42.252 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00072252799999999995 2024-04-26T03:39:42.252 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.252 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.252 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:42.252 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.722459+0000", 2024-04-26T03:39:42.253 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.1759999999999999e-06 2024-04-26T03:39:42.253 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.253 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.253 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:42.253 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.722464+0000", 2024-04-26T03:39:42.253 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.9239999999999998e-06 2024-04-26T03:39:42.253 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.253 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.253 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:42.253 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.727880+0000", 2024-04-26T03:39:42.253 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0054160809999999997 2024-04-26T03:39:42.253 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.254 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.254 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:42.254 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.727949+0000", 2024-04-26T03:39:42.254 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.9286e-05 2024-04-26T03:39:42.254 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.254 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.254 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:42.254 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.728724+0000", 2024-04-26T03:39:42.254 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00077420600000000005 2024-04-26T03:39:42.254 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.254 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.254 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:42.254 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.737782+0000", 2024-04-26T03:39:42.255 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0090582360000000008 2024-04-26T03:39:42.255 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.255 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.255 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:42.255 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.739827+0000", 2024-04-26T03:39:42.255 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0020447070000000002 2024-04-26T03:39:42.255 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.255 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.255 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:42.255 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.739908+0000", 2024-04-26T03:39:42.255 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.1007999999999994e-05 2024-04-26T03:39:42.255 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:42.255 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:42.255 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:42.255 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.256 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.256 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:748 3.bs0 3:d42e9299:::smithi09235267-13:head [append 0~229376] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:42.256 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:20.723036+0000", 2024-04-26T03:39:42.256 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 21.443898707999999, 2024-04-26T03:39:42.256 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.018068820999999999, 2024-04-26T03:39:42.256 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:42.256 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:42.256 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:42.256 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:42.256 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:42.256 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:42.256 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 748 2024-04-26T03:39:42.256 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.256 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:42.257 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.257 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:42.257 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.723036+0000", 2024-04-26T03:39:42.257 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:42.257 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.257 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.257 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:42.257 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.723036+0000", 2024-04-26T03:39:42.257 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:42.257 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.257 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.257 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:42.257 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.723037+0000", 2024-04-26T03:39:42.257 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.6099999999999999e-07 2024-04-26T03:39:42.257 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.258 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.258 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:42.258 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.723287+0000", 2024-04-26T03:39:42.258 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00025004699999999999 2024-04-26T03:39:42.258 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.258 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.258 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:42.258 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.723297+0000", 2024-04-26T03:39:42.258 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.3540000000000006e-06 2024-04-26T03:39:42.258 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.258 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.258 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:42.258 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.723306+0000", 2024-04-26T03:39:42.258 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.8729999999999996e-06 2024-04-26T03:39:42.259 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.259 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.259 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:42.259 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.735103+0000", 2024-04-26T03:39:42.259 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.011797813000000001 2024-04-26T03:39:42.259 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.259 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.259 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:42.259 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.735219+0000", 2024-04-26T03:39:42.259 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000115074 2024-04-26T03:39:42.259 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.259 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.259 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:42.259 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.735876+0000", 2024-04-26T03:39:42.259 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00065787099999999998 2024-04-26T03:39:42.260 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.260 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.260 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:42.260 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.740018+0000", 2024-04-26T03:39:42.260 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0041418519999999997 2024-04-26T03:39:42.260 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.260 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.260 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:42.260 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.741049+0000", 2024-04-26T03:39:42.260 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0010308509999999999 2024-04-26T03:39:42.260 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.260 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.260 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:42.260 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.741105+0000", 2024-04-26T03:39:42.260 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.6125000000000002e-05 2024-04-26T03:39:42.261 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:42.261 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:42.261 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:42.261 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.261 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.261 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:749 3.bs0 3:d42e9299:::smithi09235267-13:head [setxattr _header (57) in=7b] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:42.261 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:20.723321+0000", 2024-04-26T03:39:42.261 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 21.443614225000001, 2024-04-26T03:39:42.261 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.018015394000000001, 2024-04-26T03:39:42.261 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:42.261 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:42.261 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:42.261 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:42.261 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:42.262 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:42.262 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 749 2024-04-26T03:39:42.262 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.262 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:42.262 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.262 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:42.262 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.723321+0000", 2024-04-26T03:39:42.262 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:42.262 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.262 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.262 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:42.262 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.723321+0000", 2024-04-26T03:39:42.262 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:42.262 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.262 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.263 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:42.263 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.723322+0000", 2024-04-26T03:39:42.263 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 8.3500000000000005e-07 2024-04-26T03:39:42.263 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.263 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.263 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:42.263 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.723326+0000", 2024-04-26T03:39:42.263 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.8700000000000002e-06 2024-04-26T03:39:42.263 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.263 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.263 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:42.263 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.723333+0000", 2024-04-26T03:39:42.263 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.3529999999999997e-06 2024-04-26T03:39:42.263 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.264 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.264 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:42.264 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.723339+0000", 2024-04-26T03:39:42.264 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 5.7490000000000004e-06 2024-04-26T03:39:42.264 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.264 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.264 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:42.264 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.738816+0000", 2024-04-26T03:39:42.264 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.015476867 2024-04-26T03:39:42.264 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.264 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.264 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:42.264 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.738908+0000", 2024-04-26T03:39:42.264 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.2083999999999995e-05 2024-04-26T03:39:42.264 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.265 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.265 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:42.265 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.739356+0000", 2024-04-26T03:39:42.265 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00044818599999999999 2024-04-26T03:39:42.265 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.265 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.265 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:42.265 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.740366+0000", 2024-04-26T03:39:42.265 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.001010101 2024-04-26T03:39:42.265 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.265 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.265 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "commit_sent", 2024-04-26T03:39:42.265 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.741225+0000", 2024-04-26T03:39:42.265 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.00085920799999999998 2024-04-26T03:39:42.265 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.266 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.266 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:42.266 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.741336+0000", 2024-04-26T03:39:42.266 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.000111141 2024-04-26T03:39:42.266 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:42.266 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:42.266 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:42.266 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.266 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.266 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "osd_op(client.4556.0:750 3.bs0 3:d42e9299:::smithi09235267-13:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:42.266 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:20.723347+0000", 2024-04-26T03:39:42.266 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 21.443588031000001, 2024-04-26T03:39:42.266 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.019487478999999999, 2024-04-26T03:39:42.266 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:42.267 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:42.267 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "started", 2024-04-26T03:39:42.267 INFO:teuthology.orchestra.run.smithi028.stdout: "client_info": { 2024-04-26T03:39:42.267 INFO:teuthology.orchestra.run.smithi028.stdout: "client": "client.4556", 2024-04-26T03:39:42.267 INFO:teuthology.orchestra.run.smithi028.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:42.267 INFO:teuthology.orchestra.run.smithi028.stdout: "tid": 750 2024-04-26T03:39:42.267 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.267 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:42.267 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.267 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:42.267 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.723347+0000", 2024-04-26T03:39:42.267 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:42.267 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.267 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.268 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:42.268 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.723347+0000", 2024-04-26T03:39:42.268 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:42.268 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.268 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.268 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:42.268 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.723348+0000", 2024-04-26T03:39:42.268 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 4.9900000000000001e-07 2024-04-26T03:39:42.268 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.268 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.268 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:42.268 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.723350+0000", 2024-04-26T03:39:42.268 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.6800000000000002e-06 2024-04-26T03:39:42.268 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.268 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.269 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:42.269 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.723357+0000", 2024-04-26T03:39:42.269 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 6.4609999999999996e-06 2024-04-26T03:39:42.269 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.269 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.269 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:42.269 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.723360+0000", 2024-04-26T03:39:42.269 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.455e-06 2024-04-26T03:39:42.269 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.269 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.269 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:42.269 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.739606+0000", 2024-04-26T03:39:42.269 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.016245622000000001 2024-04-26T03:39:42.269 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.270 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.270 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:42.270 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.739704+0000", 2024-04-26T03:39:42.270 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 9.8397e-05 2024-04-26T03:39:42.270 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.270 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.270 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:42.270 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.741743+0000", 2024-04-26T03:39:42.270 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.002038593 2024-04-26T03:39:42.270 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.270 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.270 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "started", 2024-04-26T03:39:42.270 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.741818+0000", 2024-04-26T03:39:42.270 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.5142999999999995e-05 2024-04-26T03:39:42.270 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.271 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.271 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:42.271 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:20.742835+0000", 2024-04-26T03:39:42.271 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.001016629 2024-04-26T03:39:42.271 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:42.271 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:42.271 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:42.271 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.271 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.271 INFO:teuthology.orchestra.run.smithi028.stdout: "description": "MOSDECSubOpWrite(3.10s3 80/48 ECSubWrite(tid=322, reqid=client.4556.0:895, at_version=80'35, trim_to=0'0, roll_forward_to=75'33))", 2024-04-26T03:39:42.271 INFO:teuthology.orchestra.run.smithi028.stdout: "initiated_at": "2024-04-26T03:39:35.740560+0000", 2024-04-26T03:39:42.271 INFO:teuthology.orchestra.run.smithi028.stdout: "age": 6.4263754649999996, 2024-04-26T03:39:42.271 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.039238213000000001, 2024-04-26T03:39:42.272 INFO:teuthology.orchestra.run.smithi028.stdout: "continuous": false, 2024-04-26T03:39:42.272 INFO:teuthology.orchestra.run.smithi028.stdout: "type_data": { 2024-04-26T03:39:42.272 INFO:teuthology.orchestra.run.smithi028.stdout: "flag_point": "reached pg", 2024-04-26T03:39:42.272 INFO:teuthology.orchestra.run.smithi028.stdout: "events": [ 2024-04-26T03:39:42.272 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.272 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "initiated", 2024-04-26T03:39:42.272 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:35.740560+0000", 2024-04-26T03:39:42.272 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:42.272 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.272 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.272 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "header_read", 2024-04-26T03:39:42.272 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:35.740560+0000", 2024-04-26T03:39:42.272 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0 2024-04-26T03:39:42.272 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.273 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.273 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "throttled", 2024-04-26T03:39:42.273 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:35.740560+0000", 2024-04-26T03:39:42.273 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 2.2000000000000001e-07 2024-04-26T03:39:42.273 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.273 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.273 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "all_read", 2024-04-26T03:39:42.273 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:35.746519+0000", 2024-04-26T03:39:42.273 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.0059586279999999997 2024-04-26T03:39:42.273 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.273 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.273 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "dispatched", 2024-04-26T03:39:42.273 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:35.746538+0000", 2024-04-26T03:39:42.273 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.9894e-05 2024-04-26T03:39:42.273 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.274 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.274 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "queued_for_pg", 2024-04-26T03:39:42.274 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:35.746549+0000", 2024-04-26T03:39:42.274 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.0183999999999999e-05 2024-04-26T03:39:42.274 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.274 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.274 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "reached_pg", 2024-04-26T03:39:42.274 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:35.746626+0000", 2024-04-26T03:39:42.274 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 7.7694000000000005e-05 2024-04-26T03:39:42.274 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.274 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.274 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_started", 2024-04-26T03:39:42.274 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:35.746639+0000", 2024-04-26T03:39:42.274 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 1.279e-05 2024-04-26T03:39:42.275 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.275 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.275 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "sub_op_committed", 2024-04-26T03:39:42.275 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:35.779762+0000", 2024-04-26T03:39:42.275 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 0.033123227999999998 2024-04-26T03:39:42.275 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T03:39:42.275 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T03:39:42.275 INFO:teuthology.orchestra.run.smithi028.stdout: "event": "done", 2024-04-26T03:39:42.275 INFO:teuthology.orchestra.run.smithi028.stdout: "time": "2024-04-26T03:39:35.779798+0000", 2024-04-26T03:39:42.275 INFO:teuthology.orchestra.run.smithi028.stdout: "duration": 3.5574999999999997e-05 2024-04-26T03:39:42.275 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:42.275 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:42.275 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:42.275 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T03:39:42.275 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T03:39:42.276 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:39:42.277 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-04-26T03:39:42.278 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-04-26T03:39:42.276+0000 7f745118d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 31821) UID: 0 2024-04-26T03:39:42.378 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:39:42.376+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:39:42.411 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-26T03:39:42.426 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:39:42.426 INFO:teuthology.orchestra.run.smithi028.stdout: "ops": [], 2024-04-26T03:39:42.426 INFO:teuthology.orchestra.run.smithi028.stdout: "num_ops": 0 2024-04-26T03:39:42.426 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:39:42.435 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-04-26T03:39:42.479 INFO:tasks.ceph.osd.4.smithi028.stderr:2024-04-26T03:39:42.477+0000 7fd9511b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31963) UID: 0 2024-04-26T03:39:42.537 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0: {} 2024-04-26T03:39:42.538 INFO:teuthology.orchestra.run.smithi028.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T03:39:42.556 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1: {} 2024-04-26T03:39:42.557 INFO:teuthology.orchestra.run.smithi028.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T03:39:42.576 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2: {} 2024-04-26T03:39:42.576 INFO:teuthology.orchestra.run.smithi028.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T03:39:42.579 INFO:tasks.ceph.osd.15.smithi092.stderr:2024-04-26T03:39:42.578+0000 7f6d12dec640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 31824) UID: 0 2024-04-26T03:39:42.585 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:39:42.585 INFO:teuthology.orchestra.run.smithi028.stdout: "ops": [], 2024-04-26T03:39:42.586 INFO:teuthology.orchestra.run.smithi028.stdout: "complaint_time": 30, 2024-04-26T03:39:42.586 INFO:teuthology.orchestra.run.smithi028.stdout: "num_blocked_ops": 0 2024-04-26T03:39:42.586 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:39:42.594 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3: {} 2024-04-26T03:39:42.594 INFO:teuthology.orchestra.run.smithi028.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T03:39:42.595 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-04-26T03:39:42.611 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4: {} 2024-04-26T03:39:42.612 INFO:teuthology.orchestra.run.smithi028.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T03:39:42.630 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5: {} 2024-04-26T03:39:42.630 INFO:teuthology.orchestra.run.smithi028.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T03:39:42.665 INFO:teuthology.orchestra.run.smithi028.stdout:osd.6: {} 2024-04-26T03:39:42.665 INFO:teuthology.orchestra.run.smithi028.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T03:39:42.680 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:39:42.679+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:39:42.699 INFO:teuthology.orchestra.run.smithi028.stdout:osd.7: {} 2024-04-26T03:39:42.699 INFO:teuthology.orchestra.run.smithi028.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T03:39:42.722 INFO:teuthology.orchestra.run.smithi028.stdout:osd.8: {} 2024-04-26T03:39:42.723 INFO:teuthology.orchestra.run.smithi028.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T03:39:42.731 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:42.731 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:42.740 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-04-26T03:39:42.741 INFO:teuthology.orchestra.run.smithi028.stdout:osd.9: {} 2024-04-26T03:39:42.741 INFO:teuthology.orchestra.run.smithi028.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T03:39:42.759 INFO:teuthology.orchestra.run.smithi028.stdout:osd.10: {} 2024-04-26T03:39:42.760 INFO:teuthology.orchestra.run.smithi028.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T03:39:42.777 INFO:teuthology.orchestra.run.smithi028.stdout:osd.11: {} 2024-04-26T03:39:42.778 INFO:teuthology.orchestra.run.smithi028.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T03:39:42.781 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:39:42.779+0000 7fba0d2ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31961) UID: 0 2024-04-26T03:39:42.795 INFO:teuthology.orchestra.run.smithi028.stdout:osd.12: {} 2024-04-26T03:39:42.795 INFO:teuthology.orchestra.run.smithi028.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T03:39:42.813 INFO:teuthology.orchestra.run.smithi028.stderr:reweighted osd.14 to 1 (10000) 2024-04-26T03:39:42.814 INFO:teuthology.orchestra.run.smithi028.stdout:osd.13: {} 2024-04-26T03:39:42.814 INFO:teuthology.orchestra.run.smithi028.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T03:39:42.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:39:42.813+0000 7f146d08d640 0 -- 172.21.15.28:0/2358834001 >> [v2:172.21.15.66:6824/1534905602,v1:172.21.15.66:6825/1534905602] conn(0x7f146821fd30 msgr2=0x7f1468222120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-26T03:39:42.832 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 15 1 2024-04-26T03:39:42.834 INFO:teuthology.orchestra.run.smithi028.stdout:osd.14: {} 2024-04-26T03:39:42.835 INFO:teuthology.orchestra.run.smithi028.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T03:39:42.854 INFO:teuthology.orchestra.run.smithi028.stdout:osd.15: {} 2024-04-26T03:39:42.854 INFO:teuthology.orchestra.run.smithi028.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T03:39:42.857 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:42.857 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:42.868 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-04-26T03:39:42.909 INFO:tasks.ceph.osd.8.smithi028.stderr:2024-04-26T03:39:42.908+0000 7f5bb75d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 31968) UID: 0 2024-04-26T03:39:42.984 INFO:tasks.ceph.osd.15.smithi092.stderr:2024-04-26T03:39:42.982+0000 7f6d12dec640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 31824) UID: 0 2024-04-26T03:39:43.015 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:43.015 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:43.024 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-04-26T03:39:43.085 INFO:tasks.ceph.osd.14.smithi066.stderr:2024-04-26T03:39:43.084+0000 7f5395f08640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 31765) UID: 0 2024-04-26T03:39:43.132 INFO:teuthology.orchestra.run.smithi028.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:43.132 INFO:teuthology.orchestra.run.smithi028.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:43.141 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-26T03:39:43.187 INFO:tasks.ceph.osd.10.smithi066.stderr:2024-04-26T03:39:43.186+0000 7f130fcc7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 31763) UID: 0 2024-04-26T03:39:43.261 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:43.261 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:43.270 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-26T03:39:43.315 INFO:tasks.ceph.osd.9.smithi042.stderr:2024-04-26T03:39:43.313+0000 7fdac63b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 31960) UID: 0 2024-04-26T03:39:43.377 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:43.377 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:43.386 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-26T03:39:43.391 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-04-26T03:39:43.390+0000 7fee562fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31759) UID: 0 2024-04-26T03:39:43.491 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:39:43.490+0000 7f60d3c4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31757) UID: 0 2024-04-26T03:39:43.532 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:43.532 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:43.541 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-26T03:39:43.596 INFO:tasks.ceph.osd.14.smithi066.stderr:2024-04-26T03:39:43.596+0000 7f5395f08640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 31765) UID: 0 2024-04-26T03:39:43.648 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:43.648 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:43.656 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-26T03:39:43.699 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:39:43.698+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:39:43.763 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:43.763 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:43.773 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-26T03:39:43.800 INFO:tasks.ceph.osd.15.smithi092.stderr:2024-04-26T03:39:43.798+0000 7f6d12dec640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 31824) UID: 0 2024-04-26T03:39:43.821 INFO:teuthology.orchestra.run.smithi028.stderr:reweighted osd.15 to 1 (10000) 2024-04-26T03:39:43.822 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-26T03:39:43.836 INFO:tasks.rados.rados.0.smithi092.stdout:383: done (2 left) 2024-04-26T03:39:43.837 INFO:tasks.rados.rados.0.smithi092.stdout:384: done (1 left) 2024-04-26T03:39:43.837 INFO:tasks.rados.rados.0.smithi092.stdout:386: done (0 left) 2024-04-26T03:39:43.837 INFO:tasks.rados.rados.0.smithi092.stdout:387: delete oid 28 current snap is 30 2024-04-26T03:39:43.838 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-04-26T03:39:43.838 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T03:39:43.842 INFO:tasks.rados.rados.0.smithi092.stdout:387: done (0 left) 2024-04-26T03:39:43.842 INFO:tasks.rados.rados.0.smithi092.stdout:388: rollback oid 9 current snap is 30 2024-04-26T03:39:43.842 INFO:tasks.rados.rados.0.smithi092.stdout:rollback oid 9 to 19 2024-04-26T03:39:43.842 INFO:tasks.rados.rados.0.smithi092.stdout:389: snap_remove snap 26 2024-04-26T03:39:43.901 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-04-26T03:39:43.899+0000 7f745118d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 31821) UID: 0 2024-04-26T03:39:43.923 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:43.923 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:43.932 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-04-26T03:39:44.001 INFO:tasks.ceph.osd.15.smithi092.stderr:2024-04-26T03:39:44.000+0000 7f6d12dec640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 31824) UID: 0 2024-04-26T03:39:44.039 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:44.039 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:44.047 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-04-26T03:39:44.102 INFO:tasks.ceph.osd.14.smithi066.stderr:2024-04-26T03:39:44.102+0000 7f5395f08640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 31765) UID: 0 2024-04-26T03:39:44.155 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:44.155 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:44.163 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-04-26T03:39:44.182 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:39:44.183 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-04-26T03:39:44.194 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2024-04-26T03:39:44.063670+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":133644832,"num_objects":87,"num_object_clones":41,"num_object_copies":599,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":7,"num_read":472,"num_read_kb":236812,"num_write":930,"num_write_kb":126948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":31670272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":965,"ondisk_log_size":965,"up":144,"acting":144,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":153,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":771592,"kb_used_data":235152,"kb_used_omap":24,"kb_used_meta":536167,"kb_avail":1509177848,"statfs":{"total":1546188226560,"available":1545398116352,"internally_reserved":0,"allocated":240795648,"data_stored":315669090,"data_compressed":2033493,"data_compressed_allocated":4153344,"data_compressed_original":9199616,"omap_allocated":25522,"internal_metadata":549035086},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27607040,"num_objects":18,"num_object_clones":13,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":1,"num_read":41,"num_read_kb":39908,"num_write":77,"num_write_kb":16768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":25575424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000797"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":128,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.811530+0000","last_change":"2024-04-26T03:38:37.359252+0000","last_active":"2024-04-26T03:39:41.811530+0000","last_peered":"2024-04-26T03:39:41.811530+0000","last_clean":"2024-04-26T03:39:41.811530+0000","last_became_active":"2024-04-26T03:38:37.358910+0000","last_became_peered":"2024-04-26T03:38:37.358910+0000","last_unstale":"2024-04-26T03:39:41.811530+0000","last_undegraded":"2024-04-26T03:39:41.811530+0000","last_fullsized":"2024-04-26T03:39:41.811530+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:04.186001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"3.6","version":"81'20","reported_seq":304,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T03:39:43.074998+0000","last_change":"2024-04-26T03:39:39.705400+0000","last_active":"2024-04-26T03:39:43.074998+0000","last_peered":"2024-04-26T03:39:43.074998+0000","last_clean":"2024-04-26T03:39:43.074998+0000","last_became_active":"2024-04-26T03:39:27.699531+0000","last_became_peered":"2024-04-26T03:39:27.699531+0000","last_unstale":"2024-04-26T03:39:43.074998+0000","last_undegraded":"2024-04-26T03:39:43.074998+0000","last_fullsized":"2024-04-26T03:39:43.074998+0000","mapping_epoch":32,"log_start":"64'11","ondisk_log_start":"64'11","created":32,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'9","last_scrub_stamp":"2024-04-26T03:39:07.177824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:07.177824+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":11,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:24.874885+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0016197290000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,14,6,2,12,7,1],"acting":[0,14,6,2,12,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":128,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.822761+0000","last_change":"2024-04-26T03:38:36.484782+0000","last_active":"2024-04-26T03:39:41.822761+0000","last_peered":"2024-04-26T03:39:41.822761+0000","last_clean":"2024-04-26T03:39:41.822761+0000","last_became_active":"2024-04-26T03:38:36.484388+0000","last_became_peered":"2024-04-26T03:38:36.484388+0000","last_unstale":"2024-04-26T03:39:41.822761+0000","last_undegraded":"2024-04-26T03:39:41.822761+0000","last_fullsized":"2024-04-26T03:39:41.822761+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,13],"acting":[8,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.7","version":"49'10","reported_seq":299,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T03:39:42.815292+0000","last_change":"2024-04-26T03:39:39.696366+0000","last_active":"2024-04-26T03:39:42.815292+0000","last_peered":"2024-04-26T03:39:42.815292+0000","last_clean":"2024-04-26T03:39:42.815292+0000","last_became_active":"2024-04-26T03:39:23.698262+0000","last_became_peered":"2024-04-26T03:39:23.698262+0000","last_unstale":"2024-04-26T03:39:42.815292+0000","last_undegraded":"2024-04-26T03:39:42.815292+0000","last_fullsized":"2024-04-26T03:39:42.815292+0000","mapping_epoch":32,"log_start":"49'10","ondisk_log_start":"49'10","created":32,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'10","last_scrub_stamp":"2024-04-26T03:39:09.248660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:09.248660+0000","objects_scrubbed":1,"log_size":0,"log_dups_size":10,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:38.596496+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.00066948500000000005,"stat_sum":{"num_bytes":131072,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":385,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,13,6,0,14,10,11],"acting":[4,13,6,0,14,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":127,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.811971+0000","last_change":"2024-04-26T03:38:38.187714+0000","last_active":"2024-04-26T03:39:41.811971+0000","last_peered":"2024-04-26T03:39:41.811971+0000","last_clean":"2024-04-26T03:39:41.811971+0000","last_became_active":"2024-04-26T03:38:38.187383+0000","last_became_peered":"2024-04-26T03:38:38.187383+0000","last_unstale":"2024-04-26T03:39:41.811971+0000","last_undegraded":"2024-04-26T03:39:41.811971+0000","last_fullsized":"2024-04-26T03:39:41.811971+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:59.062623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4],"acting":[9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.4","version":"85'70","reported_seq":319,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.822867+0000","last_change":"2024-04-26T03:39:39.701091+0000","last_active":"2024-04-26T03:39:41.822867+0000","last_peered":"2024-04-26T03:39:41.822867+0000","last_clean":"2024-04-26T03:39:41.822867+0000","last_became_active":"2024-04-26T03:39:35.700637+0000","last_became_peered":"2024-04-26T03:39:35.700637+0000","last_unstale":"2024-04-26T03:39:41.822867+0000","last_undegraded":"2024-04-26T03:39:41.822867+0000","last_fullsized":"2024-04-26T03:39:41.822867+0000","mapping_epoch":32,"log_start":"75'58","ondisk_log_start":"75'58","created":32,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"42'18","last_scrub_stamp":"2024-04-26T03:39:07.342868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:07.342868+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":60,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:09.889951+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0095506840000000003,"stat_sum":{"num_bytes":15155200,"num_objects":9,"num_object_clones":4,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":21,"num_read_kb":16669,"num_write":36,"num_write_kb":9840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,13,4,2,6,10,5],"acting":[8,13,4,2,6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":129,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T03:39:42.815417+0000","last_change":"2024-04-26T03:38:37.255389+0000","last_active":"2024-04-26T03:39:42.815417+0000","last_peered":"2024-04-26T03:39:42.815417+0000","last_clean":"2024-04-26T03:39:42.815417+0000","last_became_active":"2024-04-26T03:38:37.255066+0000","last_became_peered":"2024-04-26T03:38:37.255066+0000","last_unstale":"2024-04-26T03:39:42.815417+0000","last_undegraded":"2024-04-26T03:39:42.815417+0000","last_fullsized":"2024-04-26T03:39:42.815417+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:43.860480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.5","version":"76'69","reported_seq":364,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.811633+0000","last_change":"2024-04-26T03:39:39.695213+0000","last_active":"2024-04-26T03:39:41.811633+0000","last_peered":"2024-04-26T03:39:41.811633+0000","last_clean":"2024-04-26T03:39:41.811633+0000","last_became_active":"2024-04-26T03:39:31.701065+0000","last_became_peered":"2024-04-26T03:39:31.701065+0000","last_unstale":"2024-04-26T03:39:41.811633+0000","last_undegraded":"2024-04-26T03:39:41.811633+0000","last_fullsized":"2024-04-26T03:39:41.811633+0000","mapping_epoch":32,"log_start":"69'65","ondisk_log_start":"69'65","created":32,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'51","last_scrub_stamp":"2024-04-26T03:39:06.149763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:06.149763+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":75,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:26.604335+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00052917899999999996,"stat_sum":{"num_bytes":5275648,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":27990,"num_write":25,"num_write_kb":4304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,8,4,2,7,0,5],"acting":[15,8,4,2,7,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.3","version":"85'95","reported_seq":337,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.808329+0000","last_change":"2024-04-26T03:39:39.719129+0000","last_active":"2024-04-26T03:39:41.808329+0000","last_peered":"2024-04-26T03:39:41.808329+0000","last_clean":"2024-04-26T03:39:41.808329+0000","last_became_active":"2024-04-26T03:39:39.698101+0000","last_became_peered":"2024-04-26T03:39:39.698101+0000","last_unstale":"2024-04-26T03:39:41.808329+0000","last_undegraded":"2024-04-26T03:39:41.808329+0000","last_fullsized":"2024-04-26T03:39:41.808329+0000","mapping_epoch":32,"log_start":"80'84","ondisk_log_start":"80'84","created":32,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'32","last_scrub_stamp":"2024-04-26T03:39:07.451867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:07.451867+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":120,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:35.860970+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.015853349999999999,"stat_sum":{"num_bytes":18055168,"num_objects":11,"num_object_clones":6,"num_object_copies":77,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":1,"num_read":17,"num_read_kb":15702,"num_write":43,"num_write_kb":7488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":25575424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,7,4,15,1,2,3],"acting":[10,7,4,15,1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"2.2","version":"0'0","reported_seq":116,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T03:39:40.813708+0000","last_change":"2024-04-26T03:38:48.475569+0000","last_active":"2024-04-26T03:39:40.813708+0000","last_peered":"2024-04-26T03:39:40.813708+0000","last_clean":"2024-04-26T03:39:40.813708+0000","last_became_active":"2024-04-26T03:38:48.475155+0000","last_became_peered":"2024-04-26T03:38:48.475155+0000","last_unstale":"2024-04-26T03:39:40.813708+0000","last_undegraded":"2024-04-26T03:39:40.813708+0000","last_fullsized":"2024-04-26T03:39:40.813708+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,12],"acting":[11,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.0","version":"35'9","reported_seq":280,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T03:39:43.075708+0000","last_change":"2024-04-26T03:39:39.695741+0000","last_active":"2024-04-26T03:39:43.075708+0000","last_peered":"2024-04-26T03:39:43.075708+0000","last_clean":"2024-04-26T03:39:43.075708+0000","last_became_active":"2024-04-26T03:39:01.540712+0000","last_became_peered":"2024-04-26T03:39:01.540712+0000","last_unstale":"2024-04-26T03:39:43.075708+0000","last_undegraded":"2024-04-26T03:39:43.075708+0000","last_fullsized":"2024-04-26T03:39:43.075708+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'9","last_scrub_stamp":"2024-04-26T03:39:06.119088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:06.119088+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:14.749151+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0010303230000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,9,15,2,6,14,4],"acting":[1,9,15,2,6,14,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":128,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.809092+0000","last_change":"2024-04-26T03:38:37.255450+0000","last_active":"2024-04-26T03:39:41.809092+0000","last_peered":"2024-04-26T03:39:41.809092+0000","last_clean":"2024-04-26T03:39:41.809092+0000","last_became_active":"2024-04-26T03:38:37.255181+0000","last_became_peered":"2024-04-26T03:38:37.255181+0000","last_unstale":"2024-04-26T03:39:41.809092+0000","last_undegraded":"2024-04-26T03:39:41.809092+0000","last_fullsized":"2024-04-26T03:39:41.809092+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:00.449388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0],"acting":[14,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.1","version":"42'8","reported_seq":276,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T03:39:42.815359+0000","last_change":"2024-04-26T03:39:39.696946+0000","last_active":"2024-04-26T03:39:42.815359+0000","last_peered":"2024-04-26T03:39:42.815359+0000","last_clean":"2024-04-26T03:39:42.815359+0000","last_became_active":"2024-04-26T03:39:01.547451+0000","last_became_peered":"2024-04-26T03:39:01.547451+0000","last_unstale":"2024-04-26T03:39:42.815359+0000","last_undegraded":"2024-04-26T03:39:42.815359+0000","last_fullsized":"2024-04-26T03:39:42.815359+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"42'8","last_scrub_stamp":"2024-04-26T03:39:08.263903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:08.263903+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:14.658080+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0010901210000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,14,10,9,1,2],"acting":[4,11,14,10,9,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":121,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-26T03:39:38.702286+0000","last_change":"2024-04-26T03:38:36.431325+0000","last_active":"2024-04-26T03:39:38.702286+0000","last_peered":"2024-04-26T03:39:38.702286+0000","last_clean":"2024-04-26T03:39:38.702286+0000","last_became_active":"2024-04-26T03:38:36.431121+0000","last_became_peered":"2024-04-26T03:38:36.431121+0000","last_unstale":"2024-04-26T03:39:38.702286+0000","last_undegraded":"2024-04-26T03:39:38.702286+0000","last_fullsized":"2024-04-26T03:39:38.702286+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:53.294619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"81'65","reported_seq":350,"reported_epoch":86,"state":"clean+premerge+peered","last_fresh":"2024-04-26T03:39:41.816056+0000","last_change":"2024-04-26T03:39:41.816056+0000","last_active":"2024-04-26T03:39:40.809478+0000","last_peered":"2024-04-26T03:39:41.816056+0000","last_clean":"2024-04-26T03:39:41.816056+0000","last_became_active":"2024-04-26T03:39:01.543967+0000","last_became_peered":"2024-04-26T03:39:41.813826+0000","last_unstale":"2024-04-26T03:39:41.816056+0000","last_undegraded":"2024-04-26T03:39:41.816056+0000","last_fullsized":"2024-04-26T03:39:41.816056+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'31","last_scrub_stamp":"2024-04-26T03:39:07.479325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:07.479325+0000","objects_scrubbed":3,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:27.011441+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0027817699999999998,"stat_sum":{"num_bytes":3588096,"num_objects":5,"num_object_clones":2,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":8,"num_read_kb":4259,"num_write":51,"num_write_kb":3392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,5,3,9,15,10,11],"acting":[14,5,3,9,15,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"2.3","version":"0'0","reported_seq":128,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.809145+0000","last_change":"2024-04-26T03:38:36.431667+0000","last_active":"2024-04-26T03:39:41.809145+0000","last_peered":"2024-04-26T03:39:41.809145+0000","last_clean":"2024-04-26T03:39:41.809145+0000","last_became_active":"2024-04-26T03:38:36.431449+0000","last_became_peered":"2024-04-26T03:38:36.431449+0000","last_unstale":"2024-04-26T03:39:41.809145+0000","last_undegraded":"2024-04-26T03:39:41.809145+0000","last_fullsized":"2024-04-26T03:39:41.809145+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:47.562648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"27'157","reported_seq":337,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:42.110389+0000","last_change":"2024-04-26T03:39:42.110436+0000","last_active":"2024-04-26T03:39:42.110389+0000","last_peered":"2024-04-26T03:39:42.110389+0000","last_clean":"2024-04-26T03:39:42.110389+0000","last_became_active":"2024-04-26T03:38:34.389099+0000","last_became_peered":"2024-04-26T03:38:34.389099+0000","last_unstale":"2024-04-26T03:39:42.110389+0000","last_undegraded":"2024-04-26T03:39:42.110389+0000","last_fullsized":"2024-04-26T03:39:42.110389+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'157","last_scrub_stamp":"2024-04-26T03:39:42.110300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:32.306810+0000","last_clean_scrub_stamp":"2024-04-26T03:39:42.110300+0000","objects_scrubbed":2,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:11.925418+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.8","version":"84'113","reported_seq":336,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.808247+0000","last_change":"2024-04-26T03:39:39.711675+0000","last_active":"2024-04-26T03:39:41.808247+0000","last_peered":"2024-04-26T03:39:41.808247+0000","last_clean":"2024-04-26T03:39:41.808247+0000","last_became_active":"2024-04-26T03:39:19.697021+0000","last_became_peered":"2024-04-26T03:39:19.697021+0000","last_unstale":"2024-04-26T03:39:41.808247+0000","last_undegraded":"2024-04-26T03:39:41.808247+0000","last_fullsized":"2024-04-26T03:39:41.808247+0000","mapping_epoch":32,"log_start":"60'76","ondisk_log_start":"60'76","created":32,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"42'12","last_scrub_stamp":"2024-04-26T03:39:08.136579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:08.136579+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":92,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:28.372198+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.012835276,"stat_sum":{"num_bytes":16859136,"num_objects":6,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":20,"num_read_kb":30343,"num_write":25,"num_write_kb":12208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,10,11,15,5],"acting":[6,0,1,10,11,15,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.9","version":"84'39","reported_seq":319,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.896140+0000","last_change":"2024-04-26T03:39:39.708837+0000","last_active":"2024-04-26T03:39:41.896140+0000","last_peered":"2024-04-26T03:39:41.896140+0000","last_clean":"2024-04-26T03:39:41.896140+0000","last_became_active":"2024-04-26T03:39:15.696734+0000","last_became_peered":"2024-04-26T03:39:15.696734+0000","last_unstale":"2024-04-26T03:39:41.896140+0000","last_undegraded":"2024-04-26T03:39:41.896140+0000","last_fullsized":"2024-04-26T03:39:41.896140+0000","mapping_epoch":32,"log_start":"52'24","ondisk_log_start":"52'24","created":32,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'6","last_scrub_stamp":"2024-04-26T03:39:07.345704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:07.345704+0000","objects_scrubbed":1,"log_size":15,"log_dups_size":29,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:30.212210+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.011853795,"stat_sum":{"num_bytes":4292608,"num_objects":5,"num_object_clones":2,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":2,"num_read":8,"num_read_kb":1587,"num_write":11,"num_write_kb":3248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2,10,9,15,8,4],"acting":[13,2,10,9,15,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.a","version":"83'133","reported_seq":401,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-26T03:39:38.723710+0000","last_change":"2024-04-26T03:39:33.734016+0000","last_active":"2024-04-26T03:39:38.723710+0000","last_peered":"2024-04-26T03:39:38.723710+0000","last_clean":"2024-04-26T03:39:38.723710+0000","last_became_active":"2024-04-26T03:38:49.486365+0000","last_became_peered":"2024-04-26T03:38:49.486365+0000","last_unstale":"2024-04-26T03:39:38.723710+0000","last_undegraded":"2024-04-26T03:39:38.723710+0000","last_fullsized":"2024-04-26T03:39:38.723710+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:01.744719+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.026391288999999998,"stat_sum":{"num_bytes":18563072,"num_objects":14,"num_object_clones":8,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":71,"num_read_kb":40205,"num_write":99,"num_write_kb":19904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,10,13,1,11],"acting":[3,7,6,10,13,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"10","length":"1"},{"start":"13","length":"1"},{"start":"16","length":"1"}]},{"pgid":"3.b","version":"85'130","reported_seq":392,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T03:39:42.815405+0000","last_change":"2024-04-26T03:39:39.714373+0000","last_active":"2024-04-26T03:39:42.815405+0000","last_peered":"2024-04-26T03:39:42.815405+0000","last_clean":"2024-04-26T03:39:42.815405+0000","last_became_active":"2024-04-26T03:38:49.940561+0000","last_became_peered":"2024-04-26T03:38:49.940561+0000","last_unstale":"2024-04-26T03:39:42.815405+0000","last_undegraded":"2024-04-26T03:39:42.815405+0000","last_fullsized":"2024-04-26T03:39:42.815405+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:57.848986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018406249,"stat_sum":{"num_bytes":14614528,"num_objects":10,"num_object_clones":6,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":40,"num_read_kb":12078,"num_write":163,"num_write_kb":20016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,10,12,1,7,14],"acting":[4,0,10,12,1,7,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.c","version":"84'64","reported_seq":313,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T03:39:40.812948+0000","last_change":"2024-04-26T03:39:39.710711+0000","last_active":"2024-04-26T03:39:40.812948+0000","last_peered":"2024-04-26T03:39:40.812948+0000","last_clean":"2024-04-26T03:39:40.812948+0000","last_became_active":"2024-04-26T03:38:49.941017+0000","last_became_peered":"2024-04-26T03:38:49.941017+0000","last_unstale":"2024-04-26T03:39:40.812948+0000","last_undegraded":"2024-04-26T03:39:40.812948+0000","last_fullsized":"2024-04-26T03:39:40.812948+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:18.122180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011208693,"stat_sum":{"num_bytes":3866624,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":18,"num_read_kb":10268,"num_write":64,"num_write_kb":7792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,0,1,9,12,14],"acting":[11,8,0,1,9,12,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.d","version":"84'81","reported_seq":334,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.822238+0000","last_change":"2024-04-26T03:39:39.696226+0000","last_active":"2024-04-26T03:39:41.822238+0000","last_peered":"2024-04-26T03:39:41.822238+0000","last_clean":"2024-04-26T03:39:41.822238+0000","last_became_active":"2024-04-26T03:38:49.511783+0000","last_became_peered":"2024-04-26T03:38:49.511783+0000","last_unstale":"2024-04-26T03:39:41.822238+0000","last_undegraded":"2024-04-26T03:39:41.822238+0000","last_fullsized":"2024-04-26T03:39:41.822238+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:08.988549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063040299999999997,"stat_sum":{"num_bytes":5144576,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":27,"num_read_kb":17290,"num_write":50,"num_write_kb":8048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,9,1,5,8,4],"acting":[7,3,9,1,5,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.e","version":"78'82","reported_seq":348,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.896216+0000","last_change":"2024-04-26T03:39:39.700929+0000","last_active":"2024-04-26T03:39:41.896216+0000","last_peered":"2024-04-26T03:39:41.896216+0000","last_clean":"2024-04-26T03:39:41.896216+0000","last_became_active":"2024-04-26T03:38:49.941260+0000","last_became_peered":"2024-04-26T03:38:49.941260+0000","last_unstale":"2024-04-26T03:39:41.896216+0000","last_undegraded":"2024-04-26T03:39:41.896216+0000","last_fullsized":"2024-04-26T03:39:41.896216+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:05.038642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035573390000000001,"stat_sum":{"num_bytes":6029312,"num_objects":6,"num_object_clones":2,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":2,"num_read":43,"num_read_kb":29296,"num_write":52,"num_write_kb":8432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,14,6,11,8,9,12],"acting":[13,14,6,11,8,9,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.f","version":"84'87","reported_seq":367,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T03:39:40.813048+0000","last_change":"2024-04-26T03:39:39.700458+0000","last_active":"2024-04-26T03:39:40.813048+0000","last_peered":"2024-04-26T03:39:40.813048+0000","last_clean":"2024-04-26T03:39:40.813048+0000","last_became_active":"2024-04-26T03:39:18.756713+0000","last_became_peered":"2024-04-26T03:39:18.756713+0000","last_unstale":"2024-04-26T03:39:40.813048+0000","last_undegraded":"2024-04-26T03:39:40.813048+0000","last_fullsized":"2024-04-26T03:39:40.813048+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:54.423720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001388121,"stat_sum":{"num_bytes":9191424,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":16567,"num_write":57,"num_write_kb":10368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":6094848,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,13,5,14,9,1,0],"acting":[11,13,5,14,9,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.10","version":"84'37","reported_seq":240,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.808311+0000","last_change":"2024-04-26T03:39:39.713445+0000","last_active":"2024-04-26T03:39:41.808311+0000","last_peered":"2024-04-26T03:39:41.808311+0000","last_clean":"2024-04-26T03:39:41.808311+0000","last_became_active":"2024-04-26T03:39:04.704276+0000","last_became_peered":"2024-04-26T03:39:04.704276+0000","last_unstale":"2024-04-26T03:39:41.808311+0000","last_undegraded":"2024-04-26T03:39:41.808311+0000","last_fullsized":"2024-04-26T03:39:41.808311+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":5,"last_scrub":"35'9","last_scrub_stamp":"2024-04-26T03:39:07.110098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:07.110098+0000","objects_scrubbed":1,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:34.389540+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.015083931,"stat_sum":{"num_bytes":6995968,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":6316,"num_write":29,"num_write_kb":5840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,7,4,8,0,11],"acting":[6,9,7,4,8,0,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.11","version":"83'24","reported_seq":191,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-26T03:39:38.746559+0000","last_change":"2024-04-26T03:39:33.699528+0000","last_active":"2024-04-26T03:39:38.746559+0000","last_peered":"2024-04-26T03:39:38.746559+0000","last_clean":"2024-04-26T03:39:38.746559+0000","last_became_active":"2024-04-26T03:39:07.719054+0000","last_became_peered":"2024-04-26T03:39:07.719054+0000","last_unstale":"2024-04-26T03:39:38.746559+0000","last_undegraded":"2024-04-26T03:39:38.746559+0000","last_fullsized":"2024-04-26T03:39:38.746559+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":5,"last_scrub":"42'8","last_scrub_stamp":"2024-04-26T03:39:08.233750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:08.233750+0000","objects_scrubbed":1,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:10.924564+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0011068709999999999,"stat_sum":{"num_bytes":5292032,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":7739,"num_write":21,"num_write_kb":2608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,15,10,0,14,2],"acting":[5,11,15,10,0,14,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"10","length":"1"},{"start":"13","length":"1"},{"start":"16","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":18,"stat_sum":{"num_bytes":133054464,"num_objects":85,"num_object_clones":41,"num_object_copies":595,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":7,"num_read":336,"num_read_kb":236694,"num_write":727,"num_write_kb":123488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":31670272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":227233792,"data_stored":305340416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":808,"ondisk_log_size":808,"up":126,"acting":126,"num_store_stats":16},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":13852,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":24,"seq":103079215122,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48392,"kb_used_data":13044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94323448,"statfs":{"total":96636764160,"available":96587210752,"internally_reserved":0,"allocated":13357056,"data_stored":16924139,"data_compressed":126091,"data_compressed_allocated":274432,"data_compressed_original":995328,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,4,6,7,8,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215122,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43172,"kb_used_data":11988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94328668,"statfs":{"total":96636764160,"available":96592556032,"internally_reserved":0,"allocated":12275712,"data_stored":16200798,"data_compressed":121249,"data_compressed_allocated":245760,"data_compressed_original":491520,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":23,"seq":98784247829,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53232,"kb_used_data":17892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94318608,"statfs":{"total":96636764160,"available":96582254592,"internally_reserved":0,"allocated":18321408,"data_stored":24597943,"data_compressed":129639,"data_compressed_allocated":262144,"data_compressed_original":524288,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280529,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47232,"kb_used_data":16052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94324608,"statfs":{"total":96636764160,"available":96588398592,"internally_reserved":0,"allocated":16437248,"data_stored":20111220,"data_compressed":134461,"data_compressed_allocated":290816,"data_compressed_original":1028096,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,4,5,7,8,9,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280529,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43516,"kb_used_data":16492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94328324,"statfs":{"total":96636764160,"available":96592203776,"internally_reserved":0,"allocated":16887808,"data_stored":26463572,"data_compressed":127535,"data_compressed_allocated":258048,"data_compressed_original":516096,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,3,4,5,6,8,9,10,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":22,"seq":94489280530,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39380,"kb_used_data":12360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94332460,"statfs":{"total":96636764160,"available":96596439040,"internally_reserved":0,"allocated":12656640,"data_stored":13747742,"data_compressed":125440,"data_compressed_allocated":253952,"data_compressed_original":507904,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[2,4,5,6,7,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280528,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37528,"kb_used_data":10504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94334312,"statfs":{"total":96636764160,"available":96598335488,"internally_reserved":0,"allocated":10756096,"data_stored":12909908,"data_compressed":127535,"data_compressed_allocated":258048,"data_compressed_original":516096,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,2,4,7,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280529,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55096,"kb_used_data":23916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94316744,"statfs":{"total":96636764160,"available":96580345856,"internally_reserved":0,"allocated":24489984,"data_stored":33408542,"data_compressed":125440,"data_compressed_allocated":253952,"data_compressed_original":507904,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[1,2,3,4,7,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313234,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62708,"kb_used_data":17632,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94309132,"statfs":{"total":96636764160,"available":96572551168,"internally_reserved":0,"allocated":18055168,"data_stored":22322577,"data_compressed":123349,"data_compressed_allocated":249856,"data_compressed_original":499712,"omap_allocated":1596,"internal_metadata":46135748},"hb_peers":[0,1,2,4,5,6,7,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":12,"up_from":22,"seq":94489280528,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33880,"kb_used_data":6856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337960,"statfs":{"total":96636764160,"available":96602071040,"internally_reserved":0,"allocated":7020544,"data_stored":11242836,"data_compressed":127535,"data_compressed_allocated":258048,"data_compressed_original":516096,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,7,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280528,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41008,"kb_used_data":13984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94330832,"statfs":{"total":96636764160,"available":96594771968,"internally_reserved":0,"allocated":14319616,"data_stored":17927508,"data_compressed":127535,"data_compressed_allocated":258048,"data_compressed_original":516096,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[2,4,6,7,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247826,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68944,"kb_used_data":15548,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94302896,"statfs":{"total":96636764160,"available":96566165504,"internally_reserved":0,"allocated":15921152,"data_stored":21609812,"data_compressed":127535,"data_compressed_allocated":258048,"data_compressed_original":516096,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,6,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280528,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38308,"kb_used_data":11288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94333532,"statfs":{"total":96636764160,"available":96597536768,"internally_reserved":0,"allocated":11558912,"data_stored":17249822,"data_compressed":125440,"data_compressed_allocated":253952,"data_compressed_original":507904,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,3,5,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":23,"seq":98784247828,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74824,"kb_used_data":21436,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94297016,"statfs":{"total":96636764160,"available":96560144384,"internally_reserved":0,"allocated":21950464,"data_stored":31823287,"data_compressed":129639,"data_compressed_allocated":262144,"data_compressed_original":524288,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,4,6,7,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280528,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40492,"kb_used_data":13468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94331348,"statfs":{"total":96636764160,"available":96595300352,"internally_reserved":0,"allocated":13791232,"data_stored":15289684,"data_compressed":127535,"data_compressed_allocated":258048,"data_compressed_original":516096,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,4,5,7,8,9,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247826,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43880,"kb_used_data":12692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94327960,"statfs":{"total":96636764160,"available":96591831040,"internally_reserved":0,"allocated":12996608,"data_stored":13839700,"data_compressed":127535,"data_compressed_allocated":258048,"data_compressed_original":516096,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":14942208,"data_stored":20893696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20959232,"data_stored":31096832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12017664,"data_stored":13123584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":11333632,"data_stored":15515648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":17330176,"data_stored":23871488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":14458880,"data_stored":17805312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":15405056,"data_stored":18866176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":15908864,"data_stored":25747456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":11689984,"data_stored":13041664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":9777152,"data_stored":12193792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":23523328,"data_stored":32702464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":17100800,"data_stored":21626880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":6041600,"data_stored":10526720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":13340672,"data_stored":17211392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":10592256,"data_stored":16543744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":12812288,"data_stored":14573568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T03:39:44.197 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T03:39:44.242 INFO:tasks.ceph.osd.12.smithi028.stderr:2024-04-26T03:39:44.241+0000 7f114fe14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 31967) UID: 0 2024-04-26T03:39:44.270 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:44.270 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:44.279 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-04-26T03:39:44.305 INFO:tasks.ceph.osd.15.smithi092.stderr:2024-04-26T03:39:44.303+0000 7f6d12dec640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 31824) UID: 0 2024-04-26T03:39:44.385 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:44.385 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:44.394 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-04-26T03:39:44.438 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:39:44.436+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:39:44.494 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:39:44.494 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-04-26T03:39:44.500 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:44.500 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:44.505 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:39:44.504+0000 7f60d3c4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31757) UID: 0 2024-04-26T03:39:44.506 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2024-04-26T03:39:44.063670+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":133644832,"num_objects":87,"num_object_clones":41,"num_object_copies":599,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":7,"num_read":472,"num_read_kb":236812,"num_write":930,"num_write_kb":126948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":31670272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":965,"ondisk_log_size":965,"up":144,"acting":144,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":153,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":771592,"kb_used_data":235152,"kb_used_omap":24,"kb_used_meta":536167,"kb_avail":1509177848,"statfs":{"total":1546188226560,"available":1545398116352,"internally_reserved":0,"allocated":240795648,"data_stored":315669090,"data_compressed":2033493,"data_compressed_allocated":4153344,"data_compressed_original":9199616,"omap_allocated":25522,"internal_metadata":549035086},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27607040,"num_objects":18,"num_object_clones":13,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":1,"num_read":41,"num_read_kb":39908,"num_write":77,"num_write_kb":16768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":25575424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000797"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":128,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.811530+0000","last_change":"2024-04-26T03:38:37.359252+0000","last_active":"2024-04-26T03:39:41.811530+0000","last_peered":"2024-04-26T03:39:41.811530+0000","last_clean":"2024-04-26T03:39:41.811530+0000","last_became_active":"2024-04-26T03:38:37.358910+0000","last_became_peered":"2024-04-26T03:38:37.358910+0000","last_unstale":"2024-04-26T03:39:41.811530+0000","last_undegraded":"2024-04-26T03:39:41.811530+0000","last_fullsized":"2024-04-26T03:39:41.811530+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:04.186001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"3.6","version":"81'20","reported_seq":304,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T03:39:43.074998+0000","last_change":"2024-04-26T03:39:39.705400+0000","last_active":"2024-04-26T03:39:43.074998+0000","last_peered":"2024-04-26T03:39:43.074998+0000","last_clean":"2024-04-26T03:39:43.074998+0000","last_became_active":"2024-04-26T03:39:27.699531+0000","last_became_peered":"2024-04-26T03:39:27.699531+0000","last_unstale":"2024-04-26T03:39:43.074998+0000","last_undegraded":"2024-04-26T03:39:43.074998+0000","last_fullsized":"2024-04-26T03:39:43.074998+0000","mapping_epoch":32,"log_start":"64'11","ondisk_log_start":"64'11","created":32,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'9","last_scrub_stamp":"2024-04-26T03:39:07.177824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:07.177824+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":11,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:24.874885+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0016197290000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,14,6,2,12,7,1],"acting":[0,14,6,2,12,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":128,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.822761+0000","last_change":"2024-04-26T03:38:36.484782+0000","last_active":"2024-04-26T03:39:41.822761+0000","last_peered":"2024-04-26T03:39:41.822761+0000","last_clean":"2024-04-26T03:39:41.822761+0000","last_became_active":"2024-04-26T03:38:36.484388+0000","last_became_peered":"2024-04-26T03:38:36.484388+0000","last_unstale":"2024-04-26T03:39:41.822761+0000","last_undegraded":"2024-04-26T03:39:41.822761+0000","last_fullsized":"2024-04-26T03:39:41.822761+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,13],"acting":[8,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.7","version":"49'10","reported_seq":299,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T03:39:42.815292+0000","last_change":"2024-04-26T03:39:39.696366+0000","last_active":"2024-04-26T03:39:42.815292+0000","last_peered":"2024-04-26T03:39:42.815292+0000","last_clean":"2024-04-26T03:39:42.815292+0000","last_became_active":"2024-04-26T03:39:23.698262+0000","last_became_peered":"2024-04-26T03:39:23.698262+0000","last_unstale":"2024-04-26T03:39:42.815292+0000","last_undegraded":"2024-04-26T03:39:42.815292+0000","last_fullsized":"2024-04-26T03:39:42.815292+0000","mapping_epoch":32,"log_start":"49'10","ondisk_log_start":"49'10","created":32,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'10","last_scrub_stamp":"2024-04-26T03:39:09.248660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:09.248660+0000","objects_scrubbed":1,"log_size":0,"log_dups_size":10,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:38.596496+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.00066948500000000005,"stat_sum":{"num_bytes":131072,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":385,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,13,6,0,14,10,11],"acting":[4,13,6,0,14,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":127,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.811971+0000","last_change":"2024-04-26T03:38:38.187714+0000","last_active":"2024-04-26T03:39:41.811971+0000","last_peered":"2024-04-26T03:39:41.811971+0000","last_clean":"2024-04-26T03:39:41.811971+0000","last_became_active":"2024-04-26T03:38:38.187383+0000","last_became_peered":"2024-04-26T03:38:38.187383+0000","last_unstale":"2024-04-26T03:39:41.811971+0000","last_undegraded":"2024-04-26T03:39:41.811971+0000","last_fullsized":"2024-04-26T03:39:41.811971+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:59.062623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4],"acting":[9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.4","version":"85'70","reported_seq":319,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.822867+0000","last_change":"2024-04-26T03:39:39.701091+0000","last_active":"2024-04-26T03:39:41.822867+0000","last_peered":"2024-04-26T03:39:41.822867+0000","last_clean":"2024-04-26T03:39:41.822867+0000","last_became_active":"2024-04-26T03:39:35.700637+0000","last_became_peered":"2024-04-26T03:39:35.700637+0000","last_unstale":"2024-04-26T03:39:41.822867+0000","last_undegraded":"2024-04-26T03:39:41.822867+0000","last_fullsized":"2024-04-26T03:39:41.822867+0000","mapping_epoch":32,"log_start":"75'58","ondisk_log_start":"75'58","created":32,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"42'18","last_scrub_stamp":"2024-04-26T03:39:07.342868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:07.342868+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":60,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:09.889951+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0095506840000000003,"stat_sum":{"num_bytes":15155200,"num_objects":9,"num_object_clones":4,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":21,"num_read_kb":16669,"num_write":36,"num_write_kb":9840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,13,4,2,6,10,5],"acting":[8,13,4,2,6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":129,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T03:39:42.815417+0000","last_change":"2024-04-26T03:38:37.255389+0000","last_active":"2024-04-26T03:39:42.815417+0000","last_peered":"2024-04-26T03:39:42.815417+0000","last_clean":"2024-04-26T03:39:42.815417+0000","last_became_active":"2024-04-26T03:38:37.255066+0000","last_became_peered":"2024-04-26T03:38:37.255066+0000","last_unstale":"2024-04-26T03:39:42.815417+0000","last_undegraded":"2024-04-26T03:39:42.815417+0000","last_fullsized":"2024-04-26T03:39:42.815417+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:43.860480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.5","version":"76'69","reported_seq":364,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.811633+0000","last_change":"2024-04-26T03:39:39.695213+0000","last_active":"2024-04-26T03:39:41.811633+0000","last_peered":"2024-04-26T03:39:41.811633+0000","last_clean":"2024-04-26T03:39:41.811633+0000","last_became_active":"2024-04-26T03:39:31.701065+0000","last_became_peered":"2024-04-26T03:39:31.701065+0000","last_unstale":"2024-04-26T03:39:41.811633+0000","last_undegraded":"2024-04-26T03:39:41.811633+0000","last_fullsized":"2024-04-26T03:39:41.811633+0000","mapping_epoch":32,"log_start":"69'65","ondisk_log_start":"69'65","created":32,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'51","last_scrub_stamp":"2024-04-26T03:39:06.149763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:06.149763+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":75,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:26.604335+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00052917899999999996,"stat_sum":{"num_bytes":5275648,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":27990,"num_write":25,"num_write_kb":4304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,8,4,2,7,0,5],"acting":[15,8,4,2,7,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.3","version":"85'95","reported_seq":337,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.808329+0000","last_change":"2024-04-26T03:39:39.719129+0000","last_active":"2024-04-26T03:39:41.808329+0000","last_peered":"2024-04-26T03:39:41.808329+0000","last_clean":"2024-04-26T03:39:41.808329+0000","last_became_active":"2024-04-26T03:39:39.698101+0000","last_became_peered":"2024-04-26T03:39:39.698101+0000","last_unstale":"2024-04-26T03:39:41.808329+0000","last_undegraded":"2024-04-26T03:39:41.808329+0000","last_fullsized":"2024-04-26T03:39:41.808329+0000","mapping_epoch":32,"log_start":"80'84","ondisk_log_start":"80'84","created":32,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'32","last_scrub_stamp":"2024-04-26T03:39:07.451867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:07.451867+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":120,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:35.860970+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.015853349999999999,"stat_sum":{"num_bytes":18055168,"num_objects":11,"num_object_clones":6,"num_object_copies":77,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":1,"num_read":17,"num_read_kb":15702,"num_write":43,"num_write_kb":7488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":25575424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,7,4,15,1,2,3],"acting":[10,7,4,15,1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"2.2","version":"0'0","reported_seq":116,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T03:39:40.813708+0000","last_change":"2024-04-26T03:38:48.475569+0000","last_active":"2024-04-26T03:39:40.813708+0000","last_peered":"2024-04-26T03:39:40.813708+0000","last_clean":"2024-04-26T03:39:40.813708+0000","last_became_active":"2024-04-26T03:38:48.475155+0000","last_became_peered":"2024-04-26T03:38:48.475155+0000","last_unstale":"2024-04-26T03:39:40.813708+0000","last_undegraded":"2024-04-26T03:39:40.813708+0000","last_fullsized":"2024-04-26T03:39:40.813708+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,12],"acting":[11,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.0","version":"35'9","reported_seq":280,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T03:39:43.075708+0000","last_change":"2024-04-26T03:39:39.695741+0000","last_active":"2024-04-26T03:39:43.075708+0000","last_peered":"2024-04-26T03:39:43.075708+0000","last_clean":"2024-04-26T03:39:43.075708+0000","last_became_active":"2024-04-26T03:39:01.540712+0000","last_became_peered":"2024-04-26T03:39:01.540712+0000","last_unstale":"2024-04-26T03:39:43.075708+0000","last_undegraded":"2024-04-26T03:39:43.075708+0000","last_fullsized":"2024-04-26T03:39:43.075708+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'9","last_scrub_stamp":"2024-04-26T03:39:06.119088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:06.119088+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:14.749151+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0010303230000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,9,15,2,6,14,4],"acting":[1,9,15,2,6,14,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":128,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.809092+0000","last_change":"2024-04-26T03:38:37.255450+0000","last_active":"2024-04-26T03:39:41.809092+0000","last_peered":"2024-04-26T03:39:41.809092+0000","last_clean":"2024-04-26T03:39:41.809092+0000","last_became_active":"2024-04-26T03:38:37.255181+0000","last_became_peered":"2024-04-26T03:38:37.255181+0000","last_unstale":"2024-04-26T03:39:41.809092+0000","last_undegraded":"2024-04-26T03:39:41.809092+0000","last_fullsized":"2024-04-26T03:39:41.809092+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:00.449388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0],"acting":[14,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.1","version":"42'8","reported_seq":276,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T03:39:42.815359+0000","last_change":"2024-04-26T03:39:39.696946+0000","last_active":"2024-04-26T03:39:42.815359+0000","last_peered":"2024-04-26T03:39:42.815359+0000","last_clean":"2024-04-26T03:39:42.815359+0000","last_became_active":"2024-04-26T03:39:01.547451+0000","last_became_peered":"2024-04-26T03:39:01.547451+0000","last_unstale":"2024-04-26T03:39:42.815359+0000","last_undegraded":"2024-04-26T03:39:42.815359+0000","last_fullsized":"2024-04-26T03:39:42.815359+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"42'8","last_scrub_stamp":"2024-04-26T03:39:08.263903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:08.263903+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:14.658080+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0010901210000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,14,10,9,1,2],"acting":[4,11,14,10,9,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":121,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-26T03:39:38.702286+0000","last_change":"2024-04-26T03:38:36.431325+0000","last_active":"2024-04-26T03:39:38.702286+0000","last_peered":"2024-04-26T03:39:38.702286+0000","last_clean":"2024-04-26T03:39:38.702286+0000","last_became_active":"2024-04-26T03:38:36.431121+0000","last_became_peered":"2024-04-26T03:38:36.431121+0000","last_unstale":"2024-04-26T03:39:38.702286+0000","last_undegraded":"2024-04-26T03:39:38.702286+0000","last_fullsized":"2024-04-26T03:39:38.702286+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:53.294619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"81'65","reported_seq":350,"reported_epoch":86,"state":"clean+premerge+peered","last_fresh":"2024-04-26T03:39:41.816056+0000","last_change":"2024-04-26T03:39:41.816056+0000","last_active":"2024-04-26T03:39:40.809478+0000","last_peered":"2024-04-26T03:39:41.816056+0000","last_clean":"2024-04-26T03:39:41.816056+0000","last_became_active":"2024-04-26T03:39:01.543967+0000","last_became_peered":"2024-04-26T03:39:41.813826+0000","last_unstale":"2024-04-26T03:39:41.816056+0000","last_undegraded":"2024-04-26T03:39:41.816056+0000","last_fullsized":"2024-04-26T03:39:41.816056+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'31","last_scrub_stamp":"2024-04-26T03:39:07.479325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:07.479325+0000","objects_scrubbed":3,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:27.011441+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0027817699999999998,"stat_sum":{"num_bytes":3588096,"num_objects":5,"num_object_clones":2,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":8,"num_read_kb":4259,"num_write":51,"num_write_kb":3392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,5,3,9,15,10,11],"acting":[14,5,3,9,15,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"2.3","version":"0'0","reported_seq":128,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.809145+0000","last_change":"2024-04-26T03:38:36.431667+0000","last_active":"2024-04-26T03:39:41.809145+0000","last_peered":"2024-04-26T03:39:41.809145+0000","last_clean":"2024-04-26T03:39:41.809145+0000","last_became_active":"2024-04-26T03:38:36.431449+0000","last_became_peered":"2024-04-26T03:38:36.431449+0000","last_unstale":"2024-04-26T03:39:41.809145+0000","last_undegraded":"2024-04-26T03:39:41.809145+0000","last_fullsized":"2024-04-26T03:39:41.809145+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:47.562648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"27'157","reported_seq":337,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:42.110389+0000","last_change":"2024-04-26T03:39:42.110436+0000","last_active":"2024-04-26T03:39:42.110389+0000","last_peered":"2024-04-26T03:39:42.110389+0000","last_clean":"2024-04-26T03:39:42.110389+0000","last_became_active":"2024-04-26T03:38:34.389099+0000","last_became_peered":"2024-04-26T03:38:34.389099+0000","last_unstale":"2024-04-26T03:39:42.110389+0000","last_undegraded":"2024-04-26T03:39:42.110389+0000","last_fullsized":"2024-04-26T03:39:42.110389+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'157","last_scrub_stamp":"2024-04-26T03:39:42.110300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:32.306810+0000","last_clean_scrub_stamp":"2024-04-26T03:39:42.110300+0000","objects_scrubbed":2,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:11.925418+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.8","version":"84'113","reported_seq":336,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.808247+0000","last_change":"2024-04-26T03:39:39.711675+0000","last_active":"2024-04-26T03:39:41.808247+0000","last_peered":"2024-04-26T03:39:41.808247+0000","last_clean":"2024-04-26T03:39:41.808247+0000","last_became_active":"2024-04-26T03:39:19.697021+0000","last_became_peered":"2024-04-26T03:39:19.697021+0000","last_unstale":"2024-04-26T03:39:41.808247+0000","last_undegraded":"2024-04-26T03:39:41.808247+0000","last_fullsized":"2024-04-26T03:39:41.808247+0000","mapping_epoch":32,"log_start":"60'76","ondisk_log_start":"60'76","created":32,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"42'12","last_scrub_stamp":"2024-04-26T03:39:08.136579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:08.136579+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":92,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:28.372198+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.012835276,"stat_sum":{"num_bytes":16859136,"num_objects":6,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":20,"num_read_kb":30343,"num_write":25,"num_write_kb":12208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,10,11,15,5],"acting":[6,0,1,10,11,15,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.9","version":"84'39","reported_seq":319,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.896140+0000","last_change":"2024-04-26T03:39:39.708837+0000","last_active":"2024-04-26T03:39:41.896140+0000","last_peered":"2024-04-26T03:39:41.896140+0000","last_clean":"2024-04-26T03:39:41.896140+0000","last_became_active":"2024-04-26T03:39:15.696734+0000","last_became_peered":"2024-04-26T03:39:15.696734+0000","last_unstale":"2024-04-26T03:39:41.896140+0000","last_undegraded":"2024-04-26T03:39:41.896140+0000","last_fullsized":"2024-04-26T03:39:41.896140+0000","mapping_epoch":32,"log_start":"52'24","ondisk_log_start":"52'24","created":32,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'6","last_scrub_stamp":"2024-04-26T03:39:07.345704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:07.345704+0000","objects_scrubbed":1,"log_size":15,"log_dups_size":29,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:30.212210+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.011853795,"stat_sum":{"num_bytes":4292608,"num_objects":5,"num_object_clones":2,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":2,"num_read":8,"num_read_kb":1587,"num_write":11,"num_write_kb":3248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2,10,9,15,8,4],"acting":[13,2,10,9,15,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.a","version":"83'133","reported_seq":401,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-26T03:39:38.723710+0000","last_change":"2024-04-26T03:39:33.734016+0000","last_active":"2024-04-26T03:39:38.723710+0000","last_peered":"2024-04-26T03:39:38.723710+0000","last_clean":"2024-04-26T03:39:38.723710+0000","last_became_active":"2024-04-26T03:38:49.486365+0000","last_became_peered":"2024-04-26T03:38:49.486365+0000","last_unstale":"2024-04-26T03:39:38.723710+0000","last_undegraded":"2024-04-26T03:39:38.723710+0000","last_fullsized":"2024-04-26T03:39:38.723710+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:01.744719+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.026391288999999998,"stat_sum":{"num_bytes":18563072,"num_objects":14,"num_object_clones":8,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":71,"num_read_kb":40205,"num_write":99,"num_write_kb":19904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,10,13,1,11],"acting":[3,7,6,10,13,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"10","length":"1"},{"start":"13","length":"1"},{"start":"16","length":"1"}]},{"pgid":"3.b","version":"85'130","reported_seq":392,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T03:39:42.815405+0000","last_change":"2024-04-26T03:39:39.714373+0000","last_active":"2024-04-26T03:39:42.815405+0000","last_peered":"2024-04-26T03:39:42.815405+0000","last_clean":"2024-04-26T03:39:42.815405+0000","last_became_active":"2024-04-26T03:38:49.940561+0000","last_became_peered":"2024-04-26T03:38:49.940561+0000","last_unstale":"2024-04-26T03:39:42.815405+0000","last_undegraded":"2024-04-26T03:39:42.815405+0000","last_fullsized":"2024-04-26T03:39:42.815405+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:57.848986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018406249,"stat_sum":{"num_bytes":14614528,"num_objects":10,"num_object_clones":6,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":40,"num_read_kb":12078,"num_write":163,"num_write_kb":20016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,10,12,1,7,14],"acting":[4,0,10,12,1,7,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.c","version":"84'64","reported_seq":313,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T03:39:40.812948+0000","last_change":"2024-04-26T03:39:39.710711+0000","last_active":"2024-04-26T03:39:40.812948+0000","last_peered":"2024-04-26T03:39:40.812948+0000","last_clean":"2024-04-26T03:39:40.812948+0000","last_became_active":"2024-04-26T03:38:49.941017+0000","last_became_peered":"2024-04-26T03:38:49.941017+0000","last_unstale":"2024-04-26T03:39:40.812948+0000","last_undegraded":"2024-04-26T03:39:40.812948+0000","last_fullsized":"2024-04-26T03:39:40.812948+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:18.122180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011208693,"stat_sum":{"num_bytes":3866624,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":18,"num_read_kb":10268,"num_write":64,"num_write_kb":7792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,0,1,9,12,14],"acting":[11,8,0,1,9,12,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.d","version":"84'81","reported_seq":334,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.822238+0000","last_change":"2024-04-26T03:39:39.696226+0000","last_active":"2024-04-26T03:39:41.822238+0000","last_peered":"2024-04-26T03:39:41.822238+0000","last_clean":"2024-04-26T03:39:41.822238+0000","last_became_active":"2024-04-26T03:38:49.511783+0000","last_became_peered":"2024-04-26T03:38:49.511783+0000","last_unstale":"2024-04-26T03:39:41.822238+0000","last_undegraded":"2024-04-26T03:39:41.822238+0000","last_fullsized":"2024-04-26T03:39:41.822238+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:08.988549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063040299999999997,"stat_sum":{"num_bytes":5144576,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":27,"num_read_kb":17290,"num_write":50,"num_write_kb":8048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,9,1,5,8,4],"acting":[7,3,9,1,5,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.e","version":"78'82","reported_seq":348,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.896216+0000","last_change":"2024-04-26T03:39:39.700929+0000","last_active":"2024-04-26T03:39:41.896216+0000","last_peered":"2024-04-26T03:39:41.896216+0000","last_clean":"2024-04-26T03:39:41.896216+0000","last_became_active":"2024-04-26T03:38:49.941260+0000","last_became_peered":"2024-04-26T03:38:49.941260+0000","last_unstale":"2024-04-26T03:39:41.896216+0000","last_undegraded":"2024-04-26T03:39:41.896216+0000","last_fullsized":"2024-04-26T03:39:41.896216+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:05.038642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035573390000000001,"stat_sum":{"num_bytes":6029312,"num_objects":6,"num_object_clones":2,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":2,"num_read":43,"num_read_kb":29296,"num_write":52,"num_write_kb":8432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,14,6,11,8,9,12],"acting":[13,14,6,11,8,9,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.f","version":"84'87","reported_seq":367,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T03:39:40.813048+0000","last_change":"2024-04-26T03:39:39.700458+0000","last_active":"2024-04-26T03:39:40.813048+0000","last_peered":"2024-04-26T03:39:40.813048+0000","last_clean":"2024-04-26T03:39:40.813048+0000","last_became_active":"2024-04-26T03:39:18.756713+0000","last_became_peered":"2024-04-26T03:39:18.756713+0000","last_unstale":"2024-04-26T03:39:40.813048+0000","last_undegraded":"2024-04-26T03:39:40.813048+0000","last_fullsized":"2024-04-26T03:39:40.813048+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:54.423720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001388121,"stat_sum":{"num_bytes":9191424,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":16567,"num_write":57,"num_write_kb":10368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":6094848,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,13,5,14,9,1,0],"acting":[11,13,5,14,9,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.10","version":"84'37","reported_seq":240,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.808311+0000","last_change":"2024-04-26T03:39:39.713445+0000","last_active":"2024-04-26T03:39:41.808311+0000","last_peered":"2024-04-26T03:39:41.808311+0000","last_clean":"2024-04-26T03:39:41.808311+0000","last_became_active":"2024-04-26T03:39:04.704276+0000","last_became_peered":"2024-04-26T03:39:04.704276+0000","last_unstale":"2024-04-26T03:39:41.808311+0000","last_undegraded":"2024-04-26T03:39:41.808311+0000","last_fullsized":"2024-04-26T03:39:41.808311+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":5,"last_scrub":"35'9","last_scrub_stamp":"2024-04-26T03:39:07.110098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:07.110098+0000","objects_scrubbed":1,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:34.389540+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.015083931,"stat_sum":{"num_bytes":6995968,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":6316,"num_write":29,"num_write_kb":5840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,7,4,8,0,11],"acting":[6,9,7,4,8,0,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.11","version":"83'24","reported_seq":191,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-26T03:39:38.746559+0000","last_change":"2024-04-26T03:39:33.699528+0000","last_active":"2024-04-26T03:39:38.746559+0000","last_peered":"2024-04-26T03:39:38.746559+0000","last_clean":"2024-04-26T03:39:38.746559+0000","last_became_active":"2024-04-26T03:39:07.719054+0000","last_became_peered":"2024-04-26T03:39:07.719054+0000","last_unstale":"2024-04-26T03:39:38.746559+0000","last_undegraded":"2024-04-26T03:39:38.746559+0000","last_fullsized":"2024-04-26T03:39:38.746559+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":5,"last_scrub":"42'8","last_scrub_stamp":"2024-04-26T03:39:08.233750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:08.233750+0000","objects_scrubbed":1,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:10.924564+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0011068709999999999,"stat_sum":{"num_bytes":5292032,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":7739,"num_write":21,"num_write_kb":2608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,15,10,0,14,2],"acting":[5,11,15,10,0,14,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"10","length":"1"},{"start":"13","length":"1"},{"start":"16","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":18,"stat_sum":{"num_bytes":133054464,"num_objects":85,"num_object_clones":41,"num_object_copies":595,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":7,"num_read":336,"num_read_kb":236694,"num_write":727,"num_write_kb":123488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":31670272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":227233792,"data_stored":305340416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":808,"ondisk_log_size":808,"up":126,"acting":126,"num_store_stats":16},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":13852,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":24,"seq":103079215122,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48392,"kb_used_data":13044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94323448,"statfs":{"total":96636764160,"available":96587210752,"internally_reserved":0,"allocated":13357056,"data_stored":16924139,"data_compressed":126091,"data_compressed_allocated":274432,"data_compressed_original":995328,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,4,6,7,8,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215122,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43172,"kb_used_data":11988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94328668,"statfs":{"total":96636764160,"available":96592556032,"internally_reserved":0,"allocated":12275712,"data_stored":16200798,"data_compressed":121249,"data_compressed_allocated":245760,"data_compressed_original":491520,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":23,"seq":98784247829,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53232,"kb_used_data":17892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94318608,"statfs":{"total":96636764160,"available":96582254592,"internally_reserved":0,"allocated":18321408,"data_stored":24597943,"data_compressed":129639,"data_compressed_allocated":262144,"data_compressed_original":524288,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280529,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47232,"kb_used_data":16052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94324608,"statfs":{"total":96636764160,"available":96588398592,"internally_reserved":0,"allocated":16437248,"data_stored":20111220,"data_compressed":134461,"data_compressed_allocated":290816,"data_compressed_original":1028096,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,4,5,7,8,9,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280529,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43516,"kb_used_data":16492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94328324,"statfs":{"total":96636764160,"available":96592203776,"internally_reserved":0,"allocated":16887808,"data_stored":26463572,"data_compressed":127535,"data_compressed_allocated":258048,"data_compressed_original":516096,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,3,4,5,6,8,9,10,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":22,"seq":94489280530,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39380,"kb_used_data":12360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94332460,"statfs":{"total":96636764160,"available":96596439040,"internally_reserved":0,"allocated":12656640,"data_stored":13747742,"data_compressed":125440,"data_compressed_allocated":253952,"data_compressed_original":507904,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[2,4,5,6,7,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280528,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37528,"kb_used_data":10504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94334312,"statfs":{"total":96636764160,"available":96598335488,"internally_reserved":0,"allocated":10756096,"data_stored":12909908,"data_compressed":127535,"data_compressed_allocated":258048,"data_compressed_original":516096,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,2,4,7,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280529,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55096,"kb_used_data":23916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94316744,"statfs":{"total":96636764160,"available":96580345856,"internally_reserved":0,"allocated":24489984,"data_stored":33408542,"data_compressed":125440,"data_compressed_allocated":253952,"data_compressed_original":507904,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[1,2,3,4,7,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313234,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62708,"kb_used_data":17632,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94309132,"statfs":{"total":96636764160,"available":96572551168,"internally_reserved":0,"allocated":18055168,"data_stored":22322577,"data_compressed":123349,"data_compressed_allocated":249856,"data_compressed_original":499712,"omap_allocated":1596,"internal_metadata":46135748},"hb_peers":[0,1,2,4,5,6,7,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":12,"up_from":22,"seq":94489280528,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33880,"kb_used_data":6856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337960,"statfs":{"total":96636764160,"available":96602071040,"internally_reserved":0,"allocated":7020544,"data_stored":11242836,"data_compressed":127535,"data_compressed_allocated":258048,"data_compressed_original":516096,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,7,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280528,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41008,"kb_used_data":13984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94330832,"statfs":{"total":96636764160,"available":96594771968,"internally_reserved":0,"allocated":14319616,"data_stored":17927508,"data_compressed":127535,"data_compressed_allocated":258048,"data_compressed_original":516096,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[2,4,6,7,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247826,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68944,"kb_used_data":15548,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94302896,"statfs":{"total":96636764160,"available":96566165504,"internally_reserved":0,"allocated":15921152,"data_stored":21609812,"data_compressed":127535,"data_compressed_allocated":258048,"data_compressed_original":516096,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,6,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280528,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38308,"kb_used_data":11288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94333532,"statfs":{"total":96636764160,"available":96597536768,"internally_reserved":0,"allocated":11558912,"data_stored":17249822,"data_compressed":125440,"data_compressed_allocated":253952,"data_compressed_original":507904,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,3,5,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":23,"seq":98784247828,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74824,"kb_used_data":21436,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94297016,"statfs":{"total":96636764160,"available":96560144384,"internally_reserved":0,"allocated":21950464,"data_stored":31823287,"data_compressed":129639,"data_compressed_allocated":262144,"data_compressed_original":524288,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,4,6,7,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280528,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40492,"kb_used_data":13468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94331348,"statfs":{"total":96636764160,"available":96595300352,"internally_reserved":0,"allocated":13791232,"data_stored":15289684,"data_compressed":127535,"data_compressed_allocated":258048,"data_compressed_original":516096,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,4,5,7,8,9,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247826,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43880,"kb_used_data":12692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94327960,"statfs":{"total":96636764160,"available":96591831040,"internally_reserved":0,"allocated":12996608,"data_stored":13839700,"data_compressed":127535,"data_compressed_allocated":258048,"data_compressed_original":516096,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":14942208,"data_stored":20893696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20959232,"data_stored":31096832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12017664,"data_stored":13123584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":11333632,"data_stored":15515648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":17330176,"data_stored":23871488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":14458880,"data_stored":17805312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":15405056,"data_stored":18866176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":15908864,"data_stored":25747456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":11689984,"data_stored":13041664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":9777152,"data_stored":12193792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":23523328,"data_stored":32702464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":17100800,"data_stored":21626880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":6041600,"data_stored":10526720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":13340672,"data_stored":17211392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":10592256,"data_stored":16543744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":12812288,"data_stored":14573568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T03:39:44.508 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T03:39:44.510 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-04-26T03:39:44.607 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:39:44.605+0000 7f60d3c4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31757) UID: 0 2024-04-26T03:39:44.656 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:44.656 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:44.665 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-26T03:39:44.691 INFO:teuthology.orchestra.run.smithi028.stderr:nodeep-scrub is unset 2024-04-26T03:39:44.694 INFO:tasks.rados.rados.0.smithi092.stdout:389: done (1 left) 2024-04-26T03:39:44.694 INFO:tasks.rados.rados.0.smithi092.stdout:390: read oid 11 snap 25 2024-04-26T03:39:44.694 INFO:tasks.rados.rados.0.smithi092.stdout:390: expect deleted 2024-04-26T03:39:44.694 INFO:tasks.rados.rados.0.smithi092.stdout:391: read oid 42 snap 27 2024-04-26T03:39:44.694 INFO:tasks.rados.rados.0.smithi092.stdout:391: expect (ObjNum 60 snap 0 seq_num 60) 2024-04-26T03:39:44.694 INFO:tasks.rados.rados.0.smithi092.stdout:392: snap_create 2024-04-26T03:39:44.694 INFO:tasks.rados.rados.0.smithi092.stdout:388: finishing rollback tid 1 to smithi09235267-9 2024-04-26T03:39:44.694 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 9 v 0 (ObjNum 33 snap 0 seq_num 419409136) dirty dne 2024-04-26T03:39:44.708 INFO:tasks.ceph.osd.13.smithi042.stderr:2024-04-26T03:39:44.706+0000 7f0a15837640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 31961) UID: 0 2024-04-26T03:39:44.793 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:44.793 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:44.802 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-26T03:39:44.808 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-04-26T03:39:44.807+0000 7fdc2d21c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31819) UID: 0 2024-04-26T03:39:44.868 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-26T03:39:44.907 INFO:tasks.rados.rados.0.smithi092.stdout:391: expect (ObjNum 60 snap 0 seq_num 60) 2024-04-26T03:39:44.909 INFO:tasks.ceph.osd.9.smithi042.stderr:2024-04-26T03:39:44.908+0000 7fdac63b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 31960) UID: 0 2024-04-26T03:39:44.910 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:44.911 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:44.920 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-26T03:39:44.934 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:39:44.934 INFO:teuthology.orchestra.run.smithi028.stdout:{"fsid":"c5a13c99-5c9a-4b8b-a893-4f6f4211677a","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":85,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":89,"num_osds":16,"num_up_osds":16,"osd_up_since":1714102713,"num_in_osds":16,"osd_in_since":1714102704,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":26},{"state_name":"clean+premerge+peered","count":1}],"num_pgs":27,"num_pools":3,"num_objects":87,"data_bytes":133644832,"bytes_used":790110208,"bytes_avail":1545398116352,"bytes_total":1546188226560,"inactive_pgs_ratio":0.037037037312984467,"recovering_objects_per_sec":3,"recovering_bytes_per_sec":6392580,"recovering_keys_per_sec":0,"num_objects_recovered":15,"num_bytes_recovered":25575424,"num_keys_recovered":0,"read_bytes_sec":10214410,"write_bytes_sec":4291751,"read_op_per_sec":10,"write_op_per_sec":19},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-04-26T03:39:36.062047+0000","services":{"osd":{"daemons":{"summary":"","4":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-26T03:39:44.948 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-04-26T03:39:44.948 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T03:39:45.010 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:39:45.009+0000 7f60d3c4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31757) UID: 0 2024-04-26T03:39:45.012 INFO:teuthology.orchestra.run.smithi028.stdout:osd.0: {} 2024-04-26T03:39:45.012 INFO:teuthology.orchestra.run.smithi028.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T03:39:45.027 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-04-26T03:39:45.027 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T03:39:45.030 INFO:teuthology.orchestra.run.smithi028.stdout:osd.1: {} 2024-04-26T03:39:45.031 INFO:teuthology.orchestra.run.smithi028.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T03:39:45.035 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-26T03:39:45.048 INFO:teuthology.orchestra.run.smithi028.stdout:osd.2: {} 2024-04-26T03:39:45.049 INFO:teuthology.orchestra.run.smithi028.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T03:39:45.067 INFO:teuthology.orchestra.run.smithi028.stdout:osd.3: {} 2024-04-26T03:39:45.067 INFO:teuthology.orchestra.run.smithi028.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T03:39:45.085 INFO:teuthology.orchestra.run.smithi028.stdout:osd.4: {} 2024-04-26T03:39:45.085 INFO:teuthology.orchestra.run.smithi028.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T03:39:45.102 INFO:teuthology.orchestra.run.smithi028.stdout:osd.5: {} 2024-04-26T03:39:45.103 INFO:teuthology.orchestra.run.smithi028.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T03:39:45.111 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:39:45.110+0000 7fb78c544640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31956) UID: 0 2024-04-26T03:39:45.119 INFO:teuthology.orchestra.run.smithi028.stdout:osd.6: {} 2024-04-26T03:39:45.120 INFO:teuthology.orchestra.run.smithi028.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T03:39:45.137 INFO:teuthology.orchestra.run.smithi028.stdout:osd.7: {} 2024-04-26T03:39:45.137 INFO:teuthology.orchestra.run.smithi028.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T03:39:45.154 INFO:teuthology.orchestra.run.smithi028.stdout:osd.8: {} 2024-04-26T03:39:45.154 INFO:teuthology.orchestra.run.smithi028.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T03:39:45.172 INFO:teuthology.orchestra.run.smithi028.stdout:osd.9: {} 2024-04-26T03:39:45.172 INFO:teuthology.orchestra.run.smithi028.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T03:39:45.183 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:39:45.183 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-04-26T03:39:45.183 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 0 2024-04-26T03:39:45.183 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:39:45.189 INFO:teuthology.orchestra.run.smithi028.stdout:osd.10: {} 2024-04-26T03:39:45.190 INFO:teuthology.orchestra.run.smithi028.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T03:39:45.192 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-26T03:39:45.207 INFO:teuthology.orchestra.run.smithi028.stdout:osd.11: {} 2024-04-26T03:39:45.207 INFO:teuthology.orchestra.run.smithi028.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T03:39:45.224 INFO:teuthology.orchestra.run.smithi028.stdout:osd.12: {} 2024-04-26T03:39:45.224 INFO:teuthology.orchestra.run.smithi028.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T03:39:45.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T03:39:45.223+0000 7f81715d4640 0 -- 172.21.15.28:0/3722948869 >> [v2:172.21.15.42:6817/301360893,v1:172.21.15.42:6819/301360893] conn(0x7f816c218570 msgr2=0x7f816c21a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-26T03:39:45.237 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-04-26T03:39:45.236+0000 7f60d3c4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31757) UID: 0 2024-04-26T03:39:45.261 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:39:45.261 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-04-26T03:39:45.273 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2024-04-26T03:39:44.690818+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":133644832,"num_objects":87,"num_object_clones":41,"num_object_copies":599,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":7,"num_read":472,"num_read_kb":236812,"num_write":930,"num_write_kb":126948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":31670272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":971,"ondisk_log_size":971,"up":144,"acting":144,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":151,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":773880,"kb_used_data":237448,"kb_used_omap":24,"kb_used_meta":536167,"kb_avail":1509175560,"statfs":{"total":1546188226560,"available":1545395773440,"internally_reserved":0,"allocated":243146752,"data_stored":318532779,"data_compressed":2054457,"data_compressed_allocated":4194304,"data_compressed_original":9281536,"omap_allocated":25522,"internal_metadata":549035086},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27607040,"num_objects":18,"num_object_clones":13,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":1,"num_read":41,"num_read_kb":39908,"num_write":77,"num_write_kb":16768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":25575424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.887037"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":128,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.811530+0000","last_change":"2024-04-26T03:38:37.359252+0000","last_active":"2024-04-26T03:39:41.811530+0000","last_peered":"2024-04-26T03:39:41.811530+0000","last_clean":"2024-04-26T03:39:41.811530+0000","last_became_active":"2024-04-26T03:38:37.358910+0000","last_became_peered":"2024-04-26T03:38:37.358910+0000","last_unstale":"2024-04-26T03:39:41.811530+0000","last_undegraded":"2024-04-26T03:39:41.811530+0000","last_fullsized":"2024-04-26T03:39:41.811530+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:04.186001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"3.6","version":"81'20","reported_seq":304,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T03:39:43.074998+0000","last_change":"2024-04-26T03:39:39.705400+0000","last_active":"2024-04-26T03:39:43.074998+0000","last_peered":"2024-04-26T03:39:43.074998+0000","last_clean":"2024-04-26T03:39:43.074998+0000","last_became_active":"2024-04-26T03:39:27.699531+0000","last_became_peered":"2024-04-26T03:39:27.699531+0000","last_unstale":"2024-04-26T03:39:43.074998+0000","last_undegraded":"2024-04-26T03:39:43.074998+0000","last_fullsized":"2024-04-26T03:39:43.074998+0000","mapping_epoch":32,"log_start":"64'11","ondisk_log_start":"64'11","created":32,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'9","last_scrub_stamp":"2024-04-26T03:39:07.177824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:07.177824+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":11,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:24.874885+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0016197290000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,14,6,2,12,7,1],"acting":[0,14,6,2,12,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":128,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.822761+0000","last_change":"2024-04-26T03:38:36.484782+0000","last_active":"2024-04-26T03:39:41.822761+0000","last_peered":"2024-04-26T03:39:41.822761+0000","last_clean":"2024-04-26T03:39:41.822761+0000","last_became_active":"2024-04-26T03:38:36.484388+0000","last_became_peered":"2024-04-26T03:38:36.484388+0000","last_unstale":"2024-04-26T03:39:41.822761+0000","last_undegraded":"2024-04-26T03:39:41.822761+0000","last_fullsized":"2024-04-26T03:39:41.822761+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,13],"acting":[8,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.7","version":"49'10","reported_seq":299,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T03:39:42.815292+0000","last_change":"2024-04-26T03:39:39.696366+0000","last_active":"2024-04-26T03:39:42.815292+0000","last_peered":"2024-04-26T03:39:42.815292+0000","last_clean":"2024-04-26T03:39:42.815292+0000","last_became_active":"2024-04-26T03:39:23.698262+0000","last_became_peered":"2024-04-26T03:39:23.698262+0000","last_unstale":"2024-04-26T03:39:42.815292+0000","last_undegraded":"2024-04-26T03:39:42.815292+0000","last_fullsized":"2024-04-26T03:39:42.815292+0000","mapping_epoch":32,"log_start":"49'10","ondisk_log_start":"49'10","created":32,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'10","last_scrub_stamp":"2024-04-26T03:39:09.248660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:09.248660+0000","objects_scrubbed":1,"log_size":0,"log_dups_size":10,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:38.596496+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.00066948500000000005,"stat_sum":{"num_bytes":131072,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":385,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,13,6,0,14,10,11],"acting":[4,13,6,0,14,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":127,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.811971+0000","last_change":"2024-04-26T03:38:38.187714+0000","last_active":"2024-04-26T03:39:41.811971+0000","last_peered":"2024-04-26T03:39:41.811971+0000","last_clean":"2024-04-26T03:39:41.811971+0000","last_became_active":"2024-04-26T03:38:38.187383+0000","last_became_peered":"2024-04-26T03:38:38.187383+0000","last_unstale":"2024-04-26T03:39:41.811971+0000","last_undegraded":"2024-04-26T03:39:41.811971+0000","last_fullsized":"2024-04-26T03:39:41.811971+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:59.062623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4],"acting":[9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.4","version":"85'70","reported_seq":319,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.822867+0000","last_change":"2024-04-26T03:39:39.701091+0000","last_active":"2024-04-26T03:39:41.822867+0000","last_peered":"2024-04-26T03:39:41.822867+0000","last_clean":"2024-04-26T03:39:41.822867+0000","last_became_active":"2024-04-26T03:39:35.700637+0000","last_became_peered":"2024-04-26T03:39:35.700637+0000","last_unstale":"2024-04-26T03:39:41.822867+0000","last_undegraded":"2024-04-26T03:39:41.822867+0000","last_fullsized":"2024-04-26T03:39:41.822867+0000","mapping_epoch":32,"log_start":"75'58","ondisk_log_start":"75'58","created":32,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"42'18","last_scrub_stamp":"2024-04-26T03:39:07.342868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:07.342868+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":60,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:09.889951+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0095506840000000003,"stat_sum":{"num_bytes":15155200,"num_objects":9,"num_object_clones":4,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":21,"num_read_kb":16669,"num_write":36,"num_write_kb":9840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,13,4,2,6,10,5],"acting":[8,13,4,2,6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":129,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T03:39:42.815417+0000","last_change":"2024-04-26T03:38:37.255389+0000","last_active":"2024-04-26T03:39:42.815417+0000","last_peered":"2024-04-26T03:39:42.815417+0000","last_clean":"2024-04-26T03:39:42.815417+0000","last_became_active":"2024-04-26T03:38:37.255066+0000","last_became_peered":"2024-04-26T03:38:37.255066+0000","last_unstale":"2024-04-26T03:39:42.815417+0000","last_undegraded":"2024-04-26T03:39:42.815417+0000","last_fullsized":"2024-04-26T03:39:42.815417+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:43.860480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.5","version":"76'69","reported_seq":364,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.811633+0000","last_change":"2024-04-26T03:39:39.695213+0000","last_active":"2024-04-26T03:39:41.811633+0000","last_peered":"2024-04-26T03:39:41.811633+0000","last_clean":"2024-04-26T03:39:41.811633+0000","last_became_active":"2024-04-26T03:39:31.701065+0000","last_became_peered":"2024-04-26T03:39:31.701065+0000","last_unstale":"2024-04-26T03:39:41.811633+0000","last_undegraded":"2024-04-26T03:39:41.811633+0000","last_fullsized":"2024-04-26T03:39:41.811633+0000","mapping_epoch":32,"log_start":"69'65","ondisk_log_start":"69'65","created":32,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'51","last_scrub_stamp":"2024-04-26T03:39:06.149763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:06.149763+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":75,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:26.604335+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00052917899999999996,"stat_sum":{"num_bytes":5275648,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":27990,"num_write":25,"num_write_kb":4304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,8,4,2,7,0,5],"acting":[15,8,4,2,7,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.3","version":"85'95","reported_seq":337,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.808329+0000","last_change":"2024-04-26T03:39:39.719129+0000","last_active":"2024-04-26T03:39:41.808329+0000","last_peered":"2024-04-26T03:39:41.808329+0000","last_clean":"2024-04-26T03:39:41.808329+0000","last_became_active":"2024-04-26T03:39:39.698101+0000","last_became_peered":"2024-04-26T03:39:39.698101+0000","last_unstale":"2024-04-26T03:39:41.808329+0000","last_undegraded":"2024-04-26T03:39:41.808329+0000","last_fullsized":"2024-04-26T03:39:41.808329+0000","mapping_epoch":32,"log_start":"80'84","ondisk_log_start":"80'84","created":32,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'32","last_scrub_stamp":"2024-04-26T03:39:07.451867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:07.451867+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":120,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:35.860970+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.015853349999999999,"stat_sum":{"num_bytes":18055168,"num_objects":11,"num_object_clones":6,"num_object_copies":77,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":1,"num_read":17,"num_read_kb":15702,"num_write":43,"num_write_kb":7488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":25575424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,7,4,15,1,2,3],"acting":[10,7,4,15,1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"2.2","version":"0'0","reported_seq":116,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T03:39:40.813708+0000","last_change":"2024-04-26T03:38:48.475569+0000","last_active":"2024-04-26T03:39:40.813708+0000","last_peered":"2024-04-26T03:39:40.813708+0000","last_clean":"2024-04-26T03:39:40.813708+0000","last_became_active":"2024-04-26T03:38:48.475155+0000","last_became_peered":"2024-04-26T03:38:48.475155+0000","last_unstale":"2024-04-26T03:39:40.813708+0000","last_undegraded":"2024-04-26T03:39:40.813708+0000","last_fullsized":"2024-04-26T03:39:40.813708+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,12],"acting":[11,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.0","version":"35'9","reported_seq":280,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T03:39:43.075708+0000","last_change":"2024-04-26T03:39:39.695741+0000","last_active":"2024-04-26T03:39:43.075708+0000","last_peered":"2024-04-26T03:39:43.075708+0000","last_clean":"2024-04-26T03:39:43.075708+0000","last_became_active":"2024-04-26T03:39:01.540712+0000","last_became_peered":"2024-04-26T03:39:01.540712+0000","last_unstale":"2024-04-26T03:39:43.075708+0000","last_undegraded":"2024-04-26T03:39:43.075708+0000","last_fullsized":"2024-04-26T03:39:43.075708+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'9","last_scrub_stamp":"2024-04-26T03:39:06.119088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:06.119088+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:14.749151+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0010303230000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,9,15,2,6,14,4],"acting":[1,9,15,2,6,14,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":128,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.809092+0000","last_change":"2024-04-26T03:38:37.255450+0000","last_active":"2024-04-26T03:39:41.809092+0000","last_peered":"2024-04-26T03:39:41.809092+0000","last_clean":"2024-04-26T03:39:41.809092+0000","last_became_active":"2024-04-26T03:38:37.255181+0000","last_became_peered":"2024-04-26T03:38:37.255181+0000","last_unstale":"2024-04-26T03:39:41.809092+0000","last_undegraded":"2024-04-26T03:39:41.809092+0000","last_fullsized":"2024-04-26T03:39:41.809092+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:00.449388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0],"acting":[14,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.1","version":"42'8","reported_seq":276,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T03:39:42.815359+0000","last_change":"2024-04-26T03:39:39.696946+0000","last_active":"2024-04-26T03:39:42.815359+0000","last_peered":"2024-04-26T03:39:42.815359+0000","last_clean":"2024-04-26T03:39:42.815359+0000","last_became_active":"2024-04-26T03:39:01.547451+0000","last_became_peered":"2024-04-26T03:39:01.547451+0000","last_unstale":"2024-04-26T03:39:42.815359+0000","last_undegraded":"2024-04-26T03:39:42.815359+0000","last_fullsized":"2024-04-26T03:39:42.815359+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"42'8","last_scrub_stamp":"2024-04-26T03:39:08.263903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:08.263903+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:14.658080+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0010901210000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,14,10,9,1,2],"acting":[4,11,14,10,9,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":131,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T03:39:43.827128+0000","last_change":"2024-04-26T03:38:36.431325+0000","last_active":"2024-04-26T03:39:43.827128+0000","last_peered":"2024-04-26T03:39:43.827128+0000","last_clean":"2024-04-26T03:39:43.827128+0000","last_became_active":"2024-04-26T03:38:36.431121+0000","last_became_peered":"2024-04-26T03:38:36.431121+0000","last_unstale":"2024-04-26T03:39:43.827128+0000","last_undegraded":"2024-04-26T03:39:43.827128+0000","last_fullsized":"2024-04-26T03:39:43.827128+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:53.294619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"81'65","reported_seq":350,"reported_epoch":86,"state":"clean+premerge+peered","last_fresh":"2024-04-26T03:39:41.816056+0000","last_change":"2024-04-26T03:39:41.816056+0000","last_active":"2024-04-26T03:39:40.809478+0000","last_peered":"2024-04-26T03:39:41.816056+0000","last_clean":"2024-04-26T03:39:41.816056+0000","last_became_active":"2024-04-26T03:39:01.543967+0000","last_became_peered":"2024-04-26T03:39:41.813826+0000","last_unstale":"2024-04-26T03:39:41.816056+0000","last_undegraded":"2024-04-26T03:39:41.816056+0000","last_fullsized":"2024-04-26T03:39:41.816056+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'31","last_scrub_stamp":"2024-04-26T03:39:07.479325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:07.479325+0000","objects_scrubbed":3,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:27.011441+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0027817699999999998,"stat_sum":{"num_bytes":3588096,"num_objects":5,"num_object_clones":2,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":8,"num_read_kb":4259,"num_write":51,"num_write_kb":3392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,5,3,9,15,10,11],"acting":[14,5,3,9,15,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"2.3","version":"0'0","reported_seq":128,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.809145+0000","last_change":"2024-04-26T03:38:36.431667+0000","last_active":"2024-04-26T03:39:41.809145+0000","last_peered":"2024-04-26T03:39:41.809145+0000","last_clean":"2024-04-26T03:39:41.809145+0000","last_became_active":"2024-04-26T03:38:36.431449+0000","last_became_peered":"2024-04-26T03:38:36.431449+0000","last_unstale":"2024-04-26T03:39:41.809145+0000","last_undegraded":"2024-04-26T03:39:41.809145+0000","last_fullsized":"2024-04-26T03:39:41.809145+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:47.562648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"27'157","reported_seq":337,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:42.110389+0000","last_change":"2024-04-26T03:39:42.110436+0000","last_active":"2024-04-26T03:39:42.110389+0000","last_peered":"2024-04-26T03:39:42.110389+0000","last_clean":"2024-04-26T03:39:42.110389+0000","last_became_active":"2024-04-26T03:38:34.389099+0000","last_became_peered":"2024-04-26T03:38:34.389099+0000","last_unstale":"2024-04-26T03:39:42.110389+0000","last_undegraded":"2024-04-26T03:39:42.110389+0000","last_fullsized":"2024-04-26T03:39:42.110389+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'157","last_scrub_stamp":"2024-04-26T03:39:42.110300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:32.306810+0000","last_clean_scrub_stamp":"2024-04-26T03:39:42.110300+0000","objects_scrubbed":2,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:11.925418+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.8","version":"84'113","reported_seq":336,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.808247+0000","last_change":"2024-04-26T03:39:39.711675+0000","last_active":"2024-04-26T03:39:41.808247+0000","last_peered":"2024-04-26T03:39:41.808247+0000","last_clean":"2024-04-26T03:39:41.808247+0000","last_became_active":"2024-04-26T03:39:19.697021+0000","last_became_peered":"2024-04-26T03:39:19.697021+0000","last_unstale":"2024-04-26T03:39:41.808247+0000","last_undegraded":"2024-04-26T03:39:41.808247+0000","last_fullsized":"2024-04-26T03:39:41.808247+0000","mapping_epoch":32,"log_start":"60'76","ondisk_log_start":"60'76","created":32,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"42'12","last_scrub_stamp":"2024-04-26T03:39:08.136579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:08.136579+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":92,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:28.372198+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.012835276,"stat_sum":{"num_bytes":16859136,"num_objects":6,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":20,"num_read_kb":30343,"num_write":25,"num_write_kb":12208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,10,11,15,5],"acting":[6,0,1,10,11,15,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.9","version":"84'39","reported_seq":319,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.896140+0000","last_change":"2024-04-26T03:39:39.708837+0000","last_active":"2024-04-26T03:39:41.896140+0000","last_peered":"2024-04-26T03:39:41.896140+0000","last_clean":"2024-04-26T03:39:41.896140+0000","last_became_active":"2024-04-26T03:39:15.696734+0000","last_became_peered":"2024-04-26T03:39:15.696734+0000","last_unstale":"2024-04-26T03:39:41.896140+0000","last_undegraded":"2024-04-26T03:39:41.896140+0000","last_fullsized":"2024-04-26T03:39:41.896140+0000","mapping_epoch":32,"log_start":"52'24","ondisk_log_start":"52'24","created":32,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'6","last_scrub_stamp":"2024-04-26T03:39:07.345704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:07.345704+0000","objects_scrubbed":1,"log_size":15,"log_dups_size":29,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:30.212210+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.011853795,"stat_sum":{"num_bytes":4292608,"num_objects":5,"num_object_clones":2,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":2,"num_read":8,"num_read_kb":1587,"num_write":11,"num_write_kb":3248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2,10,9,15,8,4],"acting":[13,2,10,9,15,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.a","version":"84'137","reported_seq":421,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T03:39:43.827609+0000","last_change":"2024-04-26T03:39:39.707721+0000","last_active":"2024-04-26T03:39:43.827609+0000","last_peered":"2024-04-26T03:39:43.827609+0000","last_clean":"2024-04-26T03:39:43.827609+0000","last_became_active":"2024-04-26T03:38:49.486365+0000","last_became_peered":"2024-04-26T03:38:49.486365+0000","last_unstale":"2024-04-26T03:39:43.827609+0000","last_undegraded":"2024-04-26T03:39:43.827609+0000","last_fullsized":"2024-04-26T03:39:43.827609+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:10.724979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.013310159,"stat_sum":{"num_bytes":18563072,"num_objects":14,"num_object_clones":8,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":71,"num_read_kb":40205,"num_write":99,"num_write_kb":19904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,10,13,1,11],"acting":[3,7,6,10,13,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.b","version":"85'130","reported_seq":392,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T03:39:42.815405+0000","last_change":"2024-04-26T03:39:39.714373+0000","last_active":"2024-04-26T03:39:42.815405+0000","last_peered":"2024-04-26T03:39:42.815405+0000","last_clean":"2024-04-26T03:39:42.815405+0000","last_became_active":"2024-04-26T03:38:49.940561+0000","last_became_peered":"2024-04-26T03:38:49.940561+0000","last_unstale":"2024-04-26T03:39:42.815405+0000","last_undegraded":"2024-04-26T03:39:42.815405+0000","last_fullsized":"2024-04-26T03:39:42.815405+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:57.848986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018406249,"stat_sum":{"num_bytes":14614528,"num_objects":10,"num_object_clones":6,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":40,"num_read_kb":12078,"num_write":163,"num_write_kb":20016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,10,12,1,7,14],"acting":[4,0,10,12,1,7,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.c","version":"84'64","reported_seq":313,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T03:39:40.812948+0000","last_change":"2024-04-26T03:39:39.710711+0000","last_active":"2024-04-26T03:39:40.812948+0000","last_peered":"2024-04-26T03:39:40.812948+0000","last_clean":"2024-04-26T03:39:40.812948+0000","last_became_active":"2024-04-26T03:38:49.941017+0000","last_became_peered":"2024-04-26T03:38:49.941017+0000","last_unstale":"2024-04-26T03:39:40.812948+0000","last_undegraded":"2024-04-26T03:39:40.812948+0000","last_fullsized":"2024-04-26T03:39:40.812948+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:18.122180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011208693,"stat_sum":{"num_bytes":3866624,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":18,"num_read_kb":10268,"num_write":64,"num_write_kb":7792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,0,1,9,12,14],"acting":[11,8,0,1,9,12,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.d","version":"84'81","reported_seq":334,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.822238+0000","last_change":"2024-04-26T03:39:39.696226+0000","last_active":"2024-04-26T03:39:41.822238+0000","last_peered":"2024-04-26T03:39:41.822238+0000","last_clean":"2024-04-26T03:39:41.822238+0000","last_became_active":"2024-04-26T03:38:49.511783+0000","last_became_peered":"2024-04-26T03:38:49.511783+0000","last_unstale":"2024-04-26T03:39:41.822238+0000","last_undegraded":"2024-04-26T03:39:41.822238+0000","last_fullsized":"2024-04-26T03:39:41.822238+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:08.988549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063040299999999997,"stat_sum":{"num_bytes":5144576,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":27,"num_read_kb":17290,"num_write":50,"num_write_kb":8048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,9,1,5,8,4],"acting":[7,3,9,1,5,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.e","version":"78'82","reported_seq":348,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.896216+0000","last_change":"2024-04-26T03:39:39.700929+0000","last_active":"2024-04-26T03:39:41.896216+0000","last_peered":"2024-04-26T03:39:41.896216+0000","last_clean":"2024-04-26T03:39:41.896216+0000","last_became_active":"2024-04-26T03:38:49.941260+0000","last_became_peered":"2024-04-26T03:38:49.941260+0000","last_unstale":"2024-04-26T03:39:41.896216+0000","last_undegraded":"2024-04-26T03:39:41.896216+0000","last_fullsized":"2024-04-26T03:39:41.896216+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:05.038642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035573390000000001,"stat_sum":{"num_bytes":6029312,"num_objects":6,"num_object_clones":2,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":2,"num_read":43,"num_read_kb":29296,"num_write":52,"num_write_kb":8432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,14,6,11,8,9,12],"acting":[13,14,6,11,8,9,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.f","version":"84'87","reported_seq":367,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T03:39:40.813048+0000","last_change":"2024-04-26T03:39:39.700458+0000","last_active":"2024-04-26T03:39:40.813048+0000","last_peered":"2024-04-26T03:39:40.813048+0000","last_clean":"2024-04-26T03:39:40.813048+0000","last_became_active":"2024-04-26T03:39:18.756713+0000","last_became_peered":"2024-04-26T03:39:18.756713+0000","last_unstale":"2024-04-26T03:39:40.813048+0000","last_undegraded":"2024-04-26T03:39:40.813048+0000","last_fullsized":"2024-04-26T03:39:40.813048+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:54.423720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001388121,"stat_sum":{"num_bytes":9191424,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":16567,"num_write":57,"num_write_kb":10368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":6094848,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,13,5,14,9,1,0],"acting":[11,13,5,14,9,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.10","version":"84'37","reported_seq":240,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.808311+0000","last_change":"2024-04-26T03:39:39.713445+0000","last_active":"2024-04-26T03:39:41.808311+0000","last_peered":"2024-04-26T03:39:41.808311+0000","last_clean":"2024-04-26T03:39:41.808311+0000","last_became_active":"2024-04-26T03:39:04.704276+0000","last_became_peered":"2024-04-26T03:39:04.704276+0000","last_unstale":"2024-04-26T03:39:41.808311+0000","last_undegraded":"2024-04-26T03:39:41.808311+0000","last_fullsized":"2024-04-26T03:39:41.808311+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":5,"last_scrub":"35'9","last_scrub_stamp":"2024-04-26T03:39:07.110098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:07.110098+0000","objects_scrubbed":1,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:34.389540+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.015083931,"stat_sum":{"num_bytes":6995968,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":6316,"num_write":29,"num_write_kb":5840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,7,4,8,0,11],"acting":[6,9,7,4,8,0,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.11","version":"84'26","reported_seq":210,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T03:39:43.826925+0000","last_change":"2024-04-26T03:39:39.707787+0000","last_active":"2024-04-26T03:39:43.826925+0000","last_peered":"2024-04-26T03:39:43.826925+0000","last_clean":"2024-04-26T03:39:43.826925+0000","last_became_active":"2024-04-26T03:39:07.719054+0000","last_became_peered":"2024-04-26T03:39:07.719054+0000","last_unstale":"2024-04-26T03:39:43.826925+0000","last_undegraded":"2024-04-26T03:39:43.826925+0000","last_fullsized":"2024-04-26T03:39:43.826925+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":5,"last_scrub":"42'8","last_scrub_stamp":"2024-04-26T03:39:08.233750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:08.233750+0000","objects_scrubbed":1,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:10.054003+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.011406828000000001,"stat_sum":{"num_bytes":5292032,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":7739,"num_write":21,"num_write_kb":2608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,15,10,0,14,2],"acting":[5,11,15,10,0,14,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":18,"stat_sum":{"num_bytes":133054464,"num_objects":85,"num_object_clones":41,"num_object_copies":595,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":7,"num_read":336,"num_read_kb":236694,"num_write":727,"num_write_kb":123488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":31670272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":227377152,"data_stored":306016256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":814,"ondisk_log_size":814,"up":126,"acting":126,"num_store_stats":16},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":13852,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":24,"seq":103079215123,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50620,"kb_used_data":15276,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94321220,"statfs":{"total":96636764160,"available":96584929280,"internally_reserved":0,"allocated":15642624,"data_stored":19199959,"data_compressed":136565,"data_compressed_allocated":294912,"data_compressed_original":1036288,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,4,6,7,8,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215123,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43232,"kb_used_data":12052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94328608,"statfs":{"total":96636764160,"available":96592494592,"internally_reserved":0,"allocated":12341248,"data_stored":16788667,"data_compressed":131739,"data_compressed_allocated":266240,"data_compressed_original":532480,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":23,"seq":98784247829,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53232,"kb_used_data":17892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94318608,"statfs":{"total":96636764160,"available":96582254592,"internally_reserved":0,"allocated":18321408,"data_stored":24597943,"data_compressed":129639,"data_compressed_allocated":262144,"data_compressed_original":524288,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280529,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47232,"kb_used_data":16052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94324608,"statfs":{"total":96636764160,"available":96588398592,"internally_reserved":0,"allocated":16437248,"data_stored":20111220,"data_compressed":134461,"data_compressed_allocated":290816,"data_compressed_original":1028096,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,4,5,7,8,9,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280529,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43516,"kb_used_data":16492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94328324,"statfs":{"total":96636764160,"available":96592203776,"internally_reserved":0,"allocated":16887808,"data_stored":26463572,"data_compressed":127535,"data_compressed_allocated":258048,"data_compressed_original":516096,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,3,4,5,6,8,9,10,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":22,"seq":94489280530,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39380,"kb_used_data":12360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94332460,"statfs":{"total":96636764160,"available":96596439040,"internally_reserved":0,"allocated":12656640,"data_stored":13747742,"data_compressed":125440,"data_compressed_allocated":253952,"data_compressed_original":507904,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[2,4,5,6,7,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280528,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37528,"kb_used_data":10504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94334312,"statfs":{"total":96636764160,"available":96598335488,"internally_reserved":0,"allocated":10756096,"data_stored":12909908,"data_compressed":127535,"data_compressed_allocated":258048,"data_compressed_original":516096,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,2,4,7,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280529,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55096,"kb_used_data":23916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94316744,"statfs":{"total":96636764160,"available":96580345856,"internally_reserved":0,"allocated":24489984,"data_stored":33408542,"data_compressed":125440,"data_compressed_allocated":253952,"data_compressed_original":507904,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[1,2,3,4,7,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313234,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62708,"kb_used_data":17632,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94309132,"statfs":{"total":96636764160,"available":96572551168,"internally_reserved":0,"allocated":18055168,"data_stored":22322577,"data_compressed":123349,"data_compressed_allocated":249856,"data_compressed_original":499712,"omap_allocated":1596,"internal_metadata":46135748},"hb_peers":[0,1,2,4,5,6,7,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":12,"up_from":22,"seq":94489280528,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33880,"kb_used_data":6856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337960,"statfs":{"total":96636764160,"available":96602071040,"internally_reserved":0,"allocated":7020544,"data_stored":11242836,"data_compressed":127535,"data_compressed_allocated":258048,"data_compressed_original":516096,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,7,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280528,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41008,"kb_used_data":13984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94330832,"statfs":{"total":96636764160,"available":96594771968,"internally_reserved":0,"allocated":14319616,"data_stored":17927508,"data_compressed":127535,"data_compressed_allocated":258048,"data_compressed_original":516096,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[2,4,6,7,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247826,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68944,"kb_used_data":15548,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94302896,"statfs":{"total":96636764160,"available":96566165504,"internally_reserved":0,"allocated":15921152,"data_stored":21609812,"data_compressed":127535,"data_compressed_allocated":258048,"data_compressed_original":516096,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,6,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280528,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38308,"kb_used_data":11288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94333532,"statfs":{"total":96636764160,"available":96597536768,"internally_reserved":0,"allocated":11558912,"data_stored":17249822,"data_compressed":125440,"data_compressed_allocated":253952,"data_compressed_original":507904,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,3,5,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":23,"seq":98784247828,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74824,"kb_used_data":21436,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94297016,"statfs":{"total":96636764160,"available":96560144384,"internally_reserved":0,"allocated":21950464,"data_stored":31823287,"data_compressed":129639,"data_compressed_allocated":262144,"data_compressed_original":524288,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,4,6,7,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280528,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40492,"kb_used_data":13468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94331348,"statfs":{"total":96636764160,"available":96595300352,"internally_reserved":0,"allocated":13791232,"data_stored":15289684,"data_compressed":127535,"data_compressed_allocated":258048,"data_compressed_original":516096,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,4,5,7,8,9,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247826,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43880,"kb_used_data":12692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94327960,"statfs":{"total":96636764160,"available":96591831040,"internally_reserved":0,"allocated":12996608,"data_stored":13839700,"data_compressed":127535,"data_compressed_allocated":258048,"data_compressed_original":516096,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":14942208,"data_stored":20893696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20959232,"data_stored":31096832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12017664,"data_stored":13123584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":11337728,"data_stored":16052224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":17330176,"data_stored":23871488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":14598144,"data_stored":17944576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":15405056,"data_stored":18866176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":15908864,"data_stored":25747456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":11689984,"data_stored":13041664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":9777152,"data_stored":12193792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":23523328,"data_stored":32702464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":17100800,"data_stored":21626880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":6041600,"data_stored":10526720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":13340672,"data_stored":17211392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":10592256,"data_stored":16543744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":12812288,"data_stored":14573568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T03:39:45.299 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:39:45.299 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-04-26T03:39:45.299 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-04-26T03:39:45.299 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-04-26T03:39:45.299 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:39:45.307 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-26T03:39:45.312 INFO:tasks.ceph.osd.4.smithi028.stderr:2024-04-26T03:39:45.311+0000 7fd9511b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31963) UID: 0 2024-04-26T03:39:45.413 INFO:tasks.ceph.osd.8.smithi028.stderr:2024-04-26T03:39:45.412+0000 7f5bb75d8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 31968) UID: 0 2024-04-26T03:39:45.443 INFO:teuthology.orchestra.run.smithi028.stdout:osd.13: {} 2024-04-26T03:39:45.443 INFO:teuthology.orchestra.run.smithi028.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T03:39:45.457 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:39:45.458 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-04-26T03:39:45.458 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-04-26T03:39:45.458 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-04-26T03:39:45.458 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.458 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4213.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-04-26T03:39:45.458 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:33.315386+0000", 2024-04-26T03:39:45.458 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 72.139769850999997, 2024-04-26T03:39:45.458 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.075394833, 2024-04-26T03:39:45.458 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:45.458 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:45.458 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:45.459 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:45.459 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4213", 2024-04-26T03:39:45.459 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/331201604", 2024-04-26T03:39:45.459 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 1 2024-04-26T03:39:45.459 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.459 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:45.459 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.459 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:45.459 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:33.315386+0000", 2024-04-26T03:39:45.459 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.459 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.459 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.459 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:45.459 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:33.315386+0000", 2024-04-26T03:39:45.460 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.460 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.460 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.460 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:45.460 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:33.315388+0000", 2024-04-26T03:39:45.460 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.897e-06 2024-04-26T03:39:45.460 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.460 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.460 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:45.460 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:33.315417+0000", 2024-04-26T03:39:45.460 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8983e-05 2024-04-26T03:39:45.460 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.460 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.460 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:45.461 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:33.315447+0000", 2024-04-26T03:39:45.461 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.9920000000000002e-05 2024-04-26T03:39:45.461 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.461 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.461 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:45.461 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:33.315460+0000", 2024-04-26T03:39:45.461 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3064999999999999e-05 2024-04-26T03:39:45.461 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.461 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.461 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:45.461 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:33.315770+0000", 2024-04-26T03:39:45.461 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000309687 2024-04-26T03:39:45.461 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.462 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.462 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for peered", 2024-04-26T03:39:45.462 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:33.315784+0000", 2024-04-26T03:39:45.462 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3767e-05 2024-04-26T03:39:45.462 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.462 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.462 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:45.462 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.389505+0000", 2024-04-26T03:39:45.462 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0737211259999999 2024-04-26T03:39:45.462 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.462 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.462 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:45.462 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.389667+0000", 2024-04-26T03:39:45.462 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000162435 2024-04-26T03:39:45.463 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.463 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.463 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 5", 2024-04-26T03:39:45.463 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.389837+0000", 2024-04-26T03:39:45.463 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00016988999999999999 2024-04-26T03:39:45.463 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.463 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.463 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-04-26T03:39:45.463 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.390620+0000", 2024-04-26T03:39:45.463 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000782941 2024-04-26T03:39:45.463 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.463 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.463 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-04-26T03:39:45.463 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.390712+0000", 2024-04-26T03:39:45.464 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.2119000000000004e-05 2024-04-26T03:39:45.464 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.464 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.464 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:45.464 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.390751+0000", 2024-04-26T03:39:45.464 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9168999999999997e-05 2024-04-26T03:39:45.464 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.464 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.464 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:45.464 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:34.390781+0000", 2024-04-26T03:39:45.464 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.9833999999999999e-05 2024-04-26T03:39:45.464 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.464 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:45.464 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.465 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.465 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.465 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.4s4 40/32 ECSubWrite(tid=22, reqid=client.4556.0:262, at_version=40'14, trim_to=0'0, roll_forward_to=35'12))", 2024-04-26T03:39:45.465 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:56.521692+0000", 2024-04-26T03:39:45.465 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 48.933464278999999, 2024-04-26T03:39:45.465 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.014776724, 2024-04-26T03:39:45.465 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:45.465 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:45.465 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:45.465 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:45.465 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.465 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:45.465 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.521692+0000", 2024-04-26T03:39:45.465 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.466 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.466 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.466 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:45.466 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.521692+0000", 2024-04-26T03:39:45.466 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.466 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.466 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.466 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:45.466 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.521692+0000", 2024-04-26T03:39:45.466 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.4999999999999998e-07 2024-04-26T03:39:45.466 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.466 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.466 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:45.466 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.522058+0000", 2024-04-26T03:39:45.467 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00036550299999999999 2024-04-26T03:39:45.467 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.467 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.467 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:45.467 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.522068+0000", 2024-04-26T03:39:45.467 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0263e-05 2024-04-26T03:39:45.467 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.467 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.467 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:45.467 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.522074+0000", 2024-04-26T03:39:45.467 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7209999999999999e-06 2024-04-26T03:39:45.467 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.467 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.467 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:45.468 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.522144+0000", 2024-04-26T03:39:45.468 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.0240999999999994e-05 2024-04-26T03:39:45.468 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.468 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.468 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:45.468 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.522154+0000", 2024-04-26T03:39:45.468 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0457e-05 2024-04-26T03:39:45.468 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.468 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.468 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:45.468 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.536432+0000", 2024-04-26T03:39:45.468 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.014278107999999999 2024-04-26T03:39:45.468 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.468 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.469 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:45.469 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.536469+0000", 2024-04-26T03:39:45.469 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.6081000000000002e-05 2024-04-26T03:39:45.469 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.469 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:45.469 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.469 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.469 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.469 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:266 3.8s0 3:1b3fe483:::smithi09235267-19:head [append 0~737280] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T03:39:45.469 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:56.527487+0000", 2024-04-26T03:39:45.469 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 48.92766864, 2024-04-26T03:39:45.470 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.015863135, 2024-04-26T03:39:45.470 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:45.470 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:45.470 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:45.470 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:45.470 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:45.470 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:45.470 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 266 2024-04-26T03:39:45.470 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.470 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:45.470 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.470 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:45.470 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.527487+0000", 2024-04-26T03:39:45.471 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.471 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.471 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.471 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:45.471 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.527487+0000", 2024-04-26T03:39:45.471 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.471 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.471 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.471 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:45.471 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.527489+0000", 2024-04-26T03:39:45.471 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5239999999999999e-06 2024-04-26T03:39:45.471 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.471 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.471 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:45.472 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.528331+0000", 2024-04-26T03:39:45.472 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00084205300000000005 2024-04-26T03:39:45.472 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.472 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.472 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:45.472 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.528347+0000", 2024-04-26T03:39:45.472 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5882000000000001e-05 2024-04-26T03:39:45.472 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.472 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.472 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:45.472 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.528353+0000", 2024-04-26T03:39:45.472 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.9800000000000003e-06 2024-04-26T03:39:45.472 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.473 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.473 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:45.473 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.528421+0000", 2024-04-26T03:39:45.473 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.8608000000000004e-05 2024-04-26T03:39:45.473 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.473 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.473 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:45.473 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.528526+0000", 2024-04-26T03:39:45.473 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00010430200000000001 2024-04-26T03:39:45.473 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.473 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.473 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:45.473 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.533515+0000", 2024-04-26T03:39:45.474 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0049888149999999997 2024-04-26T03:39:45.474 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.474 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.474 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:45.474 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.542563+0000", 2024-04-26T03:39:45.474 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0090485449999999999 2024-04-26T03:39:45.474 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.474 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.474 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:45.474 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.543246+0000", 2024-04-26T03:39:45.474 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00068309800000000004 2024-04-26T03:39:45.475 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.475 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.475 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:45.475 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.543351+0000", 2024-04-26T03:39:45.475 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00010432799999999999 2024-04-26T03:39:45.475 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.475 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:45.475 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.475 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.475 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.475 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:269 3.8s0 3:1b3fe483:::smithi09235267-19:head [read 0~1] snapc 0=[] RETRY=1 ondisk+retry+read+rwordered+known_if_redirected+supports_pool_eio e40)", 2024-04-26T03:39:45.475 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:56.531896+0000", 2024-04-26T03:39:45.475 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 48.923260214999999, 2024-04-26T03:39:45.475 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.017279658999999999, 2024-04-26T03:39:45.476 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:45.476 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:45.476 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:45.476 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:45.476 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:45.476 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:45.476 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 269 2024-04-26T03:39:45.476 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.476 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:45.476 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.476 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:45.476 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.531896+0000", 2024-04-26T03:39:45.476 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.477 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.477 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.477 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:45.477 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.531896+0000", 2024-04-26T03:39:45.477 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.477 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.477 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.477 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:45.477 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.531896+0000", 2024-04-26T03:39:45.477 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7599999999999997e-07 2024-04-26T03:39:45.477 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.477 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.477 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:45.477 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.531899+0000", 2024-04-26T03:39:45.478 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8550000000000002e-06 2024-04-26T03:39:45.478 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.478 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.478 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:45.478 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.531907+0000", 2024-04-26T03:39:45.478 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9349999999999994e-06 2024-04-26T03:39:45.478 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.478 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.478 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:45.478 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.531913+0000", 2024-04-26T03:39:45.478 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7540000000000003e-06 2024-04-26T03:39:45.478 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.479 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.479 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:45.479 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.542916+0000", 2024-04-26T03:39:45.479 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.01100284 2024-04-26T03:39:45.479 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.479 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.479 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:45.479 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.543008+0000", 2024-04-26T03:39:45.479 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1822000000000003e-05 2024-04-26T03:39:45.479 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.480 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.480 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:45.480 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.546345+0000", 2024-04-26T03:39:45.480 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0033376539999999998 2024-04-26T03:39:45.480 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.480 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.480 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:45.480 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.546438+0000", 2024-04-26T03:39:45.480 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.2394999999999999e-05 2024-04-26T03:39:45.480 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.480 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.480 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:45.480 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.549176+0000", 2024-04-26T03:39:45.481 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0027378279999999999 2024-04-26T03:39:45.481 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.481 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:45.481 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.481 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.481 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.481 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:443 3.18s0 3:1de5c702:::smithi09235267-2:head [copy-from ver 3] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-04-26T03:39:45.481 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:04.728363+0000", 2024-04-26T03:39:45.481 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 40.726793436000001, 2024-04-26T03:39:45.481 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.016662854000000001, 2024-04-26T03:39:45.481 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:45.481 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:45.481 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:45.481 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:45.482 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:45.482 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:45.482 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 443 2024-04-26T03:39:45.482 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.482 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:45.482 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.482 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:45.482 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728363+0000", 2024-04-26T03:39:45.482 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.482 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.482 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.482 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:45.482 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728363+0000", 2024-04-26T03:39:45.483 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.483 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.483 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.483 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:45.483 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728364+0000", 2024-04-26T03:39:45.483 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4190000000000001e-06 2024-04-26T03:39:45.483 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.483 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.483 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:45.483 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728370+0000", 2024-04-26T03:39:45.483 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.5400000000000003e-06 2024-04-26T03:39:45.483 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.483 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.483 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:45.484 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728381+0000", 2024-04-26T03:39:45.484 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0998e-05 2024-04-26T03:39:45.484 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.484 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.484 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:45.484 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728387+0000", 2024-04-26T03:39:45.484 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.2430000000000002e-06 2024-04-26T03:39:45.484 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.484 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.484 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:45.484 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728450+0000", 2024-04-26T03:39:45.484 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.3639000000000005e-05 2024-04-26T03:39:45.484 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.484 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.485 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:45.485 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728581+0000", 2024-04-26T03:39:45.485 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013051100000000001 2024-04-26T03:39:45.485 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.485 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.485 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:45.485 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.734449+0000", 2024-04-26T03:39:45.485 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0058678250000000001 2024-04-26T03:39:45.485 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.485 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.485 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:45.485 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.744550+0000", 2024-04-26T03:39:45.485 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.010101044999999999 2024-04-26T03:39:45.485 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.486 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.486 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:45.486 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.744977+0000", 2024-04-26T03:39:45.486 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00042698699999999998 2024-04-26T03:39:45.486 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.486 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.486 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:45.486 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.745025+0000", 2024-04-26T03:39:45.486 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.8646999999999997e-05 2024-04-26T03:39:45.486 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.486 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:45.486 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.487 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.487 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.487 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:484 3.18s0 3:1b3fe483:::smithi09235267-19:head [append 0~425984] snapc 9=[9] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e51)", 2024-04-26T03:39:45.487 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:06.700875+0000", 2024-04-26T03:39:45.487 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 38.754280934000001, 2024-04-26T03:39:45.487 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.016679452000000001, 2024-04-26T03:39:45.487 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:45.487 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:45.487 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:45.487 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:45.487 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:45.487 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:45.487 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 484 2024-04-26T03:39:45.487 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.488 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:45.488 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.488 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:45.488 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.700875+0000", 2024-04-26T03:39:45.488 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.488 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.488 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.488 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:45.488 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.700875+0000", 2024-04-26T03:39:45.488 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.488 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.488 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.488 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:45.488 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.700876+0000", 2024-04-26T03:39:45.489 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.2600000000000001e-07 2024-04-26T03:39:45.489 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.489 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.489 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:45.489 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.701387+0000", 2024-04-26T03:39:45.489 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00051046299999999995 2024-04-26T03:39:45.489 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.489 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.489 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:45.489 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.701400+0000", 2024-04-26T03:39:45.489 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3249e-05 2024-04-26T03:39:45.489 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.489 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.490 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:45.490 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.701419+0000", 2024-04-26T03:39:45.490 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.969e-05 2024-04-26T03:39:45.490 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.490 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.490 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:45.490 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.701489+0000", 2024-04-26T03:39:45.490 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.9465e-05 2024-04-26T03:39:45.490 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.490 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.490 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op must wait for map", 2024-04-26T03:39:45.490 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.701494+0000", 2024-04-26T03:39:45.490 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.215e-06 2024-04-26T03:39:45.491 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.491 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.491 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:45.491 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.703613+0000", 2024-04-26T03:39:45.491 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0021187630000000001 2024-04-26T03:39:45.491 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.491 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.491 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:45.491 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.703686+0000", 2024-04-26T03:39:45.491 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.2741e-05 2024-04-26T03:39:45.491 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.491 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.491 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:45.492 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.704604+0000", 2024-04-26T03:39:45.492 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00091841999999999998 2024-04-26T03:39:45.492 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.492 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.492 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:45.492 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.717233+0000", 2024-04-26T03:39:45.492 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.012629067000000001 2024-04-26T03:39:45.492 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.492 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.492 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:45.492 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.717494+0000", 2024-04-26T03:39:45.492 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00026084900000000001 2024-04-26T03:39:45.492 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.492 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.493 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:45.493 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.717555+0000", 2024-04-26T03:39:45.493 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.0603999999999998e-05 2024-04-26T03:39:45.493 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.493 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:45.493 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.493 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.493 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.493 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:488 3.18s0 3:1b3fe483:::smithi09235267-19:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e51)", 2024-04-26T03:39:45.493 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:06.707338+0000", 2024-04-26T03:39:45.493 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 38.747818256000002, 2024-04-26T03:39:45.493 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.015400540000000001, 2024-04-26T03:39:45.493 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:45.493 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:45.494 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:45.494 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:45.494 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:45.494 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:45.494 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 488 2024-04-26T03:39:45.494 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.494 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:45.494 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.494 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:45.494 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.707338+0000", 2024-04-26T03:39:45.494 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.494 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.494 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.494 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:45.495 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.707338+0000", 2024-04-26T03:39:45.495 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.495 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.495 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.495 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:45.495 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.707338+0000", 2024-04-26T03:39:45.495 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5099999999999999e-07 2024-04-26T03:39:45.495 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.495 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.495 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:45.495 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.707342+0000", 2024-04-26T03:39:45.495 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8879999999999999e-06 2024-04-26T03:39:45.495 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.495 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.496 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:45.496 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.707352+0000", 2024-04-26T03:39:45.496 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.4490000000000005e-06 2024-04-26T03:39:45.496 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.496 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.496 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:45.496 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.707360+0000", 2024-04-26T03:39:45.496 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.3229999999999998e-06 2024-04-26T03:39:45.496 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.496 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.496 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:45.496 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.718657+0000", 2024-04-26T03:39:45.496 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.011297019 2024-04-26T03:39:45.496 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.497 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.497 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:45.497 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.718712+0000", 2024-04-26T03:39:45.497 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.5241000000000002e-05 2024-04-26T03:39:45.497 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.497 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.497 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:45.497 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.721424+0000", 2024-04-26T03:39:45.497 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0027115529999999998 2024-04-26T03:39:45.497 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.497 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.497 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:45.497 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.721472+0000", 2024-04-26T03:39:45.498 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.8174999999999999e-05 2024-04-26T03:39:45.498 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.498 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.498 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:45.498 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:06.722738+0000", 2024-04-26T03:39:45.498 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001266241 2024-04-26T03:39:45.498 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.498 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:45.498 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.498 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.498 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.498 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.13s3 56/55 ECSubWrite(tid=104, reqid=client.4556.0:604, at_version=56'47, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T03:39:45.498 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:11.775017+0000", 2024-04-26T03:39:45.498 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 33.680138921999998, 2024-04-26T03:39:45.499 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.016407774, 2024-04-26T03:39:45.499 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:45.499 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:45.499 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:45.499 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:45.499 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.499 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:45.499 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:11.775017+0000", 2024-04-26T03:39:45.499 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.499 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.499 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.499 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:45.499 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:11.775017+0000", 2024-04-26T03:39:45.499 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.499 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.500 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.500 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:45.500 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:11.775019+0000", 2024-04-26T03:39:45.500 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5659999999999999e-06 2024-04-26T03:39:45.500 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.500 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.500 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:45.500 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:11.776937+0000", 2024-04-26T03:39:45.500 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001918105 2024-04-26T03:39:45.500 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.500 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.500 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:45.500 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:11.776949+0000", 2024-04-26T03:39:45.501 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1912999999999999e-05 2024-04-26T03:39:45.501 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.501 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.501 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:45.501 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:11.776955+0000", 2024-04-26T03:39:45.501 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.9390000000000002e-06 2024-04-26T03:39:45.501 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.501 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.501 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:45.501 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:11.777001+0000", 2024-04-26T03:39:45.501 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.5939999999999997e-05 2024-04-26T03:39:45.501 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.501 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.502 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:45.502 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:11.777012+0000", 2024-04-26T03:39:45.502 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.096e-05 2024-04-26T03:39:45.502 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.502 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.502 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:45.502 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:11.791372+0000", 2024-04-26T03:39:45.502 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.014360892 2024-04-26T03:39:45.502 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.502 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.502 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:45.502 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:11.791425+0000", 2024-04-26T03:39:45.503 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.2459e-05 2024-04-26T03:39:45.503 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.503 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:45.503 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.503 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.503 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.503 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:703 3.10s0 3:0a8403ef:::smithi09235267-5:head [append 0~557056] snapc f=[f,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-04-26T03:39:45.503 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:15.955111+0000", 2024-04-26T03:39:45.503 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 29.500045554, 2024-04-26T03:39:45.503 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.020941576, 2024-04-26T03:39:45.503 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:45.503 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:45.503 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:45.503 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:45.503 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:45.504 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:45.504 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 703 2024-04-26T03:39:45.504 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.504 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:45.504 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.504 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:45.504 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.955111+0000", 2024-04-26T03:39:45.504 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.504 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.504 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.504 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:45.504 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.955111+0000", 2024-04-26T03:39:45.504 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.504 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.505 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.505 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:45.505 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.955112+0000", 2024-04-26T03:39:45.505 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.606e-06 2024-04-26T03:39:45.505 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.505 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.505 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:45.505 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.957050+0000", 2024-04-26T03:39:45.505 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001937872 2024-04-26T03:39:45.505 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.505 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.505 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:45.505 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.957071+0000", 2024-04-26T03:39:45.505 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1271999999999999e-05 2024-04-26T03:39:45.505 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.506 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.506 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:45.506 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.957082+0000", 2024-04-26T03:39:45.506 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0708e-05 2024-04-26T03:39:45.506 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.506 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.506 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:45.506 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.957147+0000", 2024-04-26T03:39:45.506 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5135999999999994e-05 2024-04-26T03:39:45.506 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.506 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.506 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:45.506 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.957234+0000", 2024-04-26T03:39:45.506 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.6871000000000002e-05 2024-04-26T03:39:45.506 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.507 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.507 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:45.507 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.958104+0000", 2024-04-26T03:39:45.507 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00087001500000000002 2024-04-26T03:39:45.507 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.507 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.507 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:45.507 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.973753+0000", 2024-04-26T03:39:45.507 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.015648935999999999 2024-04-26T03:39:45.507 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.507 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.507 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:45.507 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.975989+0000", 2024-04-26T03:39:45.507 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.002236044 2024-04-26T03:39:45.507 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.508 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.508 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:45.508 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.976052+0000", 2024-04-26T03:39:45.508 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.3115999999999997e-05 2024-04-26T03:39:45.508 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.508 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:45.508 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.508 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.508 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.508 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:704 3.10s0 3:0a8403ef:::smithi09235267-5:head [append 0~475136] snapc f=[f,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-04-26T03:39:45.508 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:15.958967+0000", 2024-04-26T03:39:45.508 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 29.496189513000001, 2024-04-26T03:39:45.508 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.018099097000000001, 2024-04-26T03:39:45.508 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:45.509 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:45.509 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:45.509 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:45.509 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:45.509 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:45.509 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 704 2024-04-26T03:39:45.509 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.509 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:45.509 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.509 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:45.509 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.958967+0000", 2024-04-26T03:39:45.509 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.509 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.509 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.509 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:45.510 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.958967+0000", 2024-04-26T03:39:45.510 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.510 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.510 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.510 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:45.510 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.958968+0000", 2024-04-26T03:39:45.510 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.657e-06 2024-04-26T03:39:45.510 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.510 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.510 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:45.510 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.959507+0000", 2024-04-26T03:39:45.510 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00053840699999999997 2024-04-26T03:39:45.510 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.510 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.511 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:45.511 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.959523+0000", 2024-04-26T03:39:45.511 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6515999999999999e-05 2024-04-26T03:39:45.511 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.511 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.511 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:45.511 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.959531+0000", 2024-04-26T03:39:45.511 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9279999999999993e-06 2024-04-26T03:39:45.511 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.511 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.511 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:45.511 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.961991+0000", 2024-04-26T03:39:45.511 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0024603680000000001 2024-04-26T03:39:45.511 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.512 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.512 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:45.512 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.962070+0000", 2024-04-26T03:39:45.512 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8689999999999994e-05 2024-04-26T03:39:45.512 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.512 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.512 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:45.512 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.962702+0000", 2024-04-26T03:39:45.512 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00063166399999999999 2024-04-26T03:39:45.512 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.512 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.512 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:45.512 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.973754+0000", 2024-04-26T03:39:45.512 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.011052622 2024-04-26T03:39:45.512 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.513 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.513 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:45.513 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.977010+0000", 2024-04-26T03:39:45.513 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0032559260000000001 2024-04-26T03:39:45.513 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.513 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.513 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:45.513 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.977066+0000", 2024-04-26T03:39:45.513 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.5318999999999997e-05 2024-04-26T03:39:45.513 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.513 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:45.513 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.513 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.513 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.513 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:705 3.10s0 3:0a8403ef:::smithi09235267-5:head [append 0~507904] snapc f=[f,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-04-26T03:39:45.514 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:15.962584+0000", 2024-04-26T03:39:45.514 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 29.492572025000001, 2024-04-26T03:39:45.514 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.018831363, 2024-04-26T03:39:45.514 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:45.514 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:45.514 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:45.514 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:45.514 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:45.514 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:45.514 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 705 2024-04-26T03:39:45.514 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.514 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:45.514 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.514 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:45.515 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.962584+0000", 2024-04-26T03:39:45.515 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.515 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.515 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.515 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:45.515 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.962584+0000", 2024-04-26T03:39:45.515 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.515 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.515 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.515 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:45.515 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.962586+0000", 2024-04-26T03:39:45.515 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4810000000000001e-06 2024-04-26T03:39:45.515 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.515 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.516 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:45.516 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.963083+0000", 2024-04-26T03:39:45.516 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00049737900000000005 2024-04-26T03:39:45.516 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.516 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.516 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:45.516 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.963100+0000", 2024-04-26T03:39:45.516 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6623e-05 2024-04-26T03:39:45.516 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.516 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.516 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:45.516 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.963109+0000", 2024-04-26T03:39:45.516 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.7990000000000005e-06 2024-04-26T03:39:45.516 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.516 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.517 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:45.517 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.965845+0000", 2024-04-26T03:39:45.517 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0027354570000000002 2024-04-26T03:39:45.517 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.517 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.517 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:45.517 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.965947+0000", 2024-04-26T03:39:45.517 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000102383 2024-04-26T03:39:45.517 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.517 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.517 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:45.517 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.966825+0000", 2024-04-26T03:39:45.517 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00087811200000000001 2024-04-26T03:39:45.517 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.517 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.518 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:45.518 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.973835+0000", 2024-04-26T03:39:45.518 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0070094210000000001 2024-04-26T03:39:45.518 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.518 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.518 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:45.518 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.981333+0000", 2024-04-26T03:39:45.518 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0074981839999999998 2024-04-26T03:39:45.518 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.518 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.518 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:45.518 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.981415+0000", 2024-04-26T03:39:45.518 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2523999999999998e-05 2024-04-26T03:39:45.518 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.518 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:45.519 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.519 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.519 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.519 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:706 3.10s0 3:0a8403ef:::smithi09235267-5:head [append 0~475136] snapc f=[f,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-04-26T03:39:45.519 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:15.965849+0000", 2024-04-26T03:39:45.519 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 29.489306945999999, 2024-04-26T03:39:45.519 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.015731676999999999, 2024-04-26T03:39:45.519 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:45.519 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:45.519 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:45.519 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:45.519 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:45.519 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:45.519 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 706 2024-04-26T03:39:45.520 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.520 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:45.520 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.520 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:45.520 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.965849+0000", 2024-04-26T03:39:45.520 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.520 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.520 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.520 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:45.520 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.965849+0000", 2024-04-26T03:39:45.520 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.520 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.520 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.520 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:45.520 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.965850+0000", 2024-04-26T03:39:45.521 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1060000000000001e-06 2024-04-26T03:39:45.521 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.521 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.521 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:45.521 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.966348+0000", 2024-04-26T03:39:45.521 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00049790100000000003 2024-04-26T03:39:45.521 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.521 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.521 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:45.521 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.966361+0000", 2024-04-26T03:39:45.521 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3176000000000001e-05 2024-04-26T03:39:45.521 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.521 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.521 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:45.521 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.966371+0000", 2024-04-26T03:39:45.522 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.5599999999999999e-06 2024-04-26T03:39:45.522 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.522 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.522 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:45.522 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.970123+0000", 2024-04-26T03:39:45.522 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0037524720000000002 2024-04-26T03:39:45.522 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.522 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.522 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:45.522 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.970189+0000", 2024-04-26T03:39:45.522 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.6138999999999998e-05 2024-04-26T03:39:45.522 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.522 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.522 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:45.523 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.970750+0000", 2024-04-26T03:39:45.523 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00056040199999999999 2024-04-26T03:39:45.523 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.523 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.523 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:45.523 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.976064+0000", 2024-04-26T03:39:45.523 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0053144940000000003 2024-04-26T03:39:45.523 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.523 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.523 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:45.523 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.981522+0000", 2024-04-26T03:39:45.523 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0054579370000000004 2024-04-26T03:39:45.523 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.523 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.523 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:45.524 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.981581+0000", 2024-04-26T03:39:45.524 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.8489999999999997e-05 2024-04-26T03:39:45.524 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.524 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:45.524 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.524 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.524 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.524 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:707 3.10s0 3:0a8403ef:::smithi09235267-5:head [append 0~98304] snapc f=[f,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-04-26T03:39:45.524 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:15.966605+0000", 2024-04-26T03:39:45.524 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 29.488551408999999, 2024-04-26T03:39:45.524 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.015355298, 2024-04-26T03:39:45.524 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:45.524 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:45.524 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:45.525 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:45.525 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:45.525 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:45.525 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 707 2024-04-26T03:39:45.525 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.525 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:45.525 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.525 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:45.525 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.966605+0000", 2024-04-26T03:39:45.525 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.525 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.525 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.525 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:45.525 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.966605+0000", 2024-04-26T03:39:45.525 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.525 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.526 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.526 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:45.526 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.966606+0000", 2024-04-26T03:39:45.526 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.0599999999999999e-07 2024-04-26T03:39:45.526 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.526 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.526 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:45.526 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.966711+0000", 2024-04-26T03:39:45.526 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000105566 2024-04-26T03:39:45.526 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.526 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.526 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:45.526 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.966727+0000", 2024-04-26T03:39:45.526 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6277999999999999e-05 2024-04-26T03:39:45.527 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.527 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.527 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:45.527 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.966733+0000", 2024-04-26T03:39:45.527 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.694e-06 2024-04-26T03:39:45.527 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.527 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.527 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:45.527 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.974103+0000", 2024-04-26T03:39:45.527 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0073698349999999999 2024-04-26T03:39:45.527 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.527 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.527 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:45.527 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.974166+0000", 2024-04-26T03:39:45.528 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.2725000000000007e-05 2024-04-26T03:39:45.528 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.528 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.528 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:45.528 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.974561+0000", 2024-04-26T03:39:45.528 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000395471 2024-04-26T03:39:45.528 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.528 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.528 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:45.528 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.976345+0000", 2024-04-26T03:39:45.528 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0017836270000000001 2024-04-26T03:39:45.528 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.528 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.528 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:45.529 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.981917+0000", 2024-04-26T03:39:45.529 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0055726539999999998 2024-04-26T03:39:45.529 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.529 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.529 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:45.529 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.981960+0000", 2024-04-26T03:39:45.529 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.2542e-05 2024-04-26T03:39:45.529 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.529 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:45.529 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.529 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.529 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.529 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:708 3.10s0 3:0a8403ef:::smithi09235267-5:head [setxattr _header (56) in=7b] snapc f=[f,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-04-26T03:39:45.529 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:15.966764+0000", 2024-04-26T03:39:45.529 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 29.488391718999999, 2024-04-26T03:39:45.530 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.015696531, 2024-04-26T03:39:45.530 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:45.530 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:45.530 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:45.530 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:45.530 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:45.530 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:45.530 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 708 2024-04-26T03:39:45.530 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.530 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:45.530 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.530 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:45.530 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.966764+0000", 2024-04-26T03:39:45.530 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.531 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.531 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.531 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:45.531 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.966764+0000", 2024-04-26T03:39:45.531 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.531 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.531 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.531 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:45.531 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.966769+0000", 2024-04-26T03:39:45.531 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.1019999999999996e-06 2024-04-26T03:39:45.531 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.531 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.531 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:45.531 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.966773+0000", 2024-04-26T03:39:45.532 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.6270000000000002e-06 2024-04-26T03:39:45.532 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.532 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.532 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:45.532 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.966783+0000", 2024-04-26T03:39:45.532 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.8609999999999993e-06 2024-04-26T03:39:45.532 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.532 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.532 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:45.532 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.966787+0000", 2024-04-26T03:39:45.532 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.374e-06 2024-04-26T03:39:45.532 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.532 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.532 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:45.532 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.975352+0000", 2024-04-26T03:39:45.533 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0085645470000000005 2024-04-26T03:39:45.533 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.533 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.533 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:45.533 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.975428+0000", 2024-04-26T03:39:45.533 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.6372000000000007e-05 2024-04-26T03:39:45.533 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.533 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.533 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:45.533 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.975698+0000", 2024-04-26T03:39:45.533 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00027010100000000002 2024-04-26T03:39:45.533 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.533 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.533 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:45.533 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.976787+0000", 2024-04-26T03:39:45.534 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0010884740000000001 2024-04-26T03:39:45.534 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.534 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.534 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:45.534 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.982394+0000", 2024-04-26T03:39:45.534 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0056069789999999998 2024-04-26T03:39:45.534 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.534 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.534 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:45.534 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.982461+0000", 2024-04-26T03:39:45.534 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7094000000000005e-05 2024-04-26T03:39:45.534 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.534 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:45.534 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.535 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.535 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.535 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:709 3.10s0 3:0a8403ef:::smithi09235267-5:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e60)", 2024-04-26T03:39:45.535 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:15.966795+0000", 2024-04-26T03:39:45.535 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 29.488361247, 2024-04-26T03:39:45.535 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.01702424, 2024-04-26T03:39:45.535 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:45.535 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:45.535 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:45.535 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:45.535 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:45.535 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:45.535 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 709 2024-04-26T03:39:45.535 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.535 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:45.536 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.536 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:45.536 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.966795+0000", 2024-04-26T03:39:45.536 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.536 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.536 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.536 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:45.536 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.966795+0000", 2024-04-26T03:39:45.536 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.536 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.536 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.536 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:45.536 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.966795+0000", 2024-04-26T03:39:45.536 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.5600000000000001e-07 2024-04-26T03:39:45.536 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.536 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.537 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:45.537 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.966797+0000", 2024-04-26T03:39:45.537 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.894e-06 2024-04-26T03:39:45.537 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.537 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.537 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:45.537 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.966805+0000", 2024-04-26T03:39:45.537 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5789999999999998e-06 2024-04-26T03:39:45.537 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.537 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.537 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:45.537 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.966808+0000", 2024-04-26T03:39:45.537 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.3979999999999999e-06 2024-04-26T03:39:45.537 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.537 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.538 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:45.538 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.976218+0000", 2024-04-26T03:39:45.538 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0094095600000000008 2024-04-26T03:39:45.538 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.538 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.538 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:45.538 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.976269+0000", 2024-04-26T03:39:45.538 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.1554999999999997e-05 2024-04-26T03:39:45.538 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.538 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.538 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:45.538 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.982846+0000", 2024-04-26T03:39:45.538 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.006576363 2024-04-26T03:39:45.538 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.539 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.539 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:45.539 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.982912+0000", 2024-04-26T03:39:45.539 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.6713999999999999e-05 2024-04-26T03:39:45.539 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.539 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.539 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:45.539 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:15.983819+0000", 2024-04-26T03:39:45.539 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00090682100000000004 2024-04-26T03:39:45.539 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.539 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:45.539 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.539 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.539 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.539 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.as2 65/32 ECSubWrite(tid=120, reqid=client.4556.0:760, at_version=65'70, trim_to=0'0, roll_forward_to=64'68))", 2024-04-26T03:39:45.540 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:20.749574+0000", 2024-04-26T03:39:45.540 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 24.705581583000001, 2024-04-26T03:39:45.540 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.016094267999999998, 2024-04-26T03:39:45.540 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:45.540 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:45.540 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:45.540 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:45.540 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.540 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:45.540 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.749574+0000", 2024-04-26T03:39:45.540 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.540 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.540 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.540 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:45.540 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.749574+0000", 2024-04-26T03:39:45.541 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.541 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.541 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.541 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:45.541 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.749575+0000", 2024-04-26T03:39:45.541 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.0500000000000004e-07 2024-04-26T03:39:45.541 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.541 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.541 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:45.541 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.750293+0000", 2024-04-26T03:39:45.541 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00071780200000000002 2024-04-26T03:39:45.541 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.541 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.541 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:45.541 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.750301+0000", 2024-04-26T03:39:45.542 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.6090000000000001e-06 2024-04-26T03:39:45.542 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.542 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.542 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:45.542 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.750309+0000", 2024-04-26T03:39:45.542 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8499999999999994e-06 2024-04-26T03:39:45.542 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.542 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.542 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:45.542 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.750388+0000", 2024-04-26T03:39:45.542 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8306999999999994e-05 2024-04-26T03:39:45.542 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.542 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.542 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:45.543 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.750398+0000", 2024-04-26T03:39:45.543 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0421000000000001e-05 2024-04-26T03:39:45.543 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.543 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.543 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:45.543 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.765629+0000", 2024-04-26T03:39:45.543 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.015231246 2024-04-26T03:39:45.543 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.543 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.543 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:45.543 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.765669+0000", 2024-04-26T03:39:45.543 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9527999999999998e-05 2024-04-26T03:39:45.543 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.543 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:45.543 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.544 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.544 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.544 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:895 3.10s0 3:0a8403ef:::smithi09235267-5:head [copy-from ver 50] snapc 1b=[1b,1a,19,18,17,15,14,12] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-04-26T03:39:45.544 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:35.693527+0000", 2024-04-26T03:39:45.544 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 9.7616288820000001, 2024-04-26T03:39:45.544 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.086721122999999997, 2024-04-26T03:39:45.544 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:45.544 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:45.544 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:45.544 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:45.544 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:45.544 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:45.544 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 895 2024-04-26T03:39:45.544 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.544 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:45.545 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.545 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:45.545 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.693527+0000", 2024-04-26T03:39:45.545 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.545 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.545 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.545 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:45.545 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.693527+0000", 2024-04-26T03:39:45.545 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.545 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.545 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.545 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:45.545 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.693529+0000", 2024-04-26T03:39:45.545 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1509999999999998e-06 2024-04-26T03:39:45.545 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.546 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.546 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:45.546 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.693540+0000", 2024-04-26T03:39:45.546 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0875e-05 2024-04-26T03:39:45.546 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.546 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.546 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:45.546 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.693562+0000", 2024-04-26T03:39:45.546 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1744000000000001e-05 2024-04-26T03:39:45.546 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.546 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.546 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:45.546 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.693575+0000", 2024-04-26T03:39:45.546 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3081000000000001e-05 2024-04-26T03:39:45.546 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.547 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.547 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:45.547 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.693657+0000", 2024-04-26T03:39:45.547 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2445000000000001e-05 2024-04-26T03:39:45.547 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.547 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.547 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op must wait for map", 2024-04-26T03:39:45.547 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.693664+0000", 2024-04-26T03:39:45.547 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.0800000000000002e-06 2024-04-26T03:39:45.547 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.547 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.547 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:45.547 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.697116+0000", 2024-04-26T03:39:45.547 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0034522229999999999 2024-04-26T03:39:45.547 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.548 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.548 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:45.548 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.697271+0000", 2024-04-26T03:39:45.548 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000154819 2024-04-26T03:39:45.548 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.548 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.548 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:45.548 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.740027+0000", 2024-04-26T03:39:45.548 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.042755913 2024-04-26T03:39:45.548 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.548 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.548 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:45.548 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.765141+0000", 2024-04-26T03:39:45.548 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.025114969000000001 2024-04-26T03:39:45.548 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.549 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.549 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:45.549 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.780180+0000", 2024-04-26T03:39:45.549 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.015038133 2024-04-26T03:39:45.549 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.549 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.549 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:45.549 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.780248+0000", 2024-04-26T03:39:45.549 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.8689999999999995e-05 2024-04-26T03:39:45.549 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.549 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:45.549 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.549 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.549 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.549 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:896 3.10s0 3:0a8403ef:::smithi09235267-5:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e80)", 2024-04-26T03:39:45.550 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:35.693604+0000", 2024-04-26T03:39:45.550 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 9.7615519679999991, 2024-04-26T03:39:45.550 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.070589022000000001, 2024-04-26T03:39:45.550 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:45.550 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:45.550 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:45.550 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:45.550 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:45.550 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:45.550 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 896 2024-04-26T03:39:45.550 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.550 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:45.550 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.550 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:45.551 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.693604+0000", 2024-04-26T03:39:45.551 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.551 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.551 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.551 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:45.551 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.693604+0000", 2024-04-26T03:39:45.551 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.551 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.551 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.551 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:45.551 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.693605+0000", 2024-04-26T03:39:45.551 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.7400000000000002e-07 2024-04-26T03:39:45.551 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.551 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.551 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:45.552 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.693610+0000", 2024-04-26T03:39:45.552 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.7720000000000002e-06 2024-04-26T03:39:45.552 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.552 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.552 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:45.552 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.693621+0000", 2024-04-26T03:39:45.552 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1147e-05 2024-04-26T03:39:45.552 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.552 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.552 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:45.552 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.693628+0000", 2024-04-26T03:39:45.552 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3989999999999998e-06 2024-04-26T03:39:45.552 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.552 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.552 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:45.553 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.693789+0000", 2024-04-26T03:39:45.553 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0001606 2024-04-26T03:39:45.553 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.553 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.553 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting_for_map not empty", 2024-04-26T03:39:45.553 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.693796+0000", 2024-04-26T03:39:45.553 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.6899999999999992e-06 2024-04-26T03:39:45.553 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.553 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.553 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:45.553 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.697547+0000", 2024-04-26T03:39:45.553 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0037509150000000001 2024-04-26T03:39:45.553 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.553 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.553 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for blocked object", 2024-04-26T03:39:45.554 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.697648+0000", 2024-04-26T03:39:45.554 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000100698 2024-04-26T03:39:45.554 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.554 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.554 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:45.554 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.763922+0000", 2024-04-26T03:39:45.554 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.066273839000000001 2024-04-26T03:39:45.554 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.554 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.554 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:45.554 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.764053+0000", 2024-04-26T03:39:45.554 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013142699999999999 2024-04-26T03:39:45.554 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.554 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.555 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:45.555 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.764193+0000", 2024-04-26T03:39:45.555 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013976099999999999 2024-04-26T03:39:45.555 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.555 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:45.555 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.555 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.555 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.555 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:897 3.8s0 3:1b3fe483:::smithi09235267-19:head [copy-from ver 46] snapc 1b=[1b,1a,19,18,17,15,14,12] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-04-26T03:39:45.555 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:35.693640+0000", 2024-04-26T03:39:45.555 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 9.7615157359999998, 2024-04-26T03:39:45.555 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.043312878999999999, 2024-04-26T03:39:45.555 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:45.555 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:45.555 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:45.556 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:45.556 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:45.556 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:45.556 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 897 2024-04-26T03:39:45.556 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.556 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:45.556 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.556 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:45.556 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.693640+0000", 2024-04-26T03:39:45.556 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.556 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.556 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.556 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:45.556 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.693640+0000", 2024-04-26T03:39:45.557 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.557 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.557 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.557 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:45.557 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.693641+0000", 2024-04-26T03:39:45.557 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.4099999999999999e-07 2024-04-26T03:39:45.557 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.557 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.557 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:45.557 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.693646+0000", 2024-04-26T03:39:45.557 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.5500000000000002e-06 2024-04-26T03:39:45.557 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.557 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.557 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:45.557 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.693654+0000", 2024-04-26T03:39:45.558 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5870000000000004e-06 2024-04-26T03:39:45.558 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.558 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.558 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:45.558 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.693660+0000", 2024-04-26T03:39:45.558 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.3759999999999996e-06 2024-04-26T03:39:45.558 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.558 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.558 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:45.558 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.693733+0000", 2024-04-26T03:39:45.558 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.2438e-05 2024-04-26T03:39:45.558 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.558 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.558 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op must wait for map", 2024-04-26T03:39:45.559 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.693741+0000", 2024-04-26T03:39:45.559 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8210000000000002e-06 2024-04-26T03:39:45.559 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.559 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.559 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:45.559 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.696773+0000", 2024-04-26T03:39:45.559 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0030325199999999999 2024-04-26T03:39:45.559 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.559 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.559 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:45.559 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.696894+0000", 2024-04-26T03:39:45.559 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000120895 2024-04-26T03:39:45.559 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.559 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.560 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:45.560 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.716608+0000", 2024-04-26T03:39:45.560 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.019714461999999999 2024-04-26T03:39:45.560 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.560 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.560 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:45.560 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.736518+0000", 2024-04-26T03:39:45.560 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.019909145999999999 2024-04-26T03:39:45.560 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.560 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.560 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:45.560 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.736570+0000", 2024-04-26T03:39:45.560 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.2750000000000001e-05 2024-04-26T03:39:45.560 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.560 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.561 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:45.561 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.736953+0000", 2024-04-26T03:39:45.561 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00038289299999999998 2024-04-26T03:39:45.561 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.561 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:45.561 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.561 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.561 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.561 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:898 3.8s0 3:1b3fe483:::smithi09235267-19:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e80)", 2024-04-26T03:39:45.561 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:35.693670+0000", 2024-04-26T03:39:45.561 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 9.7614861170000005, 2024-04-26T03:39:45.561 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.041033587000000003, 2024-04-26T03:39:45.561 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:45.561 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:45.562 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:45.562 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:45.562 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:45.562 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:45.562 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 898 2024-04-26T03:39:45.562 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.562 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:45.562 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.562 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:45.562 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.693670+0000", 2024-04-26T03:39:45.562 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.562 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.562 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.562 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:45.563 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.693670+0000", 2024-04-26T03:39:45.563 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.563 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.563 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.563 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:45.563 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.693670+0000", 2024-04-26T03:39:45.563 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.5300000000000004e-07 2024-04-26T03:39:45.563 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.563 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.563 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:45.563 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.693674+0000", 2024-04-26T03:39:45.563 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.4060000000000001e-06 2024-04-26T03:39:45.563 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.563 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.563 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:45.564 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.693685+0000", 2024-04-26T03:39:45.564 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0851e-05 2024-04-26T03:39:45.564 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.564 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.564 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:45.564 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.693692+0000", 2024-04-26T03:39:45.564 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7549999999999997e-06 2024-04-26T03:39:45.564 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.564 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.564 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:45.564 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.693809+0000", 2024-04-26T03:39:45.564 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000117588 2024-04-26T03:39:45.564 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.564 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.564 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting_for_map not empty", 2024-04-26T03:39:45.565 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.693814+0000", 2024-04-26T03:39:45.565 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.8389999999999997e-06 2024-04-26T03:39:45.565 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.565 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.565 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:45.565 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.697142+0000", 2024-04-26T03:39:45.565 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.003328396 2024-04-26T03:39:45.565 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.565 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.565 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for blocked object", 2024-04-26T03:39:45.565 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.697229+0000", 2024-04-26T03:39:45.565 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7054999999999999e-05 2024-04-26T03:39:45.565 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.565 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.566 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:45.566 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.734547+0000", 2024-04-26T03:39:45.566 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.037317845000000002 2024-04-26T03:39:45.566 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.566 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.566 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:45.566 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.734620+0000", 2024-04-26T03:39:45.566 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.2795999999999999e-05 2024-04-26T03:39:45.566 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.566 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.566 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:45.566 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.734704+0000", 2024-04-26T03:39:45.566 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.3503000000000003e-05 2024-04-26T03:39:45.566 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.567 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:45.567 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.567 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.567 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:45.567 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:39:45.568 INFO:teuthology.orchestra.run.smithi028.stdout:osd.14: {} 2024-04-26T03:39:45.568 INFO:teuthology.orchestra.run.smithi028.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T03:39:45.569 INFO:teuthology.orchestra.run.smithi028.stdout:osd.15: {} 2024-04-26T03:39:45.569 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-04-26T03:39:45.570 INFO:teuthology.orchestra.run.smithi028.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T03:39:45.571 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:39:45.567+0000 7fb78c544640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31956) UID: 0 2024-04-26T03:39:45.668 INFO:tasks.ceph.osd.14.smithi066.stderr:2024-04-26T03:39:45.668+0000 7f5395f08640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 31765) UID: 0 2024-04-26T03:39:45.677 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:39:45.677 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-04-26T03:39:45.677 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 0 2024-04-26T03:39:45.677 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:39:45.686 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-04-26T03:39:45.697 INFO:tasks.rados.rados.0.smithi092.stdout:388: done (3 left) 2024-04-26T03:39:45.697 INFO:tasks.rados.rados.0.smithi092.stdout:390: done (2 left) 2024-04-26T03:39:45.697 INFO:tasks.rados.rados.0.smithi092.stdout:391: done (1 left) 2024-04-26T03:39:45.698 INFO:tasks.rados.rados.0.smithi092.stdout:392: done (0 left) 2024-04-26T03:39:45.698 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 4 current snap is 31 2024-04-26T03:39:45.698 INFO:tasks.rados.rados.0.smithi092.stdout:393: seq_num 169 ranges {0=442368} 2024-04-26T03:39:45.700 INFO:tasks.rados.rados.0.smithi092.stdout:393: writing smithi09235267-4 from 0 to 442368 tid 1 2024-04-26T03:39:45.700 INFO:tasks.rados.rados.0.smithi092.stdout:394: read oid 9 snap -1 2024-04-26T03:39:45.700 INFO:tasks.rados.rados.0.smithi092.stdout:394: expect deleted 2024-04-26T03:39:45.700 INFO:tasks.rados.rados.0.smithi092.stdout:395: delete oid 22 current snap is 31 2024-04-26T03:39:45.705 INFO:tasks.rados.rados.0.smithi092.stdout:394: done (2 left) 2024-04-26T03:39:45.705 INFO:tasks.rados.rados.0.smithi092.stdout:395: done (1 left) 2024-04-26T03:39:45.706 INFO:tasks.rados.rados.0.smithi092.stdout:396: read oid 14 snap -1 2024-04-26T03:39:45.706 INFO:tasks.rados.rados.0.smithi092.stdout:396: expect (ObjNum 52 snap 0 seq_num 52) 2024-04-26T03:39:45.706 INFO:tasks.rados.rados.0.smithi092.stdout:397: setattr oid 45 current snap is 31 2024-04-26T03:39:45.707 INFO:tasks.rados.rados.0.smithi092.stdout:398: read oid 8 snap 24 2024-04-26T03:39:45.707 INFO:tasks.rados.rados.0.smithi092.stdout:398: expect (ObjNum 7 snap 0 seq_num 7) 2024-04-26T03:39:45.707 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 50 current snap is 31 2024-04-26T03:39:45.707 INFO:tasks.rados.rados.0.smithi092.stdout:399: seq_num 171 ranges {1900544=475136} 2024-04-26T03:39:45.721 INFO:tasks.rados.rados.0.smithi092.stdout:399: writing smithi09235267-50 from 1900544 to 2375680 tid 1 2024-04-26T03:39:45.721 INFO:tasks.rados.rados.0.smithi092.stdout: waiting on 5 2024-04-26T03:39:45.721 INFO:tasks.rados.rados.0.smithi092.stdout:393: finishing write tid 1 to smithi09235267-4 2024-04-26T03:39:45.721 INFO:tasks.rados.rados.0.smithi092.stdout:398: expect (ObjNum 7 snap 0 seq_num 7) 2024-04-26T03:39:45.725 INFO:tasks.rados.rados.0.smithi092.stdout:393: finishing write tid 2 to smithi09235267-4 2024-04-26T03:39:45.725 INFO:tasks.rados.rados.0.smithi092.stdout:393: finishing write tid 3 to smithi09235267-4 2024-04-26T03:39:45.725 INFO:tasks.rados.rados.0.smithi092.stdout:393: oid 4 updating version 0 to 68 2024-04-26T03:39:45.725 INFO:tasks.rados.rados.0.smithi092.stdout:393: oid 4 version 68 is already newer than 67 2024-04-26T03:39:45.725 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 4 v 68 (ObjNum 169 snap 31 seq_num 169) dirty exists 2024-04-26T03:39:45.725 INFO:tasks.rados.rados.0.smithi092.stdout:393: left oid 4 (ObjNum 169 snap 31 seq_num 169) 2024-04-26T03:39:45.725 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 45 v 136 (ObjNum 145 snap 20 seq_num 145) dirty exists 2024-04-26T03:39:45.725 INFO:tasks.rados.rados.0.smithi092.stdout:393: done (4 left) 2024-04-26T03:39:45.725 INFO:tasks.rados.rados.0.smithi092.stdout:397: done (3 left) 2024-04-26T03:39:45.725 INFO:tasks.rados.rados.0.smithi092.stdout:398: done (2 left) 2024-04-26T03:39:45.725 INFO:tasks.rados.rados.0.smithi092.stdout: waiting on 2 2024-04-26T03:39:45.725 INFO:tasks.rados.rados.0.smithi092.stdout:396: expect (ObjNum 52 snap 0 seq_num 52) 2024-04-26T03:39:45.754 INFO:tasks.rados.rados.0.smithi092.stdout:399: finishing write tid 1 to smithi09235267-50 2024-04-26T03:39:45.754 INFO:tasks.rados.rados.0.smithi092.stdout:399: finishing write tid 2 to smithi09235267-50 2024-04-26T03:39:45.755 INFO:tasks.rados.rados.0.smithi092.stdout:396: done (1 left) 2024-04-26T03:39:45.755 INFO:tasks.rados.rados.0.smithi092.stdout: waiting on 1 2024-04-26T03:39:45.755 INFO:tasks.rados.rados.0.smithi092.stdout:399: finishing write tid 3 to smithi09235267-50 2024-04-26T03:39:45.755 INFO:tasks.rados.rados.0.smithi092.stdout:399: oid 50 updating version 0 to 142 2024-04-26T03:39:45.755 INFO:tasks.rados.rados.0.smithi092.stdout:399: oid 50 version 142 is already newer than 141 2024-04-26T03:39:45.755 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 50 v 142 (ObjNum 171 snap 31 seq_num 171) dirty exists 2024-04-26T03:39:45.755 INFO:tasks.rados.rados.0.smithi092.stdout:399: left oid 50 (ObjNum 171 snap 31 seq_num 171) 2024-04-26T03:39:45.755 INFO:tasks.rados.rados.0.smithi092.stdout:399: done (0 left) 2024-04-26T03:39:45.757 INFO:tasks.rados.rados.0.smithi092.stderr:0 errors. 2024-04-26T03:39:45.757 INFO:tasks.rados.rados.0.smithi092.stderr: 2024-04-26T03:39:45.760 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T03:39:45.801 INFO:tasks.ceph.osd.15.smithi092.stderr:2024-04-26T03:39:45.800+0000 7f6d12dec640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 31824) UID: 0 2024-04-26T03:39:45.832 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:39:45.832 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-04-26T03:39:45.832 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-04-26T03:39:45.832 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-04-26T03:39:45.832 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:39:45.842 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-04-26T03:39:45.870 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-04-26T03:39:45.869+0000 7fe9223c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31954) UID: 0 2024-04-26T03:39:45.950 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:39:45.950 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-04-26T03:39:45.950 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-04-26T03:39:45.950 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-04-26T03:39:45.951 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.951 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:18 3.3s0 3:c944ba3d:::smithi09235267-6:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:45.951 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.486578+0000", 2024-04-26T03:39:45.951 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 54.461463101, 2024-04-26T03:39:45.951 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.024780098, 2024-04-26T03:39:45.951 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:45.951 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:45.951 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:45.951 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:45.951 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:45.951 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:45.951 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 18 2024-04-26T03:39:45.951 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.951 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:45.951 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.952 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:45.952 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.486578+0000", 2024-04-26T03:39:45.952 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.952 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.952 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.952 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:45.952 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.486578+0000", 2024-04-26T03:39:45.952 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.952 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.952 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.952 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:45.952 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.486579+0000", 2024-04-26T03:39:45.952 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0529999999999999e-06 2024-04-26T03:39:45.952 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.952 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.953 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:45.953 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.486589+0000", 2024-04-26T03:39:45.953 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.9359999999999994e-06 2024-04-26T03:39:45.953 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.953 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.953 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:45.953 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.486601+0000", 2024-04-26T03:39:45.953 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2309e-05 2024-04-26T03:39:45.953 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.953 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.953 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:45.953 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.486607+0000", 2024-04-26T03:39:45.954 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5209999999999999e-06 2024-04-26T03:39:45.954 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.954 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.954 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:45.954 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.491290+0000", 2024-04-26T03:39:45.954 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0046821939999999998 2024-04-26T03:39:45.954 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.954 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.954 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:45.954 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.491369+0000", 2024-04-26T03:39:45.954 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9204999999999998e-05 2024-04-26T03:39:45.954 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.954 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.955 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:45.955 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.499348+0000", 2024-04-26T03:39:45.955 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0079792289999999991 2024-04-26T03:39:45.955 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.955 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.955 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:45.955 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.499429+0000", 2024-04-26T03:39:45.955 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.1051999999999996e-05 2024-04-26T03:39:45.955 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.955 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.955 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:45.955 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.511358+0000", 2024-04-26T03:39:45.955 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.011928599 2024-04-26T03:39:45.955 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.955 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:45.956 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.956 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.956 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.956 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:88 3.3s0 3:c8a9e28b:::smithi09235267-30:head [append 0~491520,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:45.956 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.543490+0000", 2024-04-26T03:39:45.956 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 54.404550334, 2024-04-26T03:39:45.956 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.01534287, 2024-04-26T03:39:45.956 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:45.956 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:45.956 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:45.956 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:45.956 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:45.956 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:45.956 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 88 2024-04-26T03:39:45.957 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.957 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:45.957 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.957 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:45.957 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.543490+0000", 2024-04-26T03:39:45.957 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.957 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.957 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.957 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:45.957 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.543490+0000", 2024-04-26T03:39:45.957 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.957 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.957 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.958 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:45.958 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.543492+0000", 2024-04-26T03:39:45.958 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5349999999999999e-06 2024-04-26T03:39:45.958 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.958 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.958 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:45.958 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.543954+0000", 2024-04-26T03:39:45.958 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00046193100000000001 2024-04-26T03:39:45.958 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.958 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.958 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:45.958 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.543967+0000", 2024-04-26T03:39:45.958 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3191e-05 2024-04-26T03:39:45.959 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.959 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.959 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:45.959 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.543972+0000", 2024-04-26T03:39:45.959 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.2950000000000001e-06 2024-04-26T03:39:45.959 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.959 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.959 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:45.959 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.544031+0000", 2024-04-26T03:39:45.959 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.8153999999999999e-05 2024-04-26T03:39:45.959 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.959 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.959 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:45.959 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.544176+0000", 2024-04-26T03:39:45.959 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014557900000000001 2024-04-26T03:39:45.960 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.960 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.960 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:45.960 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.545222+0000", 2024-04-26T03:39:45.960 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0010454590000000001 2024-04-26T03:39:45.960 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.960 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.960 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:45.960 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.551331+0000", 2024-04-26T03:39:45.960 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0061097180000000001 2024-04-26T03:39:45.960 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.960 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.960 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:45.961 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.558761+0000", 2024-04-26T03:39:45.961 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0074299800000000001 2024-04-26T03:39:45.961 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.961 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.961 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:45.961 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.558833+0000", 2024-04-26T03:39:45.961 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.2027999999999995e-05 2024-04-26T03:39:45.961 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.961 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:45.961 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.961 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.961 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.961 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:89 3.3s0 3:c8a9e28b:::smithi09235267-30:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:45.962 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.543986+0000", 2024-04-26T03:39:45.962 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 54.404054258999999, 2024-04-26T03:39:45.962 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.015002372, 2024-04-26T03:39:45.962 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:45.962 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:45.962 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:45.962 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:45.962 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:45.962 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:45.962 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 89 2024-04-26T03:39:45.962 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.962 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:45.962 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.962 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:45.963 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.543986+0000", 2024-04-26T03:39:45.963 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.963 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.963 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.963 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:45.963 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.543986+0000", 2024-04-26T03:39:45.963 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.963 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.963 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.963 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:45.963 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.543987+0000", 2024-04-26T03:39:45.963 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5899999999999996e-07 2024-04-26T03:39:45.963 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.963 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.964 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:45.964 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.543991+0000", 2024-04-26T03:39:45.964 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.58e-06 2024-04-26T03:39:45.964 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.964 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.964 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:45.964 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.543998+0000", 2024-04-26T03:39:45.964 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3119999999999996e-06 2024-04-26T03:39:45.964 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.964 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.964 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:45.964 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.544001+0000", 2024-04-26T03:39:45.964 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.399e-06 2024-04-26T03:39:45.964 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.964 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.964 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:45.965 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.550243+0000", 2024-04-26T03:39:45.965 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0062414180000000003 2024-04-26T03:39:45.965 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.965 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.965 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:45.965 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.550330+0000", 2024-04-26T03:39:45.965 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.6897999999999994e-05 2024-04-26T03:39:45.965 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.965 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.965 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:45.965 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.550794+0000", 2024-04-26T03:39:45.965 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00046398799999999998 2024-04-26T03:39:45.965 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.965 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.965 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:45.965 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.557987+0000", 2024-04-26T03:39:45.966 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0071936109999999999 2024-04-26T03:39:45.966 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.966 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.966 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:45.966 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.558928+0000", 2024-04-26T03:39:45.966 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000940153 2024-04-26T03:39:45.966 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.966 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.966 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:45.966 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.558989+0000", 2024-04-26T03:39:45.966 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.1353999999999996e-05 2024-04-26T03:39:45.966 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.966 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:45.967 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.967 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.967 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.967 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:90 3.3s0 3:c8a9e28b:::smithi09235267-30:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:45.967 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.544026+0000", 2024-04-26T03:39:45.967 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 54.404014367999999, 2024-04-26T03:39:45.967 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.017169266999999998, 2024-04-26T03:39:45.967 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:45.967 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:45.967 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:45.967 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:45.967 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:45.967 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:45.967 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 90 2024-04-26T03:39:45.968 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.968 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:45.968 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.968 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:45.968 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.544026+0000", 2024-04-26T03:39:45.968 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.968 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.968 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.968 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:45.968 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.544026+0000", 2024-04-26T03:39:45.968 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.968 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.968 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.968 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:45.969 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.544027+0000", 2024-04-26T03:39:45.969 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.0999999999999998e-07 2024-04-26T03:39:45.969 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.969 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.969 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:45.969 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.544030+0000", 2024-04-26T03:39:45.969 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6410000000000002e-06 2024-04-26T03:39:45.969 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.969 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.969 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:45.969 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.544037+0000", 2024-04-26T03:39:45.969 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.2459999999999997e-06 2024-04-26T03:39:45.969 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.969 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.969 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:45.970 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.544042+0000", 2024-04-26T03:39:45.970 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.9919999999999998e-06 2024-04-26T03:39:45.970 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.970 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.970 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:45.970 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.551230+0000", 2024-04-26T03:39:45.970 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0071875619999999998 2024-04-26T03:39:45.970 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.970 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.970 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:45.970 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.551311+0000", 2024-04-26T03:39:45.970 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.1182999999999998e-05 2024-04-26T03:39:45.970 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.970 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.971 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:45.971 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.559039+0000", 2024-04-26T03:39:45.971 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0077281479999999998 2024-04-26T03:39:45.971 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.971 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.971 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:45.971 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.559125+0000", 2024-04-26T03:39:45.971 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5678000000000002e-05 2024-04-26T03:39:45.971 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.971 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.971 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:45.971 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.561196+0000", 2024-04-26T03:39:45.971 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.002071107 2024-04-26T03:39:45.972 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.972 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:45.972 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.972 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.972 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.972 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:111 3.3s0 3:cad73b03:::smithi09235267-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:45.972 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.570167+0000", 2024-04-26T03:39:45.972 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 54.377873592, 2024-04-26T03:39:45.972 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.014356136, 2024-04-26T03:39:45.972 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:45.972 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:45.972 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:45.972 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:45.972 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:45.973 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:45.973 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 111 2024-04-26T03:39:45.973 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.973 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:45.973 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.973 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:45.973 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.570167+0000", 2024-04-26T03:39:45.973 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.973 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.973 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.973 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:45.973 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.570167+0000", 2024-04-26T03:39:45.973 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.974 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.974 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.974 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:45.974 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.570168+0000", 2024-04-26T03:39:45.974 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.0699999999999997e-07 2024-04-26T03:39:45.974 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.974 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.974 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:45.974 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.570170+0000", 2024-04-26T03:39:45.974 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6089999999999999e-06 2024-04-26T03:39:45.974 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.974 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.974 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:45.975 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.570178+0000", 2024-04-26T03:39:45.975 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.4449999999999999e-06 2024-04-26T03:39:45.975 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.975 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.975 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:45.975 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.570185+0000", 2024-04-26T03:39:45.975 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.9800000000000001e-06 2024-04-26T03:39:45.975 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.975 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.975 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:45.975 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.575666+0000", 2024-04-26T03:39:45.975 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0054812819999999996 2024-04-26T03:39:45.975 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.975 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.975 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:45.976 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.575751+0000", 2024-04-26T03:39:45.976 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5247999999999994e-05 2024-04-26T03:39:45.976 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.976 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.976 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:45.976 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.580678+0000", 2024-04-26T03:39:45.976 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0049265330000000003 2024-04-26T03:39:45.976 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.976 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.976 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:45.976 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.580798+0000", 2024-04-26T03:39:45.976 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000120365 2024-04-26T03:39:45.976 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.976 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.977 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:45.977 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.584523+0000", 2024-04-26T03:39:45.977 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0037251670000000001 2024-04-26T03:39:45.977 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.977 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:45.977 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.977 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.977 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.977 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.4s5 40/32 ECSubWrite(tid=22, reqid=client.4556.0:262, at_version=40'14, trim_to=0'0, roll_forward_to=35'12))", 2024-04-26T03:39:45.977 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:56.521062+0000", 2024-04-26T03:39:45.977 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 49.426978849999998, 2024-04-26T03:39:45.977 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.014879939, 2024-04-26T03:39:45.977 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:45.977 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:45.977 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:45.978 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:45.978 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.978 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:45.978 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.521062+0000", 2024-04-26T03:39:45.978 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.978 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.978 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.978 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:45.978 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.521062+0000", 2024-04-26T03:39:45.978 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.978 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.978 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.978 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:45.979 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.521062+0000", 2024-04-26T03:39:45.979 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4900000000000002e-07 2024-04-26T03:39:45.979 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.979 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.979 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:45.979 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.521346+0000", 2024-04-26T03:39:45.979 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00028391499999999998 2024-04-26T03:39:45.979 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.979 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.979 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:45.979 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.521358+0000", 2024-04-26T03:39:45.979 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1567e-05 2024-04-26T03:39:45.979 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.979 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.979 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:45.980 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.521365+0000", 2024-04-26T03:39:45.980 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.7849999999999992e-06 2024-04-26T03:39:45.980 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.980 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.980 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:45.980 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.521432+0000", 2024-04-26T03:39:45.980 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.6963000000000002e-05 2024-04-26T03:39:45.980 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.980 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.980 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:45.980 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.521442+0000", 2024-04-26T03:39:45.980 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0098000000000001e-05 2024-04-26T03:39:45.980 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.980 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.981 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:45.981 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.535904+0000", 2024-04-26T03:39:45.981 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.014461821999999999 2024-04-26T03:39:45.981 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.981 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.981 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:45.981 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:56.535942+0000", 2024-04-26T03:39:45.981 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7540000000000003e-05 2024-04-26T03:39:45.981 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.981 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:45.981 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.981 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.981 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.981 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:312 3.3s0 3:c8a9e28b:::smithi09235267-30:head [append 0~475136] snapc 6=[6,5] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-26T03:39:45.981 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:59.547999+0000", 2024-04-26T03:39:45.982 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 46.400042003999999, 2024-04-26T03:39:45.982 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.013877782, 2024-04-26T03:39:45.982 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:45.982 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:45.982 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:45.982 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:45.982 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:45.982 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:45.982 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 312 2024-04-26T03:39:45.982 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.982 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:45.982 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.983 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:45.983 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.547999+0000", 2024-04-26T03:39:45.983 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.983 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.983 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.983 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:45.983 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.547999+0000", 2024-04-26T03:39:45.983 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.983 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.983 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.983 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:45.983 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.548000+0000", 2024-04-26T03:39:45.983 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7370000000000001e-06 2024-04-26T03:39:45.983 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.983 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.984 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:45.984 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.548517+0000", 2024-04-26T03:39:45.984 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00051695699999999996 2024-04-26T03:39:45.984 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.984 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.984 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:45.984 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.548533+0000", 2024-04-26T03:39:45.984 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5985e-05 2024-04-26T03:39:45.984 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.984 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.984 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:45.984 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.548540+0000", 2024-04-26T03:39:45.984 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.0639999999999996e-06 2024-04-26T03:39:45.984 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.985 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.985 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:45.985 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.548614+0000", 2024-04-26T03:39:45.985 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3194000000000004e-05 2024-04-26T03:39:45.985 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.985 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.985 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:45.985 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.548715+0000", 2024-04-26T03:39:45.985 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000101309 2024-04-26T03:39:45.985 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.985 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.985 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:45.985 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.550003+0000", 2024-04-26T03:39:45.985 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001287722 2024-04-26T03:39:45.986 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.986 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.986 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:45.986 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.561581+0000", 2024-04-26T03:39:45.986 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.011578774999999999 2024-04-26T03:39:45.986 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.986 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.986 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:45.986 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.561791+0000", 2024-04-26T03:39:45.986 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020959900000000001 2024-04-26T03:39:45.986 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.986 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.986 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:45.986 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.561877+0000", 2024-04-26T03:39:45.987 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5439999999999995e-05 2024-04-26T03:39:45.987 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.987 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:45.987 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.987 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.987 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.987 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:314 3.3s0 3:c8a9e28b:::smithi09235267-30:head [append 0~147456] snapc 6=[6,5] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-26T03:39:45.987 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:59.552775+0000", 2024-04-26T03:39:45.987 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 46.395265727999998, 2024-04-26T03:39:45.987 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.030282514, 2024-04-26T03:39:45.987 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:45.987 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:45.987 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:45.987 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:45.988 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:45.988 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:45.988 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 314 2024-04-26T03:39:45.988 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.988 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:45.988 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.988 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:45.988 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552775+0000", 2024-04-26T03:39:45.988 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.988 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.988 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.988 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:45.988 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552775+0000", 2024-04-26T03:39:45.988 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.989 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.989 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.989 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:45.989 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552776+0000", 2024-04-26T03:39:45.989 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.026e-06 2024-04-26T03:39:45.989 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.989 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.989 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:45.989 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552935+0000", 2024-04-26T03:39:45.989 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00015929799999999999 2024-04-26T03:39:45.989 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.989 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.989 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:45.990 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552948+0000", 2024-04-26T03:39:45.990 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2716e-05 2024-04-26T03:39:45.990 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.990 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.990 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:45.990 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552955+0000", 2024-04-26T03:39:45.990 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.9609999999999999e-06 2024-04-26T03:39:45.990 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.990 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.990 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:45.990 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.562092+0000", 2024-04-26T03:39:45.990 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0091373979999999997 2024-04-26T03:39:45.991 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.991 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.991 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:45.991 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.562164+0000", 2024-04-26T03:39:45.991 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.1767999999999995e-05 2024-04-26T03:39:45.991 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.991 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.991 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:45.991 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.562591+0000", 2024-04-26T03:39:45.991 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00042723799999999998 2024-04-26T03:39:45.991 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.991 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.991 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:45.991 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.564915+0000", 2024-04-26T03:39:45.992 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0023239340000000002 2024-04-26T03:39:45.992 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.992 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.992 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:45.992 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.583009+0000", 2024-04-26T03:39:45.992 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.018093609 2024-04-26T03:39:45.992 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.992 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.992 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:45.992 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.583058+0000", 2024-04-26T03:39:45.992 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.8566000000000002e-05 2024-04-26T03:39:45.992 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.992 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:45.993 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.993 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.993 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.993 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:315 3.3s0 3:c8a9e28b:::smithi09235267-30:head [setxattr _header (56) in=7b] snapc 6=[6,5] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-26T03:39:45.993 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:59.552967+0000", 2024-04-26T03:39:45.993 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 46.395074194000003, 2024-04-26T03:39:45.993 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.030505498999999998, 2024-04-26T03:39:45.993 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:45.993 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:45.993 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:45.993 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:45.993 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:45.993 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:45.993 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 315 2024-04-26T03:39:45.993 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.994 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:45.994 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.994 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:45.994 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552967+0000", 2024-04-26T03:39:45.994 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.994 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.994 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.994 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:45.994 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552967+0000", 2024-04-26T03:39:45.994 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:45.994 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.994 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.994 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:45.995 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552967+0000", 2024-04-26T03:39:45.995 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.37e-07 2024-04-26T03:39:45.995 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.995 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.995 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:45.995 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552971+0000", 2024-04-26T03:39:45.995 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9580000000000001e-06 2024-04-26T03:39:45.995 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.995 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.995 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:45.995 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552980+0000", 2024-04-26T03:39:45.995 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.4549999999999995e-06 2024-04-26T03:39:45.995 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.995 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.995 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:45.996 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552985+0000", 2024-04-26T03:39:45.996 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.5119999999999999e-06 2024-04-26T03:39:45.996 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.996 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.996 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:45.996 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.563965+0000", 2024-04-26T03:39:45.996 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.010980177000000001 2024-04-26T03:39:45.996 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.996 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.996 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:45.996 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.564028+0000", 2024-04-26T03:39:45.996 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.2607000000000006e-05 2024-04-26T03:39:45.996 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.996 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.996 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:45.997 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.564435+0000", 2024-04-26T03:39:45.997 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00040658399999999998 2024-04-26T03:39:45.997 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.997 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.997 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:45.997 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.564918+0000", 2024-04-26T03:39:45.997 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00048333499999999999 2024-04-26T03:39:45.997 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.997 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.997 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:45.997 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.583395+0000", 2024-04-26T03:39:45.997 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.018476764999999999 2024-04-26T03:39:45.997 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.997 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.997 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:45.998 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.583472+0000", 2024-04-26T03:39:45.998 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.7368999999999997e-05 2024-04-26T03:39:45.998 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.998 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:45.998 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:45.998 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.998 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.998 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:316 3.3s0 3:c8a9e28b:::smithi09235267-30:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e43)", 2024-04-26T03:39:45.998 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:59.552993+0000", 2024-04-26T03:39:45.998 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 46.395047986000002, 2024-04-26T03:39:45.998 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.032226746000000001, 2024-04-26T03:39:45.998 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:45.999 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:45.999 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:45.999 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:45.999 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:45.999 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:45.999 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 316 2024-04-26T03:39:45.999 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:45.999 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:45.999 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:45.999 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:45.999 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552993+0000", 2024-04-26T03:39:45.999 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.000 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.000 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.000 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:46.000 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552993+0000", 2024-04-26T03:39:46.000 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.000 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.000 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.000 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:46.000 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552993+0000", 2024-04-26T03:39:46.000 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7299999999999996e-07 2024-04-26T03:39:46.000 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.000 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.000 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:46.000 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.552996+0000", 2024-04-26T03:39:46.001 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.92e-06 2024-04-26T03:39:46.001 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.001 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.001 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:46.001 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.553005+0000", 2024-04-26T03:39:46.001 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.585e-06 2024-04-26T03:39:46.001 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.001 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.001 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.001 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.553010+0000", 2024-04-26T03:39:46.001 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.0719999999999999e-06 2024-04-26T03:39:46.001 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.001 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.001 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:46.001 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.564680+0000", 2024-04-26T03:39:46.002 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.011669736 2024-04-26T03:39:46.002 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.002 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.002 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:46.002 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.564736+0000", 2024-04-26T03:39:46.002 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.6113000000000003e-05 2024-04-26T03:39:46.002 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.002 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.002 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:46.002 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.583732+0000", 2024-04-26T03:39:46.002 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.018995920999999999 2024-04-26T03:39:46.002 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.002 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.002 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:46.003 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.583823+0000", 2024-04-26T03:39:46.003 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1335999999999994e-05 2024-04-26T03:39:46.003 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.003 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.003 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:46.003 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.585220+0000", 2024-04-26T03:39:46.003 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00139649 2024-04-26T03:39:46.003 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.003 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:46.003 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.003 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.003 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.003 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.7s5 49/44 ECSubWrite(tid=83, reqid=osd.4.0:83, at_version=49'10, trim_to=0'0, roll_forward_to=48'8))", 2024-04-26T03:39:46.003 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:04.696553+0000", 2024-04-26T03:39:46.004 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 41.251488117999997, 2024-04-26T03:39:46.004 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.013590896999999999, 2024-04-26T03:39:46.004 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:46.004 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:46.004 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:46.004 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:46.004 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.004 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:46.004 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.696553+0000", 2024-04-26T03:39:46.004 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.004 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.004 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.004 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:46.004 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.696553+0000", 2024-04-26T03:39:46.005 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.005 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.005 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.005 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:46.005 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.696553+0000", 2024-04-26T03:39:46.005 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.2600000000000002e-07 2024-04-26T03:39:46.005 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.005 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.005 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:46.005 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.696575+0000", 2024-04-26T03:39:46.005 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1441e-05 2024-04-26T03:39:46.005 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.005 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.005 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:46.005 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.696587+0000", 2024-04-26T03:39:46.006 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1892000000000001e-05 2024-04-26T03:39:46.006 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.006 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.006 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.006 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.696594+0000", 2024-04-26T03:39:46.006 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8310000000000001e-06 2024-04-26T03:39:46.006 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.006 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.006 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:46.006 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.696660+0000", 2024-04-26T03:39:46.006 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5550999999999996e-05 2024-04-26T03:39:46.006 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.006 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.006 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:46.007 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.696669+0000", 2024-04-26T03:39:46.007 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1940000000000004e-06 2024-04-26T03:39:46.007 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.007 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.007 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:46.007 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.710095+0000", 2024-04-26T03:39:46.007 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.013425906 2024-04-26T03:39:46.007 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.007 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.007 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:46.007 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.710144+0000", 2024-04-26T03:39:46.007 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.8455999999999998e-05 2024-04-26T03:39:46.007 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.007 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:46.008 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.008 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.008 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.008 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.2s5 49/44 ECSubWrite(tid=32, reqid=client.4556.0:433, at_version=49'20, trim_to=0'0, roll_forward_to=48'19))", 2024-04-26T03:39:46.008 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:04.700213+0000", 2024-04-26T03:39:46.008 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 41.247827370000003, 2024-04-26T03:39:46.008 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.018834018000000001, 2024-04-26T03:39:46.008 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:46.008 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:46.008 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:46.008 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:46.008 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.008 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:46.008 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700213+0000", 2024-04-26T03:39:46.009 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.009 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.009 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.009 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:46.009 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700213+0000", 2024-04-26T03:39:46.009 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.009 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.009 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.009 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:46.009 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700214+0000", 2024-04-26T03:39:46.009 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.48e-07 2024-04-26T03:39:46.009 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.009 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.009 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:46.010 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700972+0000", 2024-04-26T03:39:46.010 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00075845299999999997 2024-04-26T03:39:46.010 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.010 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.010 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:46.010 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700983+0000", 2024-04-26T03:39:46.010 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1105e-05 2024-04-26T03:39:46.010 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.010 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.010 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.010 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700991+0000", 2024-04-26T03:39:46.010 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9140000000000008e-06 2024-04-26T03:39:46.010 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.010 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.011 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:46.011 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.701100+0000", 2024-04-26T03:39:46.011 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000108915 2024-04-26T03:39:46.011 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.011 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.011 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:46.011 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.701120+0000", 2024-04-26T03:39:46.011 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9811000000000001e-05 2024-04-26T03:39:46.011 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.011 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.011 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:46.011 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.718957+0000", 2024-04-26T03:39:46.011 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.017837604 2024-04-26T03:39:46.011 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.012 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.012 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:46.012 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.719047+0000", 2024-04-26T03:39:46.012 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.9968000000000003e-05 2024-04-26T03:39:46.012 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.012 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:46.012 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.012 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.012 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.012 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.bs2 50/32 ECSubWrite(tid=88, reqid=osd.4.0:88, at_version=50'48, trim_to=0'0, roll_forward_to=49'46))", 2024-04-26T03:39:46.012 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:05.698893+0000", 2024-04-26T03:39:46.012 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 40.249147430000001, 2024-04-26T03:39:46.012 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.015901432, 2024-04-26T03:39:46.013 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:46.013 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:46.013 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:46.013 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:46.013 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.013 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:46.013 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.698893+0000", 2024-04-26T03:39:46.013 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.013 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.013 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.013 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:46.013 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.698893+0000", 2024-04-26T03:39:46.013 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.013 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.014 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.014 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:46.014 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.698894+0000", 2024-04-26T03:39:46.014 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.1e-07 2024-04-26T03:39:46.014 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.014 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.014 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:46.014 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.698914+0000", 2024-04-26T03:39:46.014 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0613000000000001e-05 2024-04-26T03:39:46.014 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.014 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.014 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:46.014 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.698924+0000", 2024-04-26T03:39:46.014 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.3859999999999996e-06 2024-04-26T03:39:46.015 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.015 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.015 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.015 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.698931+0000", 2024-04-26T03:39:46.015 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.0829999999999998e-06 2024-04-26T03:39:46.015 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.015 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.015 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:46.015 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.699137+0000", 2024-04-26T03:39:46.015 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020594 2024-04-26T03:39:46.015 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.015 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.015 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:46.016 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.699153+0000", 2024-04-26T03:39:46.016 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6218999999999998e-05 2024-04-26T03:39:46.016 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.016 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.016 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:46.016 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.714755+0000", 2024-04-26T03:39:46.016 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.015602517 2024-04-26T03:39:46.016 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.016 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.016 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:46.016 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.714795+0000", 2024-04-26T03:39:46.016 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9363999999999998e-05 2024-04-26T03:39:46.016 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.016 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:46.017 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.017 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.017 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.017 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:446 3.13s0 3:c944ba3d:::smithi09235267-6:head [append 0~720896] snapc 8=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:46.017 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:05.700096+0000", 2024-04-26T03:39:46.017 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 40.247944849, 2024-04-26T03:39:46.017 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.019772888999999998, 2024-04-26T03:39:46.017 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:46.017 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:46.017 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:46.017 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:46.017 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:46.017 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:46.017 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 446 2024-04-26T03:39:46.017 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.018 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:46.018 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.018 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:46.018 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.700096+0000", 2024-04-26T03:39:46.018 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.018 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.018 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.018 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:46.018 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.700096+0000", 2024-04-26T03:39:46.018 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.018 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.018 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.018 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:46.018 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.700098+0000", 2024-04-26T03:39:46.019 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.464e-06 2024-04-26T03:39:46.019 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.019 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.019 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:46.019 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.700859+0000", 2024-04-26T03:39:46.019 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00076090299999999995 2024-04-26T03:39:46.019 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.019 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.019 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:46.019 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.700875+0000", 2024-04-26T03:39:46.019 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5299e-05 2024-04-26T03:39:46.019 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.019 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.019 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.019 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.700881+0000", 2024-04-26T03:39:46.020 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.528e-06 2024-04-26T03:39:46.020 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.020 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.020 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:46.020 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.700945+0000", 2024-04-26T03:39:46.020 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.3744999999999994e-05 2024-04-26T03:39:46.020 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.020 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.020 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:46.020 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.701071+0000", 2024-04-26T03:39:46.020 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012569600000000001 2024-04-26T03:39:46.020 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.020 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.021 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:46.021 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.702209+0000", 2024-04-26T03:39:46.021 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001138266 2024-04-26T03:39:46.021 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.021 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.021 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:46.021 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.714847+0000", 2024-04-26T03:39:46.021 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.012638508 2024-04-26T03:39:46.021 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.021 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.021 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:46.021 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.719810+0000", 2024-04-26T03:39:46.021 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0049625989999999998 2024-04-26T03:39:46.021 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.021 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.022 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:46.022 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.719869+0000", 2024-04-26T03:39:46.022 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.8881000000000001e-05 2024-04-26T03:39:46.022 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.022 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:46.022 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.022 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.022 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.022 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:447 3.13s0 3:c944ba3d:::smithi09235267-6:head [append 0~442368] snapc 8=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:46.022 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:05.702913+0000", 2024-04-26T03:39:46.022 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 40.245128147999999, 2024-04-26T03:39:46.022 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.01837453, 2024-04-26T03:39:46.022 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:46.022 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:46.023 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:46.023 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:46.023 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:46.023 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:46.023 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 447 2024-04-26T03:39:46.023 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.023 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:46.023 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.023 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:46.023 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.702913+0000", 2024-04-26T03:39:46.023 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.023 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.023 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.023 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:46.024 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.702913+0000", 2024-04-26T03:39:46.024 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.024 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.024 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.024 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:46.024 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.702915+0000", 2024-04-26T03:39:46.024 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0360000000000001e-06 2024-04-26T03:39:46.024 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.024 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.024 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:46.024 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.703366+0000", 2024-04-26T03:39:46.024 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00045175800000000002 2024-04-26T03:39:46.024 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.024 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.024 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:46.025 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.703382+0000", 2024-04-26T03:39:46.025 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5707999999999999e-05 2024-04-26T03:39:46.025 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.025 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.025 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.025 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.703391+0000", 2024-04-26T03:39:46.025 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.3889999999999994e-06 2024-04-26T03:39:46.025 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.025 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.025 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:46.025 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.710509+0000", 2024-04-26T03:39:46.025 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0071172650000000002 2024-04-26T03:39:46.025 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.026 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.026 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:46.026 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.710619+0000", 2024-04-26T03:39:46.026 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00011061 2024-04-26T03:39:46.026 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.026 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.026 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:46.026 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.711319+0000", 2024-04-26T03:39:46.026 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00070012799999999995 2024-04-26T03:39:46.026 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.026 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.026 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:46.026 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.719699+0000", 2024-04-26T03:39:46.027 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0083798540000000008 2024-04-26T03:39:46.027 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.027 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.027 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:46.027 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.721236+0000", 2024-04-26T03:39:46.027 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0015364129999999999 2024-04-26T03:39:46.027 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.027 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.027 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:46.027 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.721287+0000", 2024-04-26T03:39:46.027 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.1369000000000002e-05 2024-04-26T03:39:46.027 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.027 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:46.027 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.027 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.028 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.028 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:448 3.13s0 3:c944ba3d:::smithi09235267-6:head [append 0~327680] snapc 8=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:46.028 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:05.704961+0000", 2024-04-26T03:39:46.028 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 40.243079809000001, 2024-04-26T03:39:46.028 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.019901015000000001, 2024-04-26T03:39:46.028 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:46.028 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:46.028 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:46.028 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:46.028 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:46.028 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:46.028 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 448 2024-04-26T03:39:46.028 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.029 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:46.029 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.029 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:46.029 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.704961+0000", 2024-04-26T03:39:46.029 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.029 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.029 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.029 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:46.029 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.704961+0000", 2024-04-26T03:39:46.029 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.029 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.029 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.029 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:46.029 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.704963+0000", 2024-04-26T03:39:46.030 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.81e-06 2024-04-26T03:39:46.030 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.030 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.030 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:46.030 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.705283+0000", 2024-04-26T03:39:46.030 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00032049100000000001 2024-04-26T03:39:46.030 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.030 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.030 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:46.030 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.705309+0000", 2024-04-26T03:39:46.030 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5789999999999999e-05 2024-04-26T03:39:46.030 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.031 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.031 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.031 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.705352+0000", 2024-04-26T03:39:46.031 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.2865999999999999e-05 2024-04-26T03:39:46.031 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.031 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.031 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:46.031 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.714894+0000", 2024-04-26T03:39:46.031 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.009542109 2024-04-26T03:39:46.031 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.031 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.031 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:46.031 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.715004+0000", 2024-04-26T03:39:46.031 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00010983400000000001 2024-04-26T03:39:46.032 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.032 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.032 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:46.032 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.715894+0000", 2024-04-26T03:39:46.032 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00089006399999999998 2024-04-26T03:39:46.032 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.032 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.032 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:46.032 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.720882+0000", 2024-04-26T03:39:46.032 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0049879499999999997 2024-04-26T03:39:46.032 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.032 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.032 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:46.032 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.724816+0000", 2024-04-26T03:39:46.033 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0039342830000000002 2024-04-26T03:39:46.033 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.033 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.033 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:46.033 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.724862+0000", 2024-04-26T03:39:46.033 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.5818000000000001e-05 2024-04-26T03:39:46.033 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.033 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:46.033 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.033 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.033 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.033 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:449 3.13s0 3:c944ba3d:::smithi09235267-6:head [setxattr _header (55) in=7b] snapc 8=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:46.033 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:05.705415+0000", 2024-04-26T03:39:46.034 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 40.242625588999999, 2024-04-26T03:39:46.034 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.021062219, 2024-04-26T03:39:46.034 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:46.034 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:46.034 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:46.034 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:46.034 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:46.034 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:46.034 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 449 2024-04-26T03:39:46.034 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.034 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:46.034 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.034 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:46.035 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.705415+0000", 2024-04-26T03:39:46.035 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.035 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.035 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.035 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:46.035 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.705415+0000", 2024-04-26T03:39:46.035 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.035 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.035 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.035 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:46.035 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.705416+0000", 2024-04-26T03:39:46.035 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5099999999999998e-07 2024-04-26T03:39:46.035 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.035 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.035 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:46.036 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.705422+0000", 2024-04-26T03:39:46.036 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.5079999999999996e-06 2024-04-26T03:39:46.036 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.036 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.036 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:46.036 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.705507+0000", 2024-04-26T03:39:46.036 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5282000000000001e-05 2024-04-26T03:39:46.036 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.036 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.036 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.036 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.705525+0000", 2024-04-26T03:39:46.036 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8173999999999999e-05 2024-04-26T03:39:46.036 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.036 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.036 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:46.037 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.720102+0000", 2024-04-26T03:39:46.037 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.014577345 2024-04-26T03:39:46.037 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.037 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.037 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:46.037 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.720161+0000", 2024-04-26T03:39:46.037 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.9070000000000002e-05 2024-04-26T03:39:46.037 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.037 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.037 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:46.037 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.720462+0000", 2024-04-26T03:39:46.037 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00030043300000000002 2024-04-26T03:39:46.037 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.037 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.038 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:46.038 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.722295+0000", 2024-04-26T03:39:46.038 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001833339 2024-04-26T03:39:46.038 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.038 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.038 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:46.038 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.726405+0000", 2024-04-26T03:39:46.038 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0041094959999999998 2024-04-26T03:39:46.038 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.038 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.038 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:46.038 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.726477+0000", 2024-04-26T03:39:46.038 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.2720999999999997e-05 2024-04-26T03:39:46.039 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.039 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:46.039 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.039 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.039 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.039 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:450 3.13s0 3:c944ba3d:::smithi09235267-6:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:46.039 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:05.705540+0000", 2024-04-26T03:39:46.039 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 40.242500821, 2024-04-26T03:39:46.039 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.022180959, 2024-04-26T03:39:46.039 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:46.039 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:46.039 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:46.039 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:46.039 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:46.040 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:46.040 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 450 2024-04-26T03:39:46.040 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.040 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:46.040 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.040 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:46.040 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.705540+0000", 2024-04-26T03:39:46.040 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.040 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.040 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.040 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:46.040 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.705540+0000", 2024-04-26T03:39:46.040 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.040 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.041 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.041 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:46.041 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.705541+0000", 2024-04-26T03:39:46.041 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.6000000000000003e-07 2024-04-26T03:39:46.041 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.041 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.041 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:46.041 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.705655+0000", 2024-04-26T03:39:46.041 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000114528 2024-04-26T03:39:46.041 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.041 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.041 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:46.041 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.705668+0000", 2024-04-26T03:39:46.041 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2712e-05 2024-04-26T03:39:46.041 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.042 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.042 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.042 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.705676+0000", 2024-04-26T03:39:46.042 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9179999999999994e-06 2024-04-26T03:39:46.042 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.042 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.042 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:46.042 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.720674+0000", 2024-04-26T03:39:46.042 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.014998177 2024-04-26T03:39:46.042 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.042 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.042 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:46.042 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.720807+0000", 2024-04-26T03:39:46.043 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000133053 2024-04-26T03:39:46.043 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.043 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.043 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:46.043 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.726766+0000", 2024-04-26T03:39:46.043 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0059592270000000001 2024-04-26T03:39:46.043 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.043 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.043 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:46.043 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.726868+0000", 2024-04-26T03:39:46.043 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000101931 2024-04-26T03:39:46.043 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.043 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.043 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:46.044 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.727721+0000", 2024-04-26T03:39:46.044 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00085265299999999998 2024-04-26T03:39:46.044 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.044 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:46.044 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.044 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.044 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.044 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:902 3.13s0 3:cad73b03:::smithi09235267-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e80)", 2024-04-26T03:39:46.044 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:35.699236+0000", 2024-04-26T03:39:46.044 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 10.248804663, 2024-04-26T03:39:46.044 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.015128347, 2024-04-26T03:39:46.044 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:46.045 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:46.045 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:46.045 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:46.045 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:46.045 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:46.045 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 902 2024-04-26T03:39:46.045 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.045 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:46.045 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.045 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:46.045 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.699236+0000", 2024-04-26T03:39:46.045 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.045 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.045 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.046 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:46.046 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.699236+0000", 2024-04-26T03:39:46.046 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.046 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.046 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.046 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:46.046 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.699237+0000", 2024-04-26T03:39:46.046 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.2900000000000002e-07 2024-04-26T03:39:46.046 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.046 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.046 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:46.047 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.699242+0000", 2024-04-26T03:39:46.047 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.8720000000000001e-06 2024-04-26T03:39:46.047 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.047 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.047 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:46.047 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.699254+0000", 2024-04-26T03:39:46.047 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2517000000000001e-05 2024-04-26T03:39:46.047 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.047 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.047 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.047 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.699263+0000", 2024-04-26T03:39:46.047 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.9600000000000006e-06 2024-04-26T03:39:46.048 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.048 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.048 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:46.048 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.705439+0000", 2024-04-26T03:39:46.048 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0061754339999999996 2024-04-26T03:39:46.048 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.048 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.048 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:46.048 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.705535+0000", 2024-04-26T03:39:46.048 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.6512e-05 2024-04-26T03:39:46.048 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.048 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.048 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:46.049 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.711895+0000", 2024-04-26T03:39:46.049 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.006359417 2024-04-26T03:39:46.049 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.049 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.049 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:46.049 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.712005+0000", 2024-04-26T03:39:46.049 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000110191 2024-04-26T03:39:46.049 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.049 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.049 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:46.049 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.714364+0000", 2024-04-26T03:39:46.049 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.002359515 2024-04-26T03:39:46.049 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.049 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:46.050 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.050 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.050 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.050 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.8s3 80/63 ECSubWrite(tid=321, reqid=client.4556.0:897, at_version=80'111, trim_to=0'0, roll_forward_to=80'110))", 2024-04-26T03:39:46.050 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:35.716653+0000", 2024-04-26T03:39:46.050 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 10.231387889000001, 2024-04-26T03:39:46.050 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.019288957999999998, 2024-04-26T03:39:46.050 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:46.050 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:46.050 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:46.050 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:46.050 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.050 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:46.050 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.716653+0000", 2024-04-26T03:39:46.051 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.051 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.051 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.051 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:46.051 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.716653+0000", 2024-04-26T03:39:46.051 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.051 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.051 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.051 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:46.051 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.716654+0000", 2024-04-26T03:39:46.051 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3370000000000001e-06 2024-04-26T03:39:46.051 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.051 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.051 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:46.051 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.717078+0000", 2024-04-26T03:39:46.052 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00042400100000000002 2024-04-26T03:39:46.052 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.052 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.052 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:46.052 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.717093+0000", 2024-04-26T03:39:46.052 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4695000000000001e-05 2024-04-26T03:39:46.052 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.052 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.052 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.052 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.717104+0000", 2024-04-26T03:39:46.052 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0852999999999999e-05 2024-04-26T03:39:46.052 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.052 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.052 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:46.053 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.717206+0000", 2024-04-26T03:39:46.053 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000102675 2024-04-26T03:39:46.053 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.053 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.053 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:46.053 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.717222+0000", 2024-04-26T03:39:46.053 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.523e-05 2024-04-26T03:39:46.053 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.053 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.053 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:46.053 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.735899+0000", 2024-04-26T03:39:46.053 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.018676944000000001 2024-04-26T03:39:46.053 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.053 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.053 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:46.054 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:35.735942+0000", 2024-04-26T03:39:46.054 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.3223000000000003e-05 2024-04-26T03:39:46.054 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.054 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:46.054 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.054 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.054 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:46.054 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:39:46.056 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-04-26T03:39:46.056 INFO:tasks.ceph.osd.15.smithi092.stderr:2024-04-26T03:39:46.054+0000 7f6d12dec640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 31824) UID: 0 2024-04-26T03:39:46.059 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:39:46.059 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":90,"fsid":"c5a13c99-5c9a-4b8b-a893-4f6f4211677a","created":"2024-04-26T03:38:19.896648+0000","modified":"2024-04-26T03:39:45.690314+0000","last_up_change":"2024-04-26T03:38:33.308561+0000","last_in_change":"2024-04-26T03:38:24.150336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T03:38:32.306696+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-26T03:38:34.592633+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.75,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"unique_pool_0","create_time":"2024-04-26T03:38:47.497176+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":3,"size":7,"min_size":5,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":18,"pg_placement_num":18,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":18,"last_pg_merge_meta":{"source_pgid":"3.12","ready_epoch":87,"last_epoch_started":86,"last_epoch_clean":86,"source_version":"35'11","target_version":"81'65"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"87","last_force_op_resend_preluminous":"85","auid":0,"snap_mode":"selfmanaged","snap_seq":32,"snap_epoch":90,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"shecprofile","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":16384,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}}}],"osds":[{"osd":0,"uuid":"31b0e7a9-455d-4874-9c60-26efac8d0b1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2377656090},{"type":"v1","addr":"172.21.15.28:6801","nonce":2377656090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2377656090},{"type":"v1","addr":"172.21.15.28:6803","nonce":2377656090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2377656090},{"type":"v1","addr":"172.21.15.28:6807","nonce":2377656090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2377656090},{"type":"v1","addr":"172.21.15.28:6805","nonce":2377656090}]},"public_addr":"172.21.15.28:6801/2377656090","cluster_addr":"172.21.15.28:6803/2377656090","heartbeat_back_addr":"172.21.15.28:6807/2377656090","heartbeat_front_addr":"172.21.15.28:6805/2377656090","state":["exists","up"]},{"osd":1,"uuid":"52819ace-a070-4822-b33e-3c54466e61ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":1678150206},{"type":"v1","addr":"172.21.15.42:6801","nonce":1678150206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1678150206},{"type":"v1","addr":"172.21.15.42:6803","nonce":1678150206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1678150206},{"type":"v1","addr":"172.21.15.42:6807","nonce":1678150206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1678150206},{"type":"v1","addr":"172.21.15.42:6805","nonce":1678150206}]},"public_addr":"172.21.15.42:6801/1678150206","cluster_addr":"172.21.15.42:6803/1678150206","heartbeat_back_addr":"172.21.15.42:6807/1678150206","heartbeat_front_addr":"172.21.15.42:6805/1678150206","state":["exists","up"]},{"osd":2,"uuid":"858f8ac1-2edc-43d5-914c-c1914bdc0c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":714535415},{"type":"v1","addr":"172.21.15.66:6801","nonce":714535415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":714535415},{"type":"v1","addr":"172.21.15.66:6803","nonce":714535415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":714535415},{"type":"v1","addr":"172.21.15.66:6807","nonce":714535415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":714535415},{"type":"v1","addr":"172.21.15.66:6805","nonce":714535415}]},"public_addr":"172.21.15.66:6801/714535415","cluster_addr":"172.21.15.66:6803/714535415","heartbeat_back_addr":"172.21.15.66:6807/714535415","heartbeat_front_addr":"172.21.15.66:6805/714535415","state":["exists","up"]},{"osd":3,"uuid":"531a2e30-3351-4f84-804b-5fb5050b84ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2377983625},{"type":"v1","addr":"172.21.15.92:6811","nonce":2377983625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2377983625},{"type":"v1","addr":"172.21.15.92:6813","nonce":2377983625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2377983625},{"type":"v1","addr":"172.21.15.92:6817","nonce":2377983625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2377983625},{"type":"v1","addr":"172.21.15.92:6815","nonce":2377983625}]},"public_addr":"172.21.15.92:6811/2377983625","cluster_addr":"172.21.15.92:6813/2377983625","heartbeat_back_addr":"172.21.15.92:6817/2377983625","heartbeat_front_addr":"172.21.15.92:6815/2377983625","state":["exists","up"]},{"osd":4,"uuid":"6d38ffda-40eb-4fd7-9300-2a09f2673e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":183752455},{"type":"v1","addr":"172.21.15.28:6809","nonce":183752455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":183752455},{"type":"v1","addr":"172.21.15.28:6811","nonce":183752455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":183752455},{"type":"v1","addr":"172.21.15.28:6815","nonce":183752455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":183752455},{"type":"v1","addr":"172.21.15.28:6813","nonce":183752455}]},"public_addr":"172.21.15.28:6809/183752455","cluster_addr":"172.21.15.28:6811/183752455","heartbeat_back_addr":"172.21.15.28:6815/183752455","heartbeat_front_addr":"172.21.15.28:6813/183752455","state":["exists","up"]},{"osd":5,"uuid":"361aa40d-e53b-42af-afd4-81fbcac28725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2012261939},{"type":"v1","addr":"172.21.15.42:6809","nonce":2012261939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2012261939},{"type":"v1","addr":"172.21.15.42:6811","nonce":2012261939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2012261939},{"type":"v1","addr":"172.21.15.42:6815","nonce":2012261939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2012261939},{"type":"v1","addr":"172.21.15.42:6813","nonce":2012261939}]},"public_addr":"172.21.15.42:6809/2012261939","cluster_addr":"172.21.15.42:6811/2012261939","heartbeat_back_addr":"172.21.15.42:6815/2012261939","heartbeat_front_addr":"172.21.15.42:6813/2012261939","state":["exists","up"]},{"osd":6,"uuid":"f610ad3e-1fd8-42bd-af4b-0d79c3bb9cf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":3533460882},{"type":"v1","addr":"172.21.15.66:6809","nonce":3533460882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":3533460882},{"type":"v1","addr":"172.21.15.66:6811","nonce":3533460882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":3533460882},{"type":"v1","addr":"172.21.15.66:6815","nonce":3533460882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":3533460882},{"type":"v1","addr":"172.21.15.66:6813","nonce":3533460882}]},"public_addr":"172.21.15.66:6809/3533460882","cluster_addr":"172.21.15.66:6811/3533460882","heartbeat_back_addr":"172.21.15.66:6815/3533460882","heartbeat_front_addr":"172.21.15.66:6813/3533460882","state":["exists","up"]},{"osd":7,"uuid":"a4bae6fb-e087-4563-a129-6938e18eed8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":1760163933},{"type":"v1","addr":"172.21.15.92:6803","nonce":1760163933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":1760163933},{"type":"v1","addr":"172.21.15.92:6805","nonce":1760163933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1760163933},{"type":"v1","addr":"172.21.15.92:6809","nonce":1760163933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":1760163933},{"type":"v1","addr":"172.21.15.92:6807","nonce":1760163933}]},"public_addr":"172.21.15.92:6803/1760163933","cluster_addr":"172.21.15.92:6805/1760163933","heartbeat_back_addr":"172.21.15.92:6809/1760163933","heartbeat_front_addr":"172.21.15.92:6807/1760163933","state":["exists","up"]},{"osd":8,"uuid":"479f3885-452e-4231-bdc2-b79668384ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":4127602407},{"type":"v1","addr":"172.21.15.28:6817","nonce":4127602407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":4127602407},{"type":"v1","addr":"172.21.15.28:6819","nonce":4127602407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":4127602407},{"type":"v1","addr":"172.21.15.28:6823","nonce":4127602407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":4127602407},{"type":"v1","addr":"172.21.15.28:6821","nonce":4127602407}]},"public_addr":"172.21.15.28:6817/4127602407","cluster_addr":"172.21.15.28:6819/4127602407","heartbeat_back_addr":"172.21.15.28:6823/4127602407","heartbeat_front_addr":"172.21.15.28:6821/4127602407","state":["exists","up"]},{"osd":9,"uuid":"9a603a47-7c5a-47d4-a298-56b3ac991892","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":4168729845},{"type":"v1","addr":"172.21.15.42:6818","nonce":4168729845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":4168729845},{"type":"v1","addr":"172.21.15.42:6822","nonce":4168729845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":4168729845},{"type":"v1","addr":"172.21.15.42:6830","nonce":4168729845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":4168729845},{"type":"v1","addr":"172.21.15.42:6826","nonce":4168729845}]},"public_addr":"172.21.15.42:6818/4168729845","cluster_addr":"172.21.15.42:6822/4168729845","heartbeat_back_addr":"172.21.15.42:6830/4168729845","heartbeat_front_addr":"172.21.15.42:6826/4168729845","state":["exists","up"]},{"osd":10,"uuid":"1cd42945-6ee1-49f0-b17b-4093dd36b4ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4180572718},{"type":"v1","addr":"172.21.15.66:6817","nonce":4180572718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":4180572718},{"type":"v1","addr":"172.21.15.66:6819","nonce":4180572718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":4180572718},{"type":"v1","addr":"172.21.15.66:6823","nonce":4180572718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":4180572718},{"type":"v1","addr":"172.21.15.66:6821","nonce":4180572718}]},"public_addr":"172.21.15.66:6817/4180572718","cluster_addr":"172.21.15.66:6819/4180572718","heartbeat_back_addr":"172.21.15.66:6823/4180572718","heartbeat_front_addr":"172.21.15.66:6821/4180572718","state":["exists","up"]},{"osd":11,"uuid":"b41e8100-1d4f-4b7a-991a-7a27bc29feec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":295713815},{"type":"v1","addr":"172.21.15.92:6819","nonce":295713815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":295713815},{"type":"v1","addr":"172.21.15.92:6821","nonce":295713815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":295713815},{"type":"v1","addr":"172.21.15.92:6825","nonce":295713815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":295713815},{"type":"v1","addr":"172.21.15.92:6823","nonce":295713815}]},"public_addr":"172.21.15.92:6819/295713815","cluster_addr":"172.21.15.92:6821/295713815","heartbeat_back_addr":"172.21.15.92:6825/295713815","heartbeat_front_addr":"172.21.15.92:6823/295713815","state":["exists","up"]},{"osd":12,"uuid":"458cf150-32e1-449e-b533-ac6ab5226f85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":2081141953},{"type":"v1","addr":"172.21.15.28:6825","nonce":2081141953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2081141953},{"type":"v1","addr":"172.21.15.28:6827","nonce":2081141953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2081141953},{"type":"v1","addr":"172.21.15.28:6831","nonce":2081141953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":2081141953},{"type":"v1","addr":"172.21.15.28:6829","nonce":2081141953}]},"public_addr":"172.21.15.28:6825/2081141953","cluster_addr":"172.21.15.28:6827/2081141953","heartbeat_back_addr":"172.21.15.28:6831/2081141953","heartbeat_front_addr":"172.21.15.28:6829/2081141953","state":["exists","up"]},{"osd":13,"uuid":"f008462e-a735-4845-9ecf-dd7d4a6941af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6817","nonce":301360893},{"type":"v1","addr":"172.21.15.42:6819","nonce":301360893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6821","nonce":301360893},{"type":"v1","addr":"172.21.15.42:6823","nonce":301360893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6829","nonce":301360893},{"type":"v1","addr":"172.21.15.42:6831","nonce":301360893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6825","nonce":301360893},{"type":"v1","addr":"172.21.15.42:6827","nonce":301360893}]},"public_addr":"172.21.15.42:6819/301360893","cluster_addr":"172.21.15.42:6823/301360893","heartbeat_back_addr":"172.21.15.42:6831/301360893","heartbeat_front_addr":"172.21.15.42:6827/301360893","state":["exists","up"]},{"osd":14,"uuid":"6f66ae97-eb8a-48eb-9bcc-20f65fbf690b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":1534905602},{"type":"v1","addr":"172.21.15.66:6825","nonce":1534905602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1534905602},{"type":"v1","addr":"172.21.15.66:6827","nonce":1534905602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1534905602},{"type":"v1","addr":"172.21.15.66:6831","nonce":1534905602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1534905602},{"type":"v1","addr":"172.21.15.66:6829","nonce":1534905602}]},"public_addr":"172.21.15.66:6825/1534905602","cluster_addr":"172.21.15.66:6827/1534905602","heartbeat_back_addr":"172.21.15.66:6831/1534905602","heartbeat_front_addr":"172.21.15.66:6829/1534905602","state":["exists","up"]},{"osd":15,"uuid":"b856bfc8-253b-4112-be5c-f67d61d01768","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3982634323},{"type":"v1","addr":"172.21.15.92:6827","nonce":3982634323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3982634323},{"type":"v1","addr":"172.21.15.92:6829","nonce":3982634323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3982634323},{"type":"v1","addr":"172.21.15.92:6833","nonce":3982634323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3982634323},{"type":"v1","addr":"172.21.15.92:6831","nonce":3982634323}]},"public_addr":"172.21.15.92:6827/3982634323","cluster_addr":"172.21.15.92:6829/3982634323","heartbeat_back_addr":"172.21.15.92:6833/3982634323","heartbeat_front_addr":"172.21.15.92:6831/3982634323","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:40.180887+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:34.060445+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:35.302133+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:34.066992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.213076+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:33.955241+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:40.990020+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.329538+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:32.243723+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.334962+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.233032+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:35.430680+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:32.221433+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:34.790817+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:38:29.352515+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:36.409075+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.2","mappings":[{"from":3,"to":11}]},{"pgid":"3.f","mappings":[{"from":2,"to":9},{"from":5,"to":14}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"shecprofile":{"c":"2","crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","k":"4","m":"3","name":"shecprofile","plugin":"shec","technique":"multiple"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":25,"length":1},{"begin":28,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T03:39:46.071 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T03:39:46.156 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-04-26T03:39:46.155+0000 7fb78c544640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31956) UID: 0 2024-04-26T03:39:46.163 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:39:46.163 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-04-26T03:39:46.163 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 0 2024-04-26T03:39:46.163 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:39:46.171 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-04-26T03:39:46.258 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-04-26T03:39:46.256+0000 7f745118d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 31821) UID: 0 2024-04-26T03:39:46.278 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:39:46.278 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-04-26T03:39:46.278 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-04-26T03:39:46.278 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-04-26T03:39:46.279 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:39:46.287 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-04-26T03:39:46.359 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T03:39:46.358+0000 7fba0d2ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31961) UID: 0 2024-04-26T03:39:46.367 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:39:46.368 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-04-26T03:39:46.380 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2024-04-26T03:39:46.064074+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":133644832,"num_objects":87,"num_object_clones":41,"num_object_copies":599,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":7,"num_read":472,"num_read_kb":236812,"num_write":930,"num_write_kb":126948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":31670272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":971,"ondisk_log_size":971,"up":144,"acting":144,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":151,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":773880,"kb_used_data":237448,"kb_used_omap":24,"kb_used_meta":536167,"kb_avail":1509175560,"statfs":{"total":1546188226560,"available":1545395773440,"internally_reserved":0,"allocated":243146752,"data_stored":318532779,"data_compressed":2054457,"data_compressed_allocated":4194304,"data_compressed_original":9281536,"omap_allocated":25522,"internal_metadata":549035086},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27279360,"num_objects":17,"num_object_clones":12,"num_object_copies":119,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":40,"num_read_kb":39907,"num_write":74,"num_write_kb":16448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":25575424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000814"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":128,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.811530+0000","last_change":"2024-04-26T03:38:37.359252+0000","last_active":"2024-04-26T03:39:41.811530+0000","last_peered":"2024-04-26T03:39:41.811530+0000","last_clean":"2024-04-26T03:39:41.811530+0000","last_became_active":"2024-04-26T03:38:37.358910+0000","last_became_peered":"2024-04-26T03:38:37.358910+0000","last_unstale":"2024-04-26T03:39:41.811530+0000","last_undegraded":"2024-04-26T03:39:41.811530+0000","last_fullsized":"2024-04-26T03:39:41.811530+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:04.186001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"3.6","version":"81'20","reported_seq":304,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T03:39:43.074998+0000","last_change":"2024-04-26T03:39:39.705400+0000","last_active":"2024-04-26T03:39:43.074998+0000","last_peered":"2024-04-26T03:39:43.074998+0000","last_clean":"2024-04-26T03:39:43.074998+0000","last_became_active":"2024-04-26T03:39:27.699531+0000","last_became_peered":"2024-04-26T03:39:27.699531+0000","last_unstale":"2024-04-26T03:39:43.074998+0000","last_undegraded":"2024-04-26T03:39:43.074998+0000","last_fullsized":"2024-04-26T03:39:43.074998+0000","mapping_epoch":32,"log_start":"64'11","ondisk_log_start":"64'11","created":32,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'9","last_scrub_stamp":"2024-04-26T03:39:07.177824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:07.177824+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":11,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:24.874885+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0016197290000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,14,6,2,12,7,1],"acting":[0,14,6,2,12,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":128,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.822761+0000","last_change":"2024-04-26T03:38:36.484782+0000","last_active":"2024-04-26T03:39:41.822761+0000","last_peered":"2024-04-26T03:39:41.822761+0000","last_clean":"2024-04-26T03:39:41.822761+0000","last_became_active":"2024-04-26T03:38:36.484388+0000","last_became_peered":"2024-04-26T03:38:36.484388+0000","last_unstale":"2024-04-26T03:39:41.822761+0000","last_undegraded":"2024-04-26T03:39:41.822761+0000","last_fullsized":"2024-04-26T03:39:41.822761+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,13],"acting":[8,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.7","version":"49'10","reported_seq":299,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T03:39:42.815292+0000","last_change":"2024-04-26T03:39:39.696366+0000","last_active":"2024-04-26T03:39:42.815292+0000","last_peered":"2024-04-26T03:39:42.815292+0000","last_clean":"2024-04-26T03:39:42.815292+0000","last_became_active":"2024-04-26T03:39:23.698262+0000","last_became_peered":"2024-04-26T03:39:23.698262+0000","last_unstale":"2024-04-26T03:39:42.815292+0000","last_undegraded":"2024-04-26T03:39:42.815292+0000","last_fullsized":"2024-04-26T03:39:42.815292+0000","mapping_epoch":32,"log_start":"49'10","ondisk_log_start":"49'10","created":32,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'10","last_scrub_stamp":"2024-04-26T03:39:09.248660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:09.248660+0000","objects_scrubbed":1,"log_size":0,"log_dups_size":10,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:38.596496+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.00066948500000000005,"stat_sum":{"num_bytes":131072,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":385,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,13,6,0,14,10,11],"acting":[4,13,6,0,14,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":127,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.811971+0000","last_change":"2024-04-26T03:38:38.187714+0000","last_active":"2024-04-26T03:39:41.811971+0000","last_peered":"2024-04-26T03:39:41.811971+0000","last_clean":"2024-04-26T03:39:41.811971+0000","last_became_active":"2024-04-26T03:38:38.187383+0000","last_became_peered":"2024-04-26T03:38:38.187383+0000","last_unstale":"2024-04-26T03:39:41.811971+0000","last_undegraded":"2024-04-26T03:39:41.811971+0000","last_fullsized":"2024-04-26T03:39:41.811971+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:59.062623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4],"acting":[9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.4","version":"85'70","reported_seq":319,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.822867+0000","last_change":"2024-04-26T03:39:39.701091+0000","last_active":"2024-04-26T03:39:41.822867+0000","last_peered":"2024-04-26T03:39:41.822867+0000","last_clean":"2024-04-26T03:39:41.822867+0000","last_became_active":"2024-04-26T03:39:35.700637+0000","last_became_peered":"2024-04-26T03:39:35.700637+0000","last_unstale":"2024-04-26T03:39:41.822867+0000","last_undegraded":"2024-04-26T03:39:41.822867+0000","last_fullsized":"2024-04-26T03:39:41.822867+0000","mapping_epoch":32,"log_start":"75'58","ondisk_log_start":"75'58","created":32,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"42'18","last_scrub_stamp":"2024-04-26T03:39:07.342868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:07.342868+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":60,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:09.889951+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0095506840000000003,"stat_sum":{"num_bytes":15155200,"num_objects":9,"num_object_clones":4,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":21,"num_read_kb":16669,"num_write":36,"num_write_kb":9840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,13,4,2,6,10,5],"acting":[8,13,4,2,6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":129,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T03:39:42.815417+0000","last_change":"2024-04-26T03:38:37.255389+0000","last_active":"2024-04-26T03:39:42.815417+0000","last_peered":"2024-04-26T03:39:42.815417+0000","last_clean":"2024-04-26T03:39:42.815417+0000","last_became_active":"2024-04-26T03:38:37.255066+0000","last_became_peered":"2024-04-26T03:38:37.255066+0000","last_unstale":"2024-04-26T03:39:42.815417+0000","last_undegraded":"2024-04-26T03:39:42.815417+0000","last_fullsized":"2024-04-26T03:39:42.815417+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:43.860480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.5","version":"76'69","reported_seq":364,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.811633+0000","last_change":"2024-04-26T03:39:39.695213+0000","last_active":"2024-04-26T03:39:41.811633+0000","last_peered":"2024-04-26T03:39:41.811633+0000","last_clean":"2024-04-26T03:39:41.811633+0000","last_became_active":"2024-04-26T03:39:31.701065+0000","last_became_peered":"2024-04-26T03:39:31.701065+0000","last_unstale":"2024-04-26T03:39:41.811633+0000","last_undegraded":"2024-04-26T03:39:41.811633+0000","last_fullsized":"2024-04-26T03:39:41.811633+0000","mapping_epoch":32,"log_start":"69'65","ondisk_log_start":"69'65","created":32,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'51","last_scrub_stamp":"2024-04-26T03:39:06.149763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:06.149763+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":75,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:26.604335+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00052917899999999996,"stat_sum":{"num_bytes":5275648,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":27990,"num_write":25,"num_write_kb":4304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,8,4,2,7,0,5],"acting":[15,8,4,2,7,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.3","version":"85'95","reported_seq":337,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.808329+0000","last_change":"2024-04-26T03:39:39.719129+0000","last_active":"2024-04-26T03:39:41.808329+0000","last_peered":"2024-04-26T03:39:41.808329+0000","last_clean":"2024-04-26T03:39:41.808329+0000","last_became_active":"2024-04-26T03:39:39.698101+0000","last_became_peered":"2024-04-26T03:39:39.698101+0000","last_unstale":"2024-04-26T03:39:41.808329+0000","last_undegraded":"2024-04-26T03:39:41.808329+0000","last_fullsized":"2024-04-26T03:39:41.808329+0000","mapping_epoch":32,"log_start":"80'84","ondisk_log_start":"80'84","created":32,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'32","last_scrub_stamp":"2024-04-26T03:39:07.451867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:07.451867+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":120,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:35.860970+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.015853349999999999,"stat_sum":{"num_bytes":18055168,"num_objects":11,"num_object_clones":6,"num_object_copies":77,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":1,"num_read":17,"num_read_kb":15702,"num_write":43,"num_write_kb":7488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":25575424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,7,4,15,1,2,3],"acting":[10,7,4,15,1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"2.2","version":"0'0","reported_seq":116,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T03:39:40.813708+0000","last_change":"2024-04-26T03:38:48.475569+0000","last_active":"2024-04-26T03:39:40.813708+0000","last_peered":"2024-04-26T03:39:40.813708+0000","last_clean":"2024-04-26T03:39:40.813708+0000","last_became_active":"2024-04-26T03:38:48.475155+0000","last_became_peered":"2024-04-26T03:38:48.475155+0000","last_unstale":"2024-04-26T03:39:40.813708+0000","last_undegraded":"2024-04-26T03:39:40.813708+0000","last_fullsized":"2024-04-26T03:39:40.813708+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,12],"acting":[11,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.0","version":"35'9","reported_seq":280,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T03:39:43.075708+0000","last_change":"2024-04-26T03:39:39.695741+0000","last_active":"2024-04-26T03:39:43.075708+0000","last_peered":"2024-04-26T03:39:43.075708+0000","last_clean":"2024-04-26T03:39:43.075708+0000","last_became_active":"2024-04-26T03:39:01.540712+0000","last_became_peered":"2024-04-26T03:39:01.540712+0000","last_unstale":"2024-04-26T03:39:43.075708+0000","last_undegraded":"2024-04-26T03:39:43.075708+0000","last_fullsized":"2024-04-26T03:39:43.075708+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'9","last_scrub_stamp":"2024-04-26T03:39:06.119088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:06.119088+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:14.749151+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0010303230000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,9,15,2,6,14,4],"acting":[1,9,15,2,6,14,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":128,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.809092+0000","last_change":"2024-04-26T03:38:37.255450+0000","last_active":"2024-04-26T03:39:41.809092+0000","last_peered":"2024-04-26T03:39:41.809092+0000","last_clean":"2024-04-26T03:39:41.809092+0000","last_became_active":"2024-04-26T03:38:37.255181+0000","last_became_peered":"2024-04-26T03:38:37.255181+0000","last_unstale":"2024-04-26T03:39:41.809092+0000","last_undegraded":"2024-04-26T03:39:41.809092+0000","last_fullsized":"2024-04-26T03:39:41.809092+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:00.449388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0],"acting":[14,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.1","version":"42'8","reported_seq":276,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T03:39:42.815359+0000","last_change":"2024-04-26T03:39:39.696946+0000","last_active":"2024-04-26T03:39:42.815359+0000","last_peered":"2024-04-26T03:39:42.815359+0000","last_clean":"2024-04-26T03:39:42.815359+0000","last_became_active":"2024-04-26T03:39:01.547451+0000","last_became_peered":"2024-04-26T03:39:01.547451+0000","last_unstale":"2024-04-26T03:39:42.815359+0000","last_undegraded":"2024-04-26T03:39:42.815359+0000","last_fullsized":"2024-04-26T03:39:42.815359+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"42'8","last_scrub_stamp":"2024-04-26T03:39:08.263903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:08.263903+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:14.658080+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0010901210000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,14,10,9,1,2],"acting":[4,11,14,10,9,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":131,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T03:39:43.827128+0000","last_change":"2024-04-26T03:38:36.431325+0000","last_active":"2024-04-26T03:39:43.827128+0000","last_peered":"2024-04-26T03:39:43.827128+0000","last_clean":"2024-04-26T03:39:43.827128+0000","last_became_active":"2024-04-26T03:38:36.431121+0000","last_became_peered":"2024-04-26T03:38:36.431121+0000","last_unstale":"2024-04-26T03:39:43.827128+0000","last_undegraded":"2024-04-26T03:39:43.827128+0000","last_fullsized":"2024-04-26T03:39:43.827128+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:53.294619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"81'65","reported_seq":350,"reported_epoch":86,"state":"clean+premerge+peered","last_fresh":"2024-04-26T03:39:41.816056+0000","last_change":"2024-04-26T03:39:41.816056+0000","last_active":"2024-04-26T03:39:40.809478+0000","last_peered":"2024-04-26T03:39:41.816056+0000","last_clean":"2024-04-26T03:39:41.816056+0000","last_became_active":"2024-04-26T03:39:01.543967+0000","last_became_peered":"2024-04-26T03:39:41.813826+0000","last_unstale":"2024-04-26T03:39:41.816056+0000","last_undegraded":"2024-04-26T03:39:41.816056+0000","last_fullsized":"2024-04-26T03:39:41.816056+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'31","last_scrub_stamp":"2024-04-26T03:39:07.479325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:07.479325+0000","objects_scrubbed":3,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:27.011441+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0027817699999999998,"stat_sum":{"num_bytes":3588096,"num_objects":5,"num_object_clones":2,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":8,"num_read_kb":4259,"num_write":51,"num_write_kb":3392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,5,3,9,15,10,11],"acting":[14,5,3,9,15,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"2.3","version":"0'0","reported_seq":128,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.809145+0000","last_change":"2024-04-26T03:38:36.431667+0000","last_active":"2024-04-26T03:39:41.809145+0000","last_peered":"2024-04-26T03:39:41.809145+0000","last_clean":"2024-04-26T03:39:41.809145+0000","last_became_active":"2024-04-26T03:38:36.431449+0000","last_became_peered":"2024-04-26T03:38:36.431449+0000","last_unstale":"2024-04-26T03:39:41.809145+0000","last_undegraded":"2024-04-26T03:39:41.809145+0000","last_fullsized":"2024-04-26T03:39:41.809145+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:47.562648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"27'157","reported_seq":337,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:42.110389+0000","last_change":"2024-04-26T03:39:42.110436+0000","last_active":"2024-04-26T03:39:42.110389+0000","last_peered":"2024-04-26T03:39:42.110389+0000","last_clean":"2024-04-26T03:39:42.110389+0000","last_became_active":"2024-04-26T03:38:34.389099+0000","last_became_peered":"2024-04-26T03:38:34.389099+0000","last_unstale":"2024-04-26T03:39:42.110389+0000","last_undegraded":"2024-04-26T03:39:42.110389+0000","last_fullsized":"2024-04-26T03:39:42.110389+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'157","last_scrub_stamp":"2024-04-26T03:39:42.110300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:32.306810+0000","last_clean_scrub_stamp":"2024-04-26T03:39:42.110300+0000","objects_scrubbed":2,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:11.925418+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.8","version":"84'113","reported_seq":336,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.808247+0000","last_change":"2024-04-26T03:39:39.711675+0000","last_active":"2024-04-26T03:39:41.808247+0000","last_peered":"2024-04-26T03:39:41.808247+0000","last_clean":"2024-04-26T03:39:41.808247+0000","last_became_active":"2024-04-26T03:39:19.697021+0000","last_became_peered":"2024-04-26T03:39:19.697021+0000","last_unstale":"2024-04-26T03:39:41.808247+0000","last_undegraded":"2024-04-26T03:39:41.808247+0000","last_fullsized":"2024-04-26T03:39:41.808247+0000","mapping_epoch":32,"log_start":"60'76","ondisk_log_start":"60'76","created":32,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"42'12","last_scrub_stamp":"2024-04-26T03:39:08.136579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:08.136579+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":92,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:28.372198+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.012835276,"stat_sum":{"num_bytes":16859136,"num_objects":6,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":20,"num_read_kb":30343,"num_write":25,"num_write_kb":12208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,10,11,15,5],"acting":[6,0,1,10,11,15,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.9","version":"84'39","reported_seq":319,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.896140+0000","last_change":"2024-04-26T03:39:39.708837+0000","last_active":"2024-04-26T03:39:41.896140+0000","last_peered":"2024-04-26T03:39:41.896140+0000","last_clean":"2024-04-26T03:39:41.896140+0000","last_became_active":"2024-04-26T03:39:15.696734+0000","last_became_peered":"2024-04-26T03:39:15.696734+0000","last_unstale":"2024-04-26T03:39:41.896140+0000","last_undegraded":"2024-04-26T03:39:41.896140+0000","last_fullsized":"2024-04-26T03:39:41.896140+0000","mapping_epoch":32,"log_start":"52'24","ondisk_log_start":"52'24","created":32,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'6","last_scrub_stamp":"2024-04-26T03:39:07.345704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:07.345704+0000","objects_scrubbed":1,"log_size":15,"log_dups_size":29,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:30.212210+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.011853795,"stat_sum":{"num_bytes":4292608,"num_objects":5,"num_object_clones":2,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":2,"num_read":8,"num_read_kb":1587,"num_write":11,"num_write_kb":3248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2,10,9,15,8,4],"acting":[13,2,10,9,15,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.a","version":"84'137","reported_seq":421,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T03:39:43.827609+0000","last_change":"2024-04-26T03:39:39.707721+0000","last_active":"2024-04-26T03:39:43.827609+0000","last_peered":"2024-04-26T03:39:43.827609+0000","last_clean":"2024-04-26T03:39:43.827609+0000","last_became_active":"2024-04-26T03:38:49.486365+0000","last_became_peered":"2024-04-26T03:38:49.486365+0000","last_unstale":"2024-04-26T03:39:43.827609+0000","last_undegraded":"2024-04-26T03:39:43.827609+0000","last_fullsized":"2024-04-26T03:39:43.827609+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:10.724979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.013310159,"stat_sum":{"num_bytes":18563072,"num_objects":14,"num_object_clones":8,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":71,"num_read_kb":40205,"num_write":99,"num_write_kb":19904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,10,13,1,11],"acting":[3,7,6,10,13,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.b","version":"85'130","reported_seq":392,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T03:39:42.815405+0000","last_change":"2024-04-26T03:39:39.714373+0000","last_active":"2024-04-26T03:39:42.815405+0000","last_peered":"2024-04-26T03:39:42.815405+0000","last_clean":"2024-04-26T03:39:42.815405+0000","last_became_active":"2024-04-26T03:38:49.940561+0000","last_became_peered":"2024-04-26T03:38:49.940561+0000","last_unstale":"2024-04-26T03:39:42.815405+0000","last_undegraded":"2024-04-26T03:39:42.815405+0000","last_fullsized":"2024-04-26T03:39:42.815405+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:57.848986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018406249,"stat_sum":{"num_bytes":14614528,"num_objects":10,"num_object_clones":6,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":40,"num_read_kb":12078,"num_write":163,"num_write_kb":20016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,10,12,1,7,14],"acting":[4,0,10,12,1,7,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.c","version":"84'64","reported_seq":313,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T03:39:40.812948+0000","last_change":"2024-04-26T03:39:39.710711+0000","last_active":"2024-04-26T03:39:40.812948+0000","last_peered":"2024-04-26T03:39:40.812948+0000","last_clean":"2024-04-26T03:39:40.812948+0000","last_became_active":"2024-04-26T03:38:49.941017+0000","last_became_peered":"2024-04-26T03:38:49.941017+0000","last_unstale":"2024-04-26T03:39:40.812948+0000","last_undegraded":"2024-04-26T03:39:40.812948+0000","last_fullsized":"2024-04-26T03:39:40.812948+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:18.122180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011208693,"stat_sum":{"num_bytes":3866624,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":18,"num_read_kb":10268,"num_write":64,"num_write_kb":7792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,0,1,9,12,14],"acting":[11,8,0,1,9,12,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.d","version":"84'81","reported_seq":334,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.822238+0000","last_change":"2024-04-26T03:39:39.696226+0000","last_active":"2024-04-26T03:39:41.822238+0000","last_peered":"2024-04-26T03:39:41.822238+0000","last_clean":"2024-04-26T03:39:41.822238+0000","last_became_active":"2024-04-26T03:38:49.511783+0000","last_became_peered":"2024-04-26T03:38:49.511783+0000","last_unstale":"2024-04-26T03:39:41.822238+0000","last_undegraded":"2024-04-26T03:39:41.822238+0000","last_fullsized":"2024-04-26T03:39:41.822238+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:08.988549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063040299999999997,"stat_sum":{"num_bytes":5144576,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":27,"num_read_kb":17290,"num_write":50,"num_write_kb":8048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,9,1,5,8,4],"acting":[7,3,9,1,5,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.e","version":"78'82","reported_seq":348,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.896216+0000","last_change":"2024-04-26T03:39:39.700929+0000","last_active":"2024-04-26T03:39:41.896216+0000","last_peered":"2024-04-26T03:39:41.896216+0000","last_clean":"2024-04-26T03:39:41.896216+0000","last_became_active":"2024-04-26T03:38:49.941260+0000","last_became_peered":"2024-04-26T03:38:49.941260+0000","last_unstale":"2024-04-26T03:39:41.896216+0000","last_undegraded":"2024-04-26T03:39:41.896216+0000","last_fullsized":"2024-04-26T03:39:41.896216+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:05.038642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035573390000000001,"stat_sum":{"num_bytes":6029312,"num_objects":6,"num_object_clones":2,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":2,"num_read":43,"num_read_kb":29296,"num_write":52,"num_write_kb":8432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,14,6,11,8,9,12],"acting":[13,14,6,11,8,9,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.f","version":"84'87","reported_seq":367,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T03:39:40.813048+0000","last_change":"2024-04-26T03:39:39.700458+0000","last_active":"2024-04-26T03:39:40.813048+0000","last_peered":"2024-04-26T03:39:40.813048+0000","last_clean":"2024-04-26T03:39:40.813048+0000","last_became_active":"2024-04-26T03:39:18.756713+0000","last_became_peered":"2024-04-26T03:39:18.756713+0000","last_unstale":"2024-04-26T03:39:40.813048+0000","last_undegraded":"2024-04-26T03:39:40.813048+0000","last_fullsized":"2024-04-26T03:39:40.813048+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:38:48.463214+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:54.423720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001388121,"stat_sum":{"num_bytes":9191424,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":16567,"num_write":57,"num_write_kb":10368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":6094848,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,13,5,14,9,1,0],"acting":[11,13,5,14,9,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.10","version":"84'37","reported_seq":240,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-26T03:39:41.808311+0000","last_change":"2024-04-26T03:39:39.713445+0000","last_active":"2024-04-26T03:39:41.808311+0000","last_peered":"2024-04-26T03:39:41.808311+0000","last_clean":"2024-04-26T03:39:41.808311+0000","last_became_active":"2024-04-26T03:39:04.704276+0000","last_became_peered":"2024-04-26T03:39:04.704276+0000","last_unstale":"2024-04-26T03:39:41.808311+0000","last_undegraded":"2024-04-26T03:39:41.808311+0000","last_fullsized":"2024-04-26T03:39:41.808311+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":5,"last_scrub":"35'9","last_scrub_stamp":"2024-04-26T03:39:07.110098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:07.110098+0000","objects_scrubbed":1,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:34.389540+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.015083931,"stat_sum":{"num_bytes":6995968,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":6316,"num_write":29,"num_write_kb":5840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,7,4,8,0,11],"acting":[6,9,7,4,8,0,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.11","version":"84'26","reported_seq":210,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T03:39:43.826925+0000","last_change":"2024-04-26T03:39:39.707787+0000","last_active":"2024-04-26T03:39:43.826925+0000","last_peered":"2024-04-26T03:39:43.826925+0000","last_clean":"2024-04-26T03:39:43.826925+0000","last_became_active":"2024-04-26T03:39:07.719054+0000","last_became_peered":"2024-04-26T03:39:07.719054+0000","last_unstale":"2024-04-26T03:39:43.826925+0000","last_undegraded":"2024-04-26T03:39:43.826925+0000","last_fullsized":"2024-04-26T03:39:43.826925+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":5,"last_scrub":"42'8","last_scrub_stamp":"2024-04-26T03:39:08.233750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:48.463214+0000","last_clean_scrub_stamp":"2024-04-26T03:39:08.233750+0000","objects_scrubbed":1,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:10.054003+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.011406828000000001,"stat_sum":{"num_bytes":5292032,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":7739,"num_write":21,"num_write_kb":2608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,15,10,0,14,2],"acting":[5,11,15,10,0,14,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"10","length":"1"},{"start":"19","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":18,"stat_sum":{"num_bytes":133054464,"num_objects":85,"num_object_clones":41,"num_object_copies":595,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":7,"num_read":336,"num_read_kb":236694,"num_write":727,"num_write_kb":123488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":31670272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":227377152,"data_stored":306016256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":814,"ondisk_log_size":814,"up":126,"acting":126,"num_store_stats":16},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":13852,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":24,"seq":103079215123,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50620,"kb_used_data":15276,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94321220,"statfs":{"total":96636764160,"available":96584929280,"internally_reserved":0,"allocated":15642624,"data_stored":19199959,"data_compressed":136565,"data_compressed_allocated":294912,"data_compressed_original":1036288,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,4,6,7,8,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215123,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43232,"kb_used_data":12052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94328608,"statfs":{"total":96636764160,"available":96592494592,"internally_reserved":0,"allocated":12341248,"data_stored":16788667,"data_compressed":131739,"data_compressed_allocated":266240,"data_compressed_original":532480,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":23,"seq":98784247829,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53232,"kb_used_data":17892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94318608,"statfs":{"total":96636764160,"available":96582254592,"internally_reserved":0,"allocated":18321408,"data_stored":24597943,"data_compressed":129639,"data_compressed_allocated":262144,"data_compressed_original":524288,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280529,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47232,"kb_used_data":16052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94324608,"statfs":{"total":96636764160,"available":96588398592,"internally_reserved":0,"allocated":16437248,"data_stored":20111220,"data_compressed":134461,"data_compressed_allocated":290816,"data_compressed_original":1028096,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,4,5,7,8,9,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280529,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43516,"kb_used_data":16492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94328324,"statfs":{"total":96636764160,"available":96592203776,"internally_reserved":0,"allocated":16887808,"data_stored":26463572,"data_compressed":127535,"data_compressed_allocated":258048,"data_compressed_original":516096,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,3,4,5,6,8,9,10,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":22,"seq":94489280530,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39380,"kb_used_data":12360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94332460,"statfs":{"total":96636764160,"available":96596439040,"internally_reserved":0,"allocated":12656640,"data_stored":13747742,"data_compressed":125440,"data_compressed_allocated":253952,"data_compressed_original":507904,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[2,4,5,6,7,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280528,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37528,"kb_used_data":10504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94334312,"statfs":{"total":96636764160,"available":96598335488,"internally_reserved":0,"allocated":10756096,"data_stored":12909908,"data_compressed":127535,"data_compressed_allocated":258048,"data_compressed_original":516096,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,2,4,7,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280529,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55096,"kb_used_data":23916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94316744,"statfs":{"total":96636764160,"available":96580345856,"internally_reserved":0,"allocated":24489984,"data_stored":33408542,"data_compressed":125440,"data_compressed_allocated":253952,"data_compressed_original":507904,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[1,2,3,4,7,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313234,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62708,"kb_used_data":17632,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94309132,"statfs":{"total":96636764160,"available":96572551168,"internally_reserved":0,"allocated":18055168,"data_stored":22322577,"data_compressed":123349,"data_compressed_allocated":249856,"data_compressed_original":499712,"omap_allocated":1596,"internal_metadata":46135748},"hb_peers":[0,1,2,4,5,6,7,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":12,"up_from":22,"seq":94489280528,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33880,"kb_used_data":6856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337960,"statfs":{"total":96636764160,"available":96602071040,"internally_reserved":0,"allocated":7020544,"data_stored":11242836,"data_compressed":127535,"data_compressed_allocated":258048,"data_compressed_original":516096,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,7,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280528,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41008,"kb_used_data":13984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94330832,"statfs":{"total":96636764160,"available":96594771968,"internally_reserved":0,"allocated":14319616,"data_stored":17927508,"data_compressed":127535,"data_compressed_allocated":258048,"data_compressed_original":516096,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[2,4,6,7,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247826,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68944,"kb_used_data":15548,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94302896,"statfs":{"total":96636764160,"available":96566165504,"internally_reserved":0,"allocated":15921152,"data_stored":21609812,"data_compressed":127535,"data_compressed_allocated":258048,"data_compressed_original":516096,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,6,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280528,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38308,"kb_used_data":11288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94333532,"statfs":{"total":96636764160,"available":96597536768,"internally_reserved":0,"allocated":11558912,"data_stored":17249822,"data_compressed":125440,"data_compressed_allocated":253952,"data_compressed_original":507904,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,3,5,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":23,"seq":98784247828,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74824,"kb_used_data":21436,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94297016,"statfs":{"total":96636764160,"available":96560144384,"internally_reserved":0,"allocated":21950464,"data_stored":31823287,"data_compressed":129639,"data_compressed_allocated":262144,"data_compressed_original":524288,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,4,6,7,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280528,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40492,"kb_used_data":13468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94331348,"statfs":{"total":96636764160,"available":96595300352,"internally_reserved":0,"allocated":13791232,"data_stored":15289684,"data_compressed":127535,"data_compressed_allocated":258048,"data_compressed_original":516096,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,4,5,7,8,9,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247826,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43880,"kb_used_data":12692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94327960,"statfs":{"total":96636764160,"available":96591831040,"internally_reserved":0,"allocated":12996608,"data_stored":13839700,"data_compressed":127535,"data_compressed_allocated":258048,"data_compressed_original":516096,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":14942208,"data_stored":20893696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20959232,"data_stored":31096832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12017664,"data_stored":13123584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":11337728,"data_stored":16052224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":17330176,"data_stored":23871488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":14598144,"data_stored":17944576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":15405056,"data_stored":18866176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":15908864,"data_stored":25747456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":11689984,"data_stored":13041664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":9777152,"data_stored":12193792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":23523328,"data_stored":32702464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":17100800,"data_stored":21626880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":6041600,"data_stored":10526720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":13340672,"data_stored":17211392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":10592256,"data_stored":16543744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":12812288,"data_stored":14573568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T03:39:46.382 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_0 2024-04-26T03:39:46.383 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_0 unique_pool_0 --yes-i-really-really-mean-it 2024-04-26T03:39:46.395 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-26T03:39:46.395 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-04-26T03:39:46.395 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-04-26T03:39:46.395 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-04-26T03:39:46.395 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.396 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:3 3.2s0 3:42fcabaa:::smithi09235267-1:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:46.396 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.480913+0000", 2024-04-26T03:39:46.396 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 54.912043347999997, 2024-04-26T03:39:46.396 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.012266038999999999, 2024-04-26T03:39:46.396 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:46.396 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:46.396 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:46.396 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:46.396 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:46.396 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:46.396 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 3 2024-04-26T03:39:46.396 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.396 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:46.397 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.397 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:46.397 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.480913+0000", 2024-04-26T03:39:46.397 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.397 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.397 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.397 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:46.397 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.480913+0000", 2024-04-26T03:39:46.397 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.397 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.397 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.397 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:46.397 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.480914+0000", 2024-04-26T03:39:46.397 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.7999999999999996e-07 2024-04-26T03:39:46.398 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.398 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.398 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:46.398 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.480917+0000", 2024-04-26T03:39:46.398 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.366e-06 2024-04-26T03:39:46.398 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.398 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.398 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:46.398 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.480925+0000", 2024-04-26T03:39:46.398 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9759999999999995e-06 2024-04-26T03:39:46.398 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.398 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.398 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.399 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.480932+0000", 2024-04-26T03:39:46.399 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.0339999999999999e-06 2024-04-26T03:39:46.399 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.399 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.399 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:46.399 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.483324+0000", 2024-04-26T03:39:46.399 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.002392323 2024-04-26T03:39:46.399 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.399 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.399 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:46.399 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.483402+0000", 2024-04-26T03:39:46.399 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.7745999999999997e-05 2024-04-26T03:39:46.399 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.399 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.399 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:46.400 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.487467+0000", 2024-04-26T03:39:46.400 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0040650310000000002 2024-04-26T03:39:46.400 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.400 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.400 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:46.400 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.487555+0000", 2024-04-26T03:39:46.400 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.8271999999999998e-05 2024-04-26T03:39:46.400 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.400 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.400 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:46.400 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.493179+0000", 2024-04-26T03:39:46.400 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0056238110000000003 2024-04-26T03:39:46.400 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.400 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:46.401 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.401 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.401 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.401 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:79 3.2s0 3:41b9d80c:::smithi09235267-27:head [append 0~442368,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:46.401 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.531542+0000", 2024-04-26T03:39:46.401 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 54.861414295000003, 2024-04-26T03:39:46.401 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.013543507999999999, 2024-04-26T03:39:46.401 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:46.401 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:46.401 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:46.401 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:46.401 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:46.401 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:46.401 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 79 2024-04-26T03:39:46.402 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.402 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:46.402 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.402 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:46.402 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.531542+0000", 2024-04-26T03:39:46.402 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.402 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.402 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.402 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:46.402 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.531542+0000", 2024-04-26T03:39:46.402 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.402 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.402 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.402 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:46.403 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.531544+0000", 2024-04-26T03:39:46.403 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9719999999999999e-06 2024-04-26T03:39:46.403 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.403 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.403 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:46.403 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.531978+0000", 2024-04-26T03:39:46.403 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00043396899999999998 2024-04-26T03:39:46.403 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.403 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.403 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:46.403 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.531990+0000", 2024-04-26T03:39:46.403 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2150999999999999e-05 2024-04-26T03:39:46.403 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.404 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.404 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.404 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.531997+0000", 2024-04-26T03:39:46.404 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.6320000000000002e-06 2024-04-26T03:39:46.404 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.404 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.404 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:46.404 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532044+0000", 2024-04-26T03:39:46.404 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.7095999999999998e-05 2024-04-26T03:39:46.404 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.404 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.404 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:46.404 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532156+0000", 2024-04-26T03:39:46.404 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000112131 2024-04-26T03:39:46.405 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.405 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.405 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:46.405 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532990+0000", 2024-04-26T03:39:46.405 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00083437099999999998 2024-04-26T03:39:46.405 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.405 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.405 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:46.405 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.537471+0000", 2024-04-26T03:39:46.405 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0044802160000000004 2024-04-26T03:39:46.405 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.405 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.405 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:46.405 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.545008+0000", 2024-04-26T03:39:46.405 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0075372089999999996 2024-04-26T03:39:46.406 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.406 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.406 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:46.406 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.545086+0000", 2024-04-26T03:39:46.406 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.7761000000000003e-05 2024-04-26T03:39:46.406 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.406 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:46.406 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.406 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.406 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.406 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:80 3.2s0 3:41b9d80c:::smithi09235267-27:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:46.406 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.532011+0000", 2024-04-26T03:39:46.406 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 54.860944967000002, 2024-04-26T03:39:46.406 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.013370922, 2024-04-26T03:39:46.407 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:46.407 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:46.407 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:46.407 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:46.407 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:46.407 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:46.407 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 80 2024-04-26T03:39:46.407 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.407 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:46.407 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.407 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:46.407 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532011+0000", 2024-04-26T03:39:46.407 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.407 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.408 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.408 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:46.408 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532011+0000", 2024-04-26T03:39:46.408 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.408 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.408 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.408 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:46.408 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532012+0000", 2024-04-26T03:39:46.408 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.1900000000000003e-07 2024-04-26T03:39:46.408 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.408 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.408 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:46.408 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532015+0000", 2024-04-26T03:39:46.408 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5919999999999999e-06 2024-04-26T03:39:46.409 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.409 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.409 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:46.409 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532025+0000", 2024-04-26T03:39:46.409 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0132999999999999e-05 2024-04-26T03:39:46.409 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.409 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.409 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.409 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532029+0000", 2024-04-26T03:39:46.409 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9820000000000002e-06 2024-04-26T03:39:46.409 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.409 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.409 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:46.409 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.536416+0000", 2024-04-26T03:39:46.410 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0043876480000000001 2024-04-26T03:39:46.410 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.410 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.410 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:46.410 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.536558+0000", 2024-04-26T03:39:46.410 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000141994 2024-04-26T03:39:46.410 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.410 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.410 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:46.410 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.537081+0000", 2024-04-26T03:39:46.410 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00052260200000000005 2024-04-26T03:39:46.410 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.411 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.411 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:46.411 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.537579+0000", 2024-04-26T03:39:46.411 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00049790100000000003 2024-04-26T03:39:46.411 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.411 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.411 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:46.411 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.545304+0000", 2024-04-26T03:39:46.411 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.007724936 2024-04-26T03:39:46.411 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.411 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.411 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:46.411 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.545382+0000", 2024-04-26T03:39:46.411 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8615000000000005e-05 2024-04-26T03:39:46.412 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.412 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:46.412 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.412 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.412 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.412 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:81 3.2s0 3:41b9d80c:::smithi09235267-27:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:46.412 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.532034+0000", 2024-04-26T03:39:46.412 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 54.860922053000003, 2024-04-26T03:39:46.412 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.015088236, 2024-04-26T03:39:46.412 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:46.412 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:46.412 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:46.412 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:46.412 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:46.413 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:46.413 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 81 2024-04-26T03:39:46.413 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.413 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:46.413 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.413 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:46.413 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532034+0000", 2024-04-26T03:39:46.413 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.413 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.413 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.413 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:46.413 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532034+0000", 2024-04-26T03:39:46.413 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.413 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.414 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.414 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:46.414 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532035+0000", 2024-04-26T03:39:46.414 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.22e-07 2024-04-26T03:39:46.414 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.414 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.414 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:46.414 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532036+0000", 2024-04-26T03:39:46.414 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6279999999999999e-06 2024-04-26T03:39:46.414 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.414 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.414 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:46.414 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532041+0000", 2024-04-26T03:39:46.414 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.391e-06 2024-04-26T03:39:46.415 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.415 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.415 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.415 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.532046+0000", 2024-04-26T03:39:46.415 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.4090000000000002e-06 2024-04-26T03:39:46.415 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.415 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.415 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:46.415 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.537306+0000", 2024-04-26T03:39:46.415 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0052594319999999996 2024-04-26T03:39:46.415 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.415 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.415 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:46.415 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.537400+0000", 2024-04-26T03:39:46.415 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.4753999999999994e-05 2024-04-26T03:39:46.416 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.416 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.416 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:46.416 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.545720+0000", 2024-04-26T03:39:46.416 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.008319557 2024-04-26T03:39:46.416 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.416 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.416 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:46.416 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.545802+0000", 2024-04-26T03:39:46.416 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.1898999999999996e-05 2024-04-26T03:39:46.416 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.416 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.416 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:46.416 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.547123+0000", 2024-04-26T03:39:46.417 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0013208440000000001 2024-04-26T03:39:46.417 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.417 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:46.417 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.417 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.417 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.417 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:139 3.2s0 3:450a1cc5:::smithi09235267-47:head [append 0~786432,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:46.417 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.619094+0000", 2024-04-26T03:39:46.417 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 54.773862006999998, 2024-04-26T03:39:46.417 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.015918081000000001, 2024-04-26T03:39:46.417 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:46.417 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:46.417 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:46.417 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:46.418 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:46.418 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:46.418 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 139 2024-04-26T03:39:46.418 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.418 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:46.418 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.418 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:46.418 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619094+0000", 2024-04-26T03:39:46.418 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.418 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.418 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.418 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:46.418 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619094+0000", 2024-04-26T03:39:46.419 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.419 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.419 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.419 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:46.419 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619096+0000", 2024-04-26T03:39:46.419 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6559999999999999e-06 2024-04-26T03:39:46.419 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.419 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.419 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:46.419 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619809+0000", 2024-04-26T03:39:46.419 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00071288600000000003 2024-04-26T03:39:46.419 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.419 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.419 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:46.420 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619821+0000", 2024-04-26T03:39:46.420 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1586000000000001e-05 2024-04-26T03:39:46.420 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.420 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.420 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.420 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619826+0000", 2024-04-26T03:39:46.420 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.3870000000000003e-06 2024-04-26T03:39:46.420 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.420 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.420 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:46.420 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619879+0000", 2024-04-26T03:39:46.420 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.2660000000000001e-05 2024-04-26T03:39:46.420 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.421 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.421 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:46.421 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619966+0000", 2024-04-26T03:39:46.421 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7777999999999999e-05 2024-04-26T03:39:46.421 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.421 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.421 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:46.421 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.620812+0000", 2024-04-26T03:39:46.421 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00084571199999999998 2024-04-26T03:39:46.421 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.421 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.421 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:46.421 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.627482+0000", 2024-04-26T03:39:46.421 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0066701590000000002 2024-04-26T03:39:46.422 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.422 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.422 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:46.422 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.634919+0000", 2024-04-26T03:39:46.422 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.007437056 2024-04-26T03:39:46.422 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.422 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.422 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:46.422 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.635013+0000", 2024-04-26T03:39:46.422 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.3201000000000004e-05 2024-04-26T03:39:46.422 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.422 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:46.422 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.422 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.423 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.423 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:140 3.2s0 3:450a1cc5:::smithi09235267-47:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:46.423 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.619840+0000", 2024-04-26T03:39:46.423 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 54.773116946000002, 2024-04-26T03:39:46.423 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.015356137000000001, 2024-04-26T03:39:46.423 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:46.423 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:46.423 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:46.423 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:46.423 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:46.423 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:46.423 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 140 2024-04-26T03:39:46.423 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.423 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:46.424 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.424 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:46.424 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619840+0000", 2024-04-26T03:39:46.424 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.424 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.424 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.424 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:46.424 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619840+0000", 2024-04-26T03:39:46.424 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.424 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.424 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.424 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:46.424 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619840+0000", 2024-04-26T03:39:46.424 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.8100000000000002e-07 2024-04-26T03:39:46.424 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.425 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.425 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:46.425 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619843+0000", 2024-04-26T03:39:46.425 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.785e-06 2024-04-26T03:39:46.425 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.425 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.425 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:46.425 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619848+0000", 2024-04-26T03:39:46.425 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.2490000000000001e-06 2024-04-26T03:39:46.425 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.425 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.425 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.425 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619852+0000", 2024-04-26T03:39:46.425 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.1869999999999999e-06 2024-04-26T03:39:46.426 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.426 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.426 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:46.426 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.626570+0000", 2024-04-26T03:39:46.426 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0067177679999999998 2024-04-26T03:39:46.426 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.426 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.426 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:46.426 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.626669+0000", 2024-04-26T03:39:46.426 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.9177000000000003e-05 2024-04-26T03:39:46.426 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.426 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.427 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:46.427 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.627020+0000", 2024-04-26T03:39:46.427 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00035101199999999999 2024-04-26T03:39:46.427 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.427 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.427 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:46.427 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.627486+0000", 2024-04-26T03:39:46.427 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00046560299999999998 2024-04-26T03:39:46.427 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.427 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.427 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:46.427 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.635101+0000", 2024-04-26T03:39:46.427 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0076154680000000002 2024-04-26T03:39:46.427 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.427 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.428 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:46.428 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.635196+0000", 2024-04-26T03:39:46.428 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.4207000000000001e-05 2024-04-26T03:39:46.428 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.428 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:46.428 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.428 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.428 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.428 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:141 3.2s0 3:450a1cc5:::smithi09235267-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:46.428 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.619858+0000", 2024-04-26T03:39:46.428 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 54.773098613999998, 2024-04-26T03:39:46.428 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.017889577, 2024-04-26T03:39:46.428 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:46.429 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:46.429 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:46.429 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:46.429 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:46.429 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:46.429 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 141 2024-04-26T03:39:46.429 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.429 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:46.429 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.429 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:46.429 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619858+0000", 2024-04-26T03:39:46.429 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.429 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.429 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.430 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:46.430 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619858+0000", 2024-04-26T03:39:46.430 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.430 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.430 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.430 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:46.430 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619858+0000", 2024-04-26T03:39:46.430 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7000000000000001e-07 2024-04-26T03:39:46.430 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.430 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.430 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:46.430 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619860+0000", 2024-04-26T03:39:46.430 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1090000000000001e-06 2024-04-26T03:39:46.431 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.431 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.431 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:46.431 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619864+0000", 2024-04-26T03:39:46.431 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.1330000000000001e-06 2024-04-26T03:39:46.431 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.431 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.431 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.431 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.619867+0000", 2024-04-26T03:39:46.431 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.9100000000000001e-06 2024-04-26T03:39:46.431 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.431 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.431 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:46.431 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.627233+0000", 2024-04-26T03:39:46.432 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0073655659999999996 2024-04-26T03:39:46.432 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.432 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.432 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:46.432 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.627322+0000", 2024-04-26T03:39:46.432 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.9456999999999994e-05 2024-04-26T03:39:46.432 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.432 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.432 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:46.432 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.635530+0000", 2024-04-26T03:39:46.432 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0082076349999999996 2024-04-26T03:39:46.432 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.432 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.432 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:46.433 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.635609+0000", 2024-04-26T03:39:46.433 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8724e-05 2024-04-26T03:39:46.433 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.433 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.433 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:46.433 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.637747+0000", 2024-04-26T03:39:46.433 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0021387730000000001 2024-04-26T03:39:46.433 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.433 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:46.433 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.433 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.433 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.433 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:223 3.2s0 3:42fcabaa:::smithi09235267-1:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:46.433 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:51.838898+0000", 2024-04-26T03:39:46.434 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 54.554058089999998, 2024-04-26T03:39:46.434 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.012225915, 2024-04-26T03:39:46.434 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:46.434 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:46.434 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:46.434 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:46.434 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:46.434 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:46.434 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 223 2024-04-26T03:39:46.434 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.434 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:46.434 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.434 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:46.435 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.838898+0000", 2024-04-26T03:39:46.435 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.435 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.435 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.435 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:46.435 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.838898+0000", 2024-04-26T03:39:46.435 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.435 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.435 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.435 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:46.435 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.838899+0000", 2024-04-26T03:39:46.435 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.7e-07 2024-04-26T03:39:46.435 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.435 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.436 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:46.436 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.838901+0000", 2024-04-26T03:39:46.436 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5799999999999999e-06 2024-04-26T03:39:46.436 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.436 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.436 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:46.436 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.838910+0000", 2024-04-26T03:39:46.436 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.1189999999999997e-06 2024-04-26T03:39:46.436 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.436 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.436 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.436 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.838916+0000", 2024-04-26T03:39:46.436 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.8630000000000002e-06 2024-04-26T03:39:46.436 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.437 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.437 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:46.437 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.845205+0000", 2024-04-26T03:39:46.437 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0062883449999999999 2024-04-26T03:39:46.437 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.437 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.437 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:46.437 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.845254+0000", 2024-04-26T03:39:46.437 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.9101000000000003e-05 2024-04-26T03:39:46.437 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.437 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.437 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:46.437 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.850264+0000", 2024-04-26T03:39:46.437 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0050106079999999997 2024-04-26T03:39:46.438 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.438 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.438 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:46.438 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.850341+0000", 2024-04-26T03:39:46.438 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.6654000000000002e-05 2024-04-26T03:39:46.438 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.438 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.438 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:46.438 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:51.851124+0000", 2024-04-26T03:39:46.438 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000783175 2024-04-26T03:39:46.438 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.438 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:46.438 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.438 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.439 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.439 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.es1 43/32 ECSubWrite(tid=48, reqid=client.4556.0:301, at_version=43'16, trim_to=0'0, roll_forward_to=35'14))", 2024-04-26T03:39:46.439 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:38:59.531315+0000", 2024-04-26T03:39:46.439 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 46.861641139, 2024-04-26T03:39:46.439 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.014140454, 2024-04-26T03:39:46.439 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:46.439 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:46.439 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:46.439 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:46.439 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.439 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:46.439 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.531315+0000", 2024-04-26T03:39:46.439 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.439 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.440 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.440 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:46.440 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.531315+0000", 2024-04-26T03:39:46.440 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.440 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.440 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.440 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:46.440 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.531316+0000", 2024-04-26T03:39:46.440 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1799999999999999e-07 2024-04-26T03:39:46.440 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.440 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.440 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:46.440 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.531691+0000", 2024-04-26T03:39:46.440 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00037520700000000001 2024-04-26T03:39:46.440 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.441 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.441 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:46.441 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.531705+0000", 2024-04-26T03:39:46.441 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.395e-05 2024-04-26T03:39:46.441 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.441 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.441 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.441 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.531713+0000", 2024-04-26T03:39:46.441 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.0150000000000003e-06 2024-04-26T03:39:46.441 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.441 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.441 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:46.441 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.531798+0000", 2024-04-26T03:39:46.442 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5714e-05 2024-04-26T03:39:46.442 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.442 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.442 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:46.442 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.531811+0000", 2024-04-26T03:39:46.442 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2782000000000001e-05 2024-04-26T03:39:46.442 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.442 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.442 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:46.442 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.545405+0000", 2024-04-26T03:39:46.442 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.013593888 2024-04-26T03:39:46.442 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.442 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.442 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:46.443 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:38:59.545456+0000", 2024-04-26T03:39:46.443 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.0680000000000003e-05 2024-04-26T03:39:46.443 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.443 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:46.443 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.443 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.443 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.443 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.fs3 45/32 ECSubWrite(tid=49, reqid=client.4556.0:325, at_version=45'17, trim_to=0'0, roll_forward_to=42'15))", 2024-04-26T03:39:46.443 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:01.554312+0000", 2024-04-26T03:39:46.443 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 44.838644043999999, 2024-04-26T03:39:46.443 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.012529861, 2024-04-26T03:39:46.443 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:46.443 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:46.443 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:46.444 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:46.444 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.444 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:46.444 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.554312+0000", 2024-04-26T03:39:46.444 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.444 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.444 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.444 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:46.444 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.554312+0000", 2024-04-26T03:39:46.444 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.444 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.444 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.444 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:46.444 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.554313+0000", 2024-04-26T03:39:46.445 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.9999999999999996e-07 2024-04-26T03:39:46.445 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.445 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.445 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:46.445 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.554865+0000", 2024-04-26T03:39:46.445 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00055152399999999996 2024-04-26T03:39:46.445 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.445 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.445 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:46.445 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.554886+0000", 2024-04-26T03:39:46.445 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1418999999999999e-05 2024-04-26T03:39:46.445 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.445 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.445 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.446 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.554897+0000", 2024-04-26T03:39:46.446 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0994e-05 2024-04-26T03:39:46.446 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.446 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.446 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:46.446 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.554971+0000", 2024-04-26T03:39:46.446 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3516000000000006e-05 2024-04-26T03:39:46.446 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.446 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.446 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:46.446 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.554983+0000", 2024-04-26T03:39:46.446 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2133999999999999e-05 2024-04-26T03:39:46.446 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.446 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.447 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:46.447 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.566763+0000", 2024-04-26T03:39:46.447 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.011780067 2024-04-26T03:39:46.447 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.447 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.447 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:46.447 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.566842+0000", 2024-04-26T03:39:46.447 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9307000000000005e-05 2024-04-26T03:39:46.447 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.447 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:46.447 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.447 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.447 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.447 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:333 3.2s0 3:450a1cc5:::smithi09235267-47:head [copy-from ver 6] snapc 7=[7,6,5] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-26T03:39:46.448 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:01.558145+0000", 2024-04-26T03:39:46.448 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 44.834811705, 2024-04-26T03:39:46.448 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.017282072999999998, 2024-04-26T03:39:46.448 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:46.448 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:46.448 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:46.448 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:46.448 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:46.448 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:46.448 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 333 2024-04-26T03:39:46.448 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.448 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:46.448 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.448 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:46.448 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558145+0000", 2024-04-26T03:39:46.449 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.449 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.449 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.449 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:46.449 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558145+0000", 2024-04-26T03:39:46.449 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.449 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.449 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.449 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:46.449 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558147+0000", 2024-04-26T03:39:46.449 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4360000000000001e-06 2024-04-26T03:39:46.449 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.449 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.449 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:46.450 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558166+0000", 2024-04-26T03:39:46.450 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8411999999999999e-05 2024-04-26T03:39:46.450 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.450 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.450 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:46.450 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558181+0000", 2024-04-26T03:39:46.450 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4956e-05 2024-04-26T03:39:46.450 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.450 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.450 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.450 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558190+0000", 2024-04-26T03:39:46.450 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.7599999999999997e-06 2024-04-26T03:39:46.450 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.450 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.451 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:46.451 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558309+0000", 2024-04-26T03:39:46.451 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000118893 2024-04-26T03:39:46.451 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.451 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.451 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:46.451 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558491+0000", 2024-04-26T03:39:46.451 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00018157000000000001 2024-04-26T03:39:46.451 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.451 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.451 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:46.451 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.566556+0000", 2024-04-26T03:39:46.451 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0080647549999999998 2024-04-26T03:39:46.451 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.452 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.452 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:46.452 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.572889+0000", 2024-04-26T03:39:46.452 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0063337100000000002 2024-04-26T03:39:46.452 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.452 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.452 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:46.452 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.575342+0000", 2024-04-26T03:39:46.452 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0024523269999999998 2024-04-26T03:39:46.452 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.452 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.452 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:46.452 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.575427+0000", 2024-04-26T03:39:46.452 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5253999999999994e-05 2024-04-26T03:39:46.453 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.453 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:46.453 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.453 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.453 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.453 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:334 3.2s0 3:450a1cc5:::smithi09235267-47:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e45)", 2024-04-26T03:39:46.453 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:01.558215+0000", 2024-04-26T03:39:46.453 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 44.834741553999997, 2024-04-26T03:39:46.453 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.013358238999999999, 2024-04-26T03:39:46.453 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:46.453 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:46.453 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:46.453 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:46.453 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:46.453 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:46.454 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 334 2024-04-26T03:39:46.454 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.454 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:46.454 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.454 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:46.454 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558215+0000", 2024-04-26T03:39:46.454 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.454 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.454 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.454 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:46.454 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558215+0000", 2024-04-26T03:39:46.454 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.454 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.454 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.455 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:46.455 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558216+0000", 2024-04-26T03:39:46.455 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.0800000000000004e-07 2024-04-26T03:39:46.455 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.455 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.455 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:46.455 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558220+0000", 2024-04-26T03:39:46.455 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8410000000000002e-06 2024-04-26T03:39:46.455 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.455 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.455 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:46.455 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558233+0000", 2024-04-26T03:39:46.455 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3556e-05 2024-04-26T03:39:46.456 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.456 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.456 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.456 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558241+0000", 2024-04-26T03:39:46.456 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.4080000000000001e-06 2024-04-26T03:39:46.456 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.456 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.456 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:46.456 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558731+0000", 2024-04-26T03:39:46.456 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00049039899999999998 2024-04-26T03:39:46.456 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.456 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.456 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for blocked object", 2024-04-26T03:39:46.456 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.558777+0000", 2024-04-26T03:39:46.457 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6301000000000003e-05 2024-04-26T03:39:46.457 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.457 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.457 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:46.457 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.571433+0000", 2024-04-26T03:39:46.457 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.012656137 2024-04-26T03:39:46.457 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.457 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.457 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:46.457 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.571501+0000", 2024-04-26T03:39:46.457 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7319999999999999e-05 2024-04-26T03:39:46.457 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.457 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.457 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:46.458 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:01.571573+0000", 2024-04-26T03:39:46.458 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.2469e-05 2024-04-26T03:39:46.458 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.458 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:46.458 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.458 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.458 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.458 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:433 3.2s0 3:42fcabaa:::smithi09235267-1:head [append 0~671744] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-04-26T03:39:46.458 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:04.692691+0000", 2024-04-26T03:39:46.458 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 41.700265063000003, 2024-04-26T03:39:46.458 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.026947377000000002, 2024-04-26T03:39:46.458 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:46.458 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:46.458 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:46.459 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:46.459 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:46.459 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:46.459 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 433 2024-04-26T03:39:46.459 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.459 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:46.459 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.459 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:46.459 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.692691+0000", 2024-04-26T03:39:46.459 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.459 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.459 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.459 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:46.459 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.692691+0000", 2024-04-26T03:39:46.459 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.460 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.460 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.460 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:46.460 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.692694+0000", 2024-04-26T03:39:46.460 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.216e-06 2024-04-26T03:39:46.460 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.460 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.460 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:46.460 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.693610+0000", 2024-04-26T03:39:46.460 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00091655899999999995 2024-04-26T03:39:46.460 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.460 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.460 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:46.460 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.693622+0000", 2024-04-26T03:39:46.461 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1976e-05 2024-04-26T03:39:46.461 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.461 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.461 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.461 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.693629+0000", 2024-04-26T03:39:46.461 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5749999999999997e-06 2024-04-26T03:39:46.461 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.461 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.461 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:46.461 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.693698+0000", 2024-04-26T03:39:46.461 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.9529000000000005e-05 2024-04-26T03:39:46.461 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.461 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.461 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op must wait for map", 2024-04-26T03:39:46.461 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.693708+0000", 2024-04-26T03:39:46.462 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.6649999999999998e-06 2024-04-26T03:39:46.462 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.462 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.462 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:46.462 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.698905+0000", 2024-04-26T03:39:46.462 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.005197279 2024-04-26T03:39:46.462 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.462 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.462 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:46.462 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.698993+0000", 2024-04-26T03:39:46.462 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.8029999999999996e-05 2024-04-26T03:39:46.462 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.462 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.463 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:46.463 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.699957+0000", 2024-04-26T03:39:46.463 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00096368199999999999 2024-04-26T03:39:46.463 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.463 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.463 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:46.463 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.718295+0000", 2024-04-26T03:39:46.463 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.018338255000000001 2024-04-26T03:39:46.463 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.463 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.463 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:46.463 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.719556+0000", 2024-04-26T03:39:46.463 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001261086 2024-04-26T03:39:46.463 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.464 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.464 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:46.464 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.719639+0000", 2024-04-26T03:39:46.464 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2525e-05 2024-04-26T03:39:46.464 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.464 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:46.464 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.464 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.464 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.464 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:434 3.2s0 3:42fcabaa:::smithi09235267-1:head [append 0~770048] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-04-26T03:39:46.464 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:04.699291+0000", 2024-04-26T03:39:46.464 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 41.693665365999998, 2024-04-26T03:39:46.464 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.029702725999999999, 2024-04-26T03:39:46.465 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:46.465 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:46.465 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:46.465 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:46.465 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:46.465 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:46.465 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 434 2024-04-26T03:39:46.465 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.465 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:46.465 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.465 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:46.465 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.699291+0000", 2024-04-26T03:39:46.465 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.466 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.466 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.466 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:46.466 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.699291+0000", 2024-04-26T03:39:46.466 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.466 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.466 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.466 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:46.466 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.699292+0000", 2024-04-26T03:39:46.466 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.192e-06 2024-04-26T03:39:46.466 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.466 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.466 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:46.466 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700019+0000", 2024-04-26T03:39:46.467 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00072667100000000002 2024-04-26T03:39:46.467 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.467 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.467 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:46.467 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700032+0000", 2024-04-26T03:39:46.467 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3064999999999999e-05 2024-04-26T03:39:46.467 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.467 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.467 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.467 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700040+0000", 2024-04-26T03:39:46.467 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5299999999999999e-06 2024-04-26T03:39:46.467 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.467 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.467 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:46.468 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.711078+0000", 2024-04-26T03:39:46.468 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.011038536 2024-04-26T03:39:46.468 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.468 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.468 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:46.468 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.711181+0000", 2024-04-26T03:39:46.468 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000102833 2024-04-26T03:39:46.468 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.468 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.468 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:46.468 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.712518+0000", 2024-04-26T03:39:46.468 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0013371769999999999 2024-04-26T03:39:46.468 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.468 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.469 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:46.469 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.719306+0000", 2024-04-26T03:39:46.469 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0067879009999999998 2024-04-26T03:39:46.469 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.469 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.469 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:46.469 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728932+0000", 2024-04-26T03:39:46.469 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0096258230000000004 2024-04-26T03:39:46.469 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.469 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.469 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:46.469 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.728994+0000", 2024-04-26T03:39:46.469 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.1997999999999998e-05 2024-04-26T03:39:46.469 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.470 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:46.470 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.470 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.470 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.470 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:435 3.2s0 3:42fcabaa:::smithi09235267-1:head [setxattr _header (55) in=7b] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-04-26T03:39:46.470 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:04.700054+0000", 2024-04-26T03:39:46.470 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 41.692902109000002, 2024-04-26T03:39:46.470 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.029111279, 2024-04-26T03:39:46.470 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:46.470 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:46.470 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:46.470 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:46.470 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:46.470 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:46.471 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 435 2024-04-26T03:39:46.471 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.471 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:46.471 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.471 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:46.471 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700054+0000", 2024-04-26T03:39:46.471 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.471 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.471 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.471 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:46.471 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700054+0000", 2024-04-26T03:39:46.471 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.471 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.471 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.472 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:46.472 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700055+0000", 2024-04-26T03:39:46.472 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0470000000000001e-06 2024-04-26T03:39:46.472 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.472 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.472 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:46.472 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700059+0000", 2024-04-26T03:39:46.472 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8909999999999997e-06 2024-04-26T03:39:46.472 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.472 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.472 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:46.472 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700067+0000", 2024-04-26T03:39:46.472 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.447e-06 2024-04-26T03:39:46.472 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.472 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.473 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.473 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700072+0000", 2024-04-26T03:39:46.473 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7529999999999998e-06 2024-04-26T03:39:46.473 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.473 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.473 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:46.473 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.718645+0000", 2024-04-26T03:39:46.473 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.018572564 2024-04-26T03:39:46.473 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.473 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.473 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:46.473 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.718711+0000", 2024-04-26T03:39:46.474 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5743999999999999e-05 2024-04-26T03:39:46.474 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.474 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.474 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:46.474 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.718985+0000", 2024-04-26T03:39:46.474 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00027395400000000003 2024-04-26T03:39:46.474 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.474 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.474 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:46.474 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.719895+0000", 2024-04-26T03:39:46.474 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00091011499999999997 2024-04-26T03:39:46.474 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.474 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.475 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:46.475 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.729090+0000", 2024-04-26T03:39:46.475 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0091950920000000002 2024-04-26T03:39:46.475 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.475 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.475 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:46.475 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.729166+0000", 2024-04-26T03:39:46.475 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5672000000000003e-05 2024-04-26T03:39:46.475 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.475 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:46.475 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.475 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.475 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.475 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:436 3.2s0 3:42fcabaa:::smithi09235267-1:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e49)", 2024-04-26T03:39:46.475 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:04.700082+0000", 2024-04-26T03:39:46.476 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 41.692874463999999, 2024-04-26T03:39:46.476 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.032317973, 2024-04-26T03:39:46.476 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:46.476 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:46.476 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:46.476 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:46.476 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:46.476 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:46.476 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 436 2024-04-26T03:39:46.476 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.476 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:46.476 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.476 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:46.476 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700082+0000", 2024-04-26T03:39:46.477 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.477 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.477 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.477 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:46.477 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700082+0000", 2024-04-26T03:39:46.477 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.477 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.477 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.477 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:46.477 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700082+0000", 2024-04-26T03:39:46.477 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.3000000000000001e-07 2024-04-26T03:39:46.477 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.477 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.477 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:46.477 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700085+0000", 2024-04-26T03:39:46.478 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4739999999999999e-06 2024-04-26T03:39:46.478 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.478 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.478 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:46.478 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700092+0000", 2024-04-26T03:39:46.478 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3000000000000004e-06 2024-04-26T03:39:46.478 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.478 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.478 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.478 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.700097+0000", 2024-04-26T03:39:46.478 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.2290000000000002e-06 2024-04-26T03:39:46.478 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.478 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.478 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:46.478 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.719230+0000", 2024-04-26T03:39:46.479 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.019132564000000001 2024-04-26T03:39:46.479 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.479 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.479 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:46.479 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.719296+0000", 2024-04-26T03:39:46.479 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.6370000000000003e-05 2024-04-26T03:39:46.479 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.479 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.479 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:46.479 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.729620+0000", 2024-04-26T03:39:46.479 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.010323183999999999 2024-04-26T03:39:46.479 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.479 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.479 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:46.480 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.729695+0000", 2024-04-26T03:39:46.480 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5500000000000006e-05 2024-04-26T03:39:46.480 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.480 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.480 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:46.480 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:04.732400+0000", 2024-04-26T03:39:46.480 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0027049219999999998 2024-04-26T03:39:46.480 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.480 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:46.480 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.480 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.480 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.480 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:478 3.2s0 3:450a1cc5:::smithi09235267-47:head [append 0~802816] snapc 8=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:46.480 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:05.910200+0000", 2024-04-26T03:39:46.481 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 40.482756780999999, 2024-04-26T03:39:46.481 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.013459199999999999, 2024-04-26T03:39:46.481 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:46.481 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:46.481 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:46.481 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:46.481 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:46.481 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:46.481 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 478 2024-04-26T03:39:46.481 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.481 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:46.481 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.481 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:46.481 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.910200+0000", 2024-04-26T03:39:46.481 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.482 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.482 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.482 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:46.482 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.910200+0000", 2024-04-26T03:39:46.482 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.482 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.482 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.482 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:46.482 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.910202+0000", 2024-04-26T03:39:46.482 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.683e-06 2024-04-26T03:39:46.482 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.482 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.483 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:46.483 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.910944+0000", 2024-04-26T03:39:46.483 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00074142299999999995 2024-04-26T03:39:46.483 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.483 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.483 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:46.483 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.910959+0000", 2024-04-26T03:39:46.483 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4909000000000001e-05 2024-04-26T03:39:46.483 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.483 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.483 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.483 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.910966+0000", 2024-04-26T03:39:46.483 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.827e-06 2024-04-26T03:39:46.484 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.484 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.484 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:46.484 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.911019+0000", 2024-04-26T03:39:46.484 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.3817999999999999e-05 2024-04-26T03:39:46.484 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.484 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.484 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:46.484 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.911094+0000", 2024-04-26T03:39:46.484 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5059000000000001e-05 2024-04-26T03:39:46.484 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.484 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.484 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:46.484 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.911822+0000", 2024-04-26T03:39:46.484 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00072724499999999995 2024-04-26T03:39:46.485 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.485 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.485 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:46.485 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.923328+0000", 2024-04-26T03:39:46.485 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.01150674 2024-04-26T03:39:46.485 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.485 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.485 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-04-26T03:39:46.485 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.923574+0000", 2024-04-26T03:39:46.485 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00024563200000000002 2024-04-26T03:39:46.485 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.485 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.485 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:46.485 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.923659+0000", 2024-04-26T03:39:46.486 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.4864000000000006e-05 2024-04-26T03:39:46.486 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.486 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:46.486 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.486 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.486 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.486 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:482 3.2s0 3:450a1cc5:::smithi09235267-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:46.486 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:05.914446+0000", 2024-04-26T03:39:46.486 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 40.478510214000003, 2024-04-26T03:39:46.486 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.01384406, 2024-04-26T03:39:46.486 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:46.486 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:46.486 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-04-26T03:39:46.486 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:46.487 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:46.487 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:46.487 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 482 2024-04-26T03:39:46.487 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.487 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:46.487 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.487 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:46.487 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.914446+0000", 2024-04-26T03:39:46.487 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.487 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.487 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.487 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:46.487 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.914446+0000", 2024-04-26T03:39:46.487 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.488 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.488 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.488 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:46.488 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.914447+0000", 2024-04-26T03:39:46.488 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.1600000000000001e-07 2024-04-26T03:39:46.488 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.488 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.488 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:46.488 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.914449+0000", 2024-04-26T03:39:46.488 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4229999999999999e-06 2024-04-26T03:39:46.488 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.488 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.488 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:46.488 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.914458+0000", 2024-04-26T03:39:46.489 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.9299999999999992e-06 2024-04-26T03:39:46.489 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.489 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.489 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.489 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.914465+0000", 2024-04-26T03:39:46.489 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.6599999999999998e-06 2024-04-26T03:39:46.489 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.489 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.489 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:46.489 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.924755+0000", 2024-04-26T03:39:46.489 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.010289777 2024-04-26T03:39:46.489 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.489 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.489 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:46.490 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.924801+0000", 2024-04-26T03:39:46.490 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6109000000000002e-05 2024-04-26T03:39:46.490 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.490 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.490 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:46.490 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.926051+0000", 2024-04-26T03:39:46.490 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001250393 2024-04-26T03:39:46.490 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.490 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.490 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-04-26T03:39:46.490 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.926140+0000", 2024-04-26T03:39:46.490 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.9227999999999993e-05 2024-04-26T03:39:46.490 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.490 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.491 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:46.491 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:05.928290+0000", 2024-04-26T03:39:46.491 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0021500239999999999 2024-04-26T03:39:46.491 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.491 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:46.491 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.491 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.491 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.491 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.fs3 65/62 ECSubWrite(tid=178, reqid=client.4556.0:782, at_version=65'60, trim_to=0'0, roll_forward_to=64'57))", 2024-04-26T03:39:46.491 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:20.829001+0000", 2024-04-26T03:39:46.491 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 25.563955246999999, 2024-04-26T03:39:46.491 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.012433502000000001, 2024-04-26T03:39:46.491 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:46.491 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:46.492 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:46.492 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:46.492 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.492 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:46.492 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.829001+0000", 2024-04-26T03:39:46.492 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.492 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.492 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.492 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:46.492 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.829001+0000", 2024-04-26T03:39:46.492 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.492 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.492 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.492 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:46.492 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.829002+0000", 2024-04-26T03:39:46.493 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.6000000000000003e-07 2024-04-26T03:39:46.493 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.493 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.493 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:46.493 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.829342+0000", 2024-04-26T03:39:46.493 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00033970400000000003 2024-04-26T03:39:46.493 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.493 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.493 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:46.493 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.829356+0000", 2024-04-26T03:39:46.493 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.434e-05 2024-04-26T03:39:46.493 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.493 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.494 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.494 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.829365+0000", 2024-04-26T03:39:46.494 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.0149999999999992e-06 2024-04-26T03:39:46.494 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.494 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.494 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:46.494 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.829431+0000", 2024-04-26T03:39:46.494 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.6080000000000004e-05 2024-04-26T03:39:46.494 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.494 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.494 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-04-26T03:39:46.494 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.829442+0000", 2024-04-26T03:39:46.494 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1439e-05 2024-04-26T03:39:46.494 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.495 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.495 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-04-26T03:39:46.495 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.841390+0000", 2024-04-26T03:39:46.495 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.01194714 2024-04-26T03:39:46.495 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.495 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.495 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:46.495 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:20.841435+0000", 2024-04-26T03:39:46.495 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.5123999999999997e-05 2024-04-26T03:39:46.495 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.495 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:46.495 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.495 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.495 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.496 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4556.0:969 3.2s0 3.55d53f42 (undecoded) ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e85)", 2024-04-26T03:39:46.496 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-04-26T03:39:40.923521+0000", 2024-04-26T03:39:46.496 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 5.4694350299999996, 2024-04-26T03:39:46.496 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.89280510199999996, 2024-04-26T03:39:46.496 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-04-26T03:39:46.496 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-04-26T03:39:46.496 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-04-26T03:39:46.496 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-04-26T03:39:46.496 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4556", 2024-04-26T03:39:46.496 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:46.496 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 969 2024-04-26T03:39:46.496 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.496 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-04-26T03:39:46.497 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.497 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-04-26T03:39:46.497 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:40.923521+0000", 2024-04-26T03:39:46.497 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.497 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.497 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.497 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-04-26T03:39:46.497 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:40.923521+0000", 2024-04-26T03:39:46.497 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-04-26T03:39:46.497 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.497 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.497 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-04-26T03:39:46.497 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:40.923524+0000", 2024-04-26T03:39:46.497 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.627e-06 2024-04-26T03:39:46.497 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.498 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.498 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-04-26T03:39:46.498 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:40.923535+0000", 2024-04-26T03:39:46.498 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1260000000000001e-05 2024-04-26T03:39:46.498 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.498 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.498 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-04-26T03:39:46.498 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:40.923567+0000", 2024-04-26T03:39:46.498 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.1900000000000003e-05 2024-04-26T03:39:46.498 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.498 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.498 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.498 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:40.923583+0000", 2024-04-26T03:39:46.498 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5389000000000001e-05 2024-04-26T03:39:46.499 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.499 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.499 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:46.499 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:40.923706+0000", 2024-04-26T03:39:46.499 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000123583 2024-04-26T03:39:46.499 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.499 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.499 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for peered", 2024-04-26T03:39:46.499 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:40.923711+0000", 2024-04-26T03:39:46.499 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.578e-06 2024-04-26T03:39:46.499 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.499 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.499 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-04-26T03:39:46.499 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:41.816284+0000", 2024-04-26T03:39:46.500 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.89257323300000002 2024-04-26T03:39:46.500 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-26T03:39:46.500 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-04-26T03:39:46.500 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-04-26T03:39:46.500 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-04-26T03:39:41.816327+0000", 2024-04-26T03:39:46.500 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.2531999999999999e-05 2024-04-26T03:39:46.500 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.500 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:46.500 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.500 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-26T03:39:46.500 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-26T03:39:46.500 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-26T03:39:46.502 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-26T03:39:46.503 INFO:tasks.ceph.osd.14.smithi066.stderr:2024-04-26T03:39:46.501+0000 7f5395f08640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 31765) UID: 0 2024-04-26T03:39:46.603 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-04-26T03:39:46.601+0000 7f610b3a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31816) UID: 0 2024-04-26T03:39:46.620 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:46.620 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-26T03:39:46.620 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-26T03:39:46.620 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:46.629 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-26T03:39:46.704 INFO:tasks.ceph.osd.4.smithi028.stderr:2024-04-26T03:39:46.703+0000 7fd9511b2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31963) UID: 0 2024-04-26T03:39:46.712 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-26T03:39:46.774 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:46.774 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-26T03:39:46.774 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-26T03:39:46.774 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-26T03:39:46.774 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:46.783 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-26T03:39:46.826 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-04-26T03:39:46.825+0000 7f745118d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 31821) UID: 0 2024-04-26T03:39:46.830 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'unique_pool_0' removed 2024-04-26T03:39:46.844 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-04-26T03:39:46.855 INFO:tasks.thrashosds:joining thrashosds 2024-04-26T03:39:46.890 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:46.890 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-26T03:39:46.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-26T03:39:46.891 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-26T03:39:46.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.891 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:20 3.as0 3:5f4e6bd0:::smithi09235267-7:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:46.891 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.485002+0000", 2024-04-26T03:39:46.891 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.402553326000003, 2024-04-26T03:39:46.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017259429, 2024-04-26T03:39:46.891 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:46.891 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:46.891 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:46.891 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:46.891 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:46.891 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:46.892 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 20 2024-04-26T03:39:46.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.892 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:46.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:46.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485002+0000", 2024-04-26T03:39:46.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:46.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:46.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485002+0000", 2024-04-26T03:39:46.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:46.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:46.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485003+0000", 2024-04-26T03:39:46.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6400000000000002e-07 2024-04-26T03:39:46.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:46.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485007+0000", 2024-04-26T03:39:46.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4390000000000002e-06 2024-04-26T03:39:46.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:46.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485015+0000", 2024-04-26T03:39:46.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9389999999999997e-06 2024-04-26T03:39:46.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485243+0000", 2024-04-26T03:39:46.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000227345 2024-04-26T03:39:46.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:46.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.487060+0000", 2024-04-26T03:39:46.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001817117 2024-04-26T03:39:46.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:46.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.487148+0000", 2024-04-26T03:39:46.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8712e-05 2024-04-26T03:39:46.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:46.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.487611+0000", 2024-04-26T03:39:46.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000462223 2024-04-26T03:39:46.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:46.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.491670+0000", 2024-04-26T03:39:46.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004058842 2024-04-26T03:39:46.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:46.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.502181+0000", 2024-04-26T03:39:46.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010511360000000001 2024-04-26T03:39:46.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:46.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.502262+0000", 2024-04-26T03:39:46.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0888000000000002e-05 2024-04-26T03:39:46.897 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:46.897 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:46.897 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:46.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.897 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:21 3.as0 3:5f4e6bd0:::smithi09235267-7:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:46.897 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.485256+0000", 2024-04-26T03:39:46.897 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.402299689000003, 2024-04-26T03:39:46.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026302736, 2024-04-26T03:39:46.898 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:46.898 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:46.898 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:46.898 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:46.898 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:46.898 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:46.898 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 21 2024-04-26T03:39:46.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.898 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:46.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:46.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485256+0000", 2024-04-26T03:39:46.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:46.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:46.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485256+0000", 2024-04-26T03:39:46.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:46.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:46.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485257+0000", 2024-04-26T03:39:46.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4000000000000001e-07 2024-04-26T03:39:46.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:46.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485261+0000", 2024-04-26T03:39:46.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2910000000000001e-06 2024-04-26T03:39:46.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:46.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485269+0000", 2024-04-26T03:39:46.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1780000000000003e-06 2024-04-26T03:39:46.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.485275+0000", 2024-04-26T03:39:46.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2419999999999997e-06 2024-04-26T03:39:46.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:46.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.487806+0000", 2024-04-26T03:39:46.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025310369999999999 2024-04-26T03:39:46.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:46.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.488019+0000", 2024-04-26T03:39:46.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021297800000000001 2024-04-26T03:39:46.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:46.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.502612+0000", 2024-04-26T03:39:46.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014592327 2024-04-26T03:39:46.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:46.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.502711+0000", 2024-04-26T03:39:46.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9432999999999995e-05 2024-04-26T03:39:46.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:46.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.511559+0000", 2024-04-26T03:39:46.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0088476100000000005 2024-04-26T03:39:46.903 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:46.903 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:46.903 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:46.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.903 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:75 3.as0 3:50cd80b4:::smithi09235267-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:46.903 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.524874+0000", 2024-04-26T03:39:46.903 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.362681846999998, 2024-04-26T03:39:46.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017219323000000002, 2024-04-26T03:39:46.904 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:46.904 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:46.904 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:46.904 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:46.904 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:46.904 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:46.904 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 75 2024-04-26T03:39:46.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.904 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:46.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:46.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.524874+0000", 2024-04-26T03:39:46.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:46.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:46.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.524874+0000", 2024-04-26T03:39:46.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:46.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:46.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.524874+0000", 2024-04-26T03:39:46.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9200000000000001e-07 2024-04-26T03:39:46.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:46.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.524877+0000", 2024-04-26T03:39:46.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6079999999999998e-06 2024-04-26T03:39:46.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:46.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.524883+0000", 2024-04-26T03:39:46.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0039999999999996e-06 2024-04-26T03:39:46.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.524888+0000", 2024-04-26T03:39:46.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3070000000000002e-06 2024-04-26T03:39:46.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:46.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.530308+0000", 2024-04-26T03:39:46.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054201850000000001 2024-04-26T03:39:46.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:46.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.530412+0000", 2024-04-26T03:39:46.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010306699999999999 2024-04-26T03:39:46.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:46.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.534668+0000", 2024-04-26T03:39:46.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042560949999999997 2024-04-26T03:39:46.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:46.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.534740+0000", 2024-04-26T03:39:46.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2082000000000005e-05 2024-04-26T03:39:46.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:46.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.542093+0000", 2024-04-26T03:39:46.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0073534830000000001 2024-04-26T03:39:46.909 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:46.909 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:46.909 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:46.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.909 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:105 3.as0 3:527f9dfd:::smithi09235267-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:46.909 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.562626+0000", 2024-04-26T03:39:46.909 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.324930008000003, 2024-04-26T03:39:46.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.023769061000000001, 2024-04-26T03:39:46.910 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:46.910 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:46.910 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:46.910 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:46.910 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:46.910 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:46.910 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 105 2024-04-26T03:39:46.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.910 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:46.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:46.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.562626+0000", 2024-04-26T03:39:46.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:46.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:46.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.562626+0000", 2024-04-26T03:39:46.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:46.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:46.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.562626+0000", 2024-04-26T03:39:46.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7800000000000002e-07 2024-04-26T03:39:46.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:46.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.562629+0000", 2024-04-26T03:39:46.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7779999999999999e-06 2024-04-26T03:39:46.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:46.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.562636+0000", 2024-04-26T03:39:46.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3520000000000001e-06 2024-04-26T03:39:46.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.562641+0000", 2024-04-26T03:39:46.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4789999999999998e-06 2024-04-26T03:39:46.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:46.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.568071+0000", 2024-04-26T03:39:46.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054305719999999998 2024-04-26T03:39:46.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:46.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.568158+0000", 2024-04-26T03:39:46.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6726999999999998e-05 2024-04-26T03:39:46.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:46.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.575494+0000", 2024-04-26T03:39:46.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0073362180000000003 2024-04-26T03:39:46.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:46.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.575571+0000", 2024-04-26T03:39:46.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7134999999999999e-05 2024-04-26T03:39:46.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:46.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.586395+0000", 2024-04-26T03:39:46.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010823322 2024-04-26T03:39:46.915 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:46.915 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:46.915 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:46.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.916 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:117 3.as0 3:58923ed4:::smithi09235267-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:46.916 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.578829+0000", 2024-04-26T03:39:46.916 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.308726890000003, 2024-04-26T03:39:46.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017648347000000002, 2024-04-26T03:39:46.916 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:46.916 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:46.916 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:46.916 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:46.916 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:46.916 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:46.916 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 117 2024-04-26T03:39:46.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.917 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:46.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:46.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.578829+0000", 2024-04-26T03:39:46.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:46.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:46.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.578829+0000", 2024-04-26T03:39:46.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:46.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:46.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.578829+0000", 2024-04-26T03:39:46.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4900000000000001e-07 2024-04-26T03:39:46.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:46.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.578831+0000", 2024-04-26T03:39:46.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0700000000000001e-06 2024-04-26T03:39:46.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:46.919 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.578838+0000", 2024-04-26T03:39:46.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2689999999999996e-06 2024-04-26T03:39:46.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.919 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.919 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.578843+0000", 2024-04-26T03:39:46.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5300000000000004e-06 2024-04-26T03:39:46.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.919 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:46.919 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.586870+0000", 2024-04-26T03:39:46.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0080264919999999997 2024-04-26T03:39:46.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:46.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.586955+0000", 2024-04-26T03:39:46.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5753000000000004e-05 2024-04-26T03:39:46.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:46.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.589337+0000", 2024-04-26T03:39:46.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023813010000000002 2024-04-26T03:39:46.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:46.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.589419+0000", 2024-04-26T03:39:46.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2078999999999998e-05 2024-04-26T03:39:46.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:46.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.596477+0000", 2024-04-26T03:39:46.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0070584039999999999 2024-04-26T03:39:46.921 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:46.921 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:46.921 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:46.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.922 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:150 3.as0 3:5930de0f:::smithi09235267-50:head [append 0~458752,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:46.922 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.634461+0000", 2024-04-26T03:39:46.922 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.253094400999998, 2024-04-26T03:39:46.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017493549000000001, 2024-04-26T03:39:46.922 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:46.922 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:46.922 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:46.922 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:46.922 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:46.922 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:46.922 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 150 2024-04-26T03:39:46.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.923 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:46.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:46.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.634461+0000", 2024-04-26T03:39:46.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:46.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:46.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.634461+0000", 2024-04-26T03:39:46.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:46.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:46.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.634463+0000", 2024-04-26T03:39:46.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.618e-06 2024-04-26T03:39:46.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:46.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.634611+0000", 2024-04-26T03:39:46.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000148214 2024-04-26T03:39:46.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:46.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.634628+0000", 2024-04-26T03:39:46.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7105999999999999e-05 2024-04-26T03:39:46.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.634636+0000", 2024-04-26T03:39:46.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7829999999999999e-06 2024-04-26T03:39:46.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:46.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.634700+0000", 2024-04-26T03:39:46.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4286000000000001e-05 2024-04-26T03:39:46.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:46.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.634839+0000", 2024-04-26T03:39:46.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013907200000000001 2024-04-26T03:39:46.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:46.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.635607+0000", 2024-04-26T03:39:46.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00076726300000000002 2024-04-26T03:39:46.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:46.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.647963+0000", 2024-04-26T03:39:46.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012356525 2024-04-26T03:39:46.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:46.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.651885+0000", 2024-04-26T03:39:46.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039220180000000002 2024-04-26T03:39:46.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:46.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.651955+0000", 2024-04-26T03:39:46.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9664000000000003e-05 2024-04-26T03:39:46.928 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:46.928 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:46.928 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:46.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.928 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:153 3.as0 3:5930de0f:::smithi09235267-50:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:46.928 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.637431+0000", 2024-04-26T03:39:46.928 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.250124839999998, 2024-04-26T03:39:46.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019190048000000001, 2024-04-26T03:39:46.928 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:46.929 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:46.929 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:46.929 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:46.929 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:46.929 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:46.929 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 153 2024-04-26T03:39:46.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.929 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:46.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:46.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637431+0000", 2024-04-26T03:39:46.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:46.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:46.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637431+0000", 2024-04-26T03:39:46.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:46.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:46.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637431+0000", 2024-04-26T03:39:46.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2024-04-26T03:39:46.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:46.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637434+0000", 2024-04-26T03:39:46.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7690000000000001e-06 2024-04-26T03:39:46.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:46.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637439+0000", 2024-04-26T03:39:46.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4190000000000001e-06 2024-04-26T03:39:46.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.637444+0000", 2024-04-26T03:39:46.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6800000000000001e-06 2024-04-26T03:39:46.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:46.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.643745+0000", 2024-04-26T03:39:46.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0063006570000000003 2024-04-26T03:39:46.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:46.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.643818+0000", 2024-04-26T03:39:46.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3558000000000003e-05 2024-04-26T03:39:46.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:46.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.654106+0000", 2024-04-26T03:39:46.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010287436 2024-04-26T03:39:46.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:46.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.654189+0000", 2024-04-26T03:39:46.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2719999999999994e-05 2024-04-26T03:39:46.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:46.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.656621+0000", 2024-04-26T03:39:46.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024323690000000002 2024-04-26T03:39:46.934 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:46.934 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:46.934 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:46.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.934 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:310 3.as0 3:50cd80b4:::smithi09235267-25:head [copy-from ver 3] snapc 6=[6,5] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-26T03:39:46.934 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:59.536617+0000", 2024-04-26T03:39:46.934 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.350938247999999, 2024-04-26T03:39:46.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022665805000000001, 2024-04-26T03:39:46.934 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:46.934 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:46.934 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:46.935 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:46.935 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:46.935 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:46.935 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 310 2024-04-26T03:39:46.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.935 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:46.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:46.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536617+0000", 2024-04-26T03:39:46.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:46.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:46.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536617+0000", 2024-04-26T03:39:46.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:46.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:46.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536620+0000", 2024-04-26T03:39:46.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0540000000000002e-06 2024-04-26T03:39:46.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:46.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536636+0000", 2024-04-26T03:39:46.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6963e-05 2024-04-26T03:39:46.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:46.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536710+0000", 2024-04-26T03:39:46.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3711e-05 2024-04-26T03:39:46.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536719+0000", 2024-04-26T03:39:46.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9889999999999998e-06 2024-04-26T03:39:46.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:46.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536789+0000", 2024-04-26T03:39:46.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9884999999999999e-05 2024-04-26T03:39:46.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-04-26T03:39:46.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536797+0000", 2024-04-26T03:39:46.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5909999999999998e-06 2024-04-26T03:39:46.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:46.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.538587+0000", 2024-04-26T03:39:46.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017899249999999999 2024-04-26T03:39:46.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:46.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.538669+0000", 2024-04-26T03:39:46.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2045000000000005e-05 2024-04-26T03:39:46.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:46.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.549606+0000", 2024-04-26T03:39:46.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010937448000000001 2024-04-26T03:39:46.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:46.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.558484+0000", 2024-04-26T03:39:46.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00887808 2024-04-26T03:39:46.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:46.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.559195+0000", 2024-04-26T03:39:46.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00071067799999999996 2024-04-26T03:39:46.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:46.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.559283+0000", 2024-04-26T03:39:46.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8436000000000005e-05 2024-04-26T03:39:46.941 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:46.941 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:46.941 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:46.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.941 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:311 3.as0 3:50cd80b4:::smithi09235267-25:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e43)", 2024-04-26T03:39:46.941 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:59.536742+0000", 2024-04-26T03:39:46.941 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.350813356000003, 2024-04-26T03:39:46.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021335742000000001, 2024-04-26T03:39:46.941 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:46.942 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:46.942 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:46.942 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:46.942 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:46.942 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:46.942 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 311 2024-04-26T03:39:46.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.942 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:46.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:46.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536742+0000", 2024-04-26T03:39:46.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:46.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:46.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536742+0000", 2024-04-26T03:39:46.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:46.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:46.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536743+0000", 2024-04-26T03:39:46.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5000000000000003e-07 2024-04-26T03:39:46.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:46.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536746+0000", 2024-04-26T03:39:46.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8720000000000002e-06 2024-04-26T03:39:46.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:46.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536771+0000", 2024-04-26T03:39:46.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4949000000000001e-05 2024-04-26T03:39:46.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536776+0000", 2024-04-26T03:39:46.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1089999999999997e-06 2024-04-26T03:39:46.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:46.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536853+0000", 2024-04-26T03:39:46.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7348000000000006e-05 2024-04-26T03:39:46.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-04-26T03:39:46.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.536859+0000", 2024-04-26T03:39:46.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.784e-06 2024-04-26T03:39:46.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:46.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.538806+0000", 2024-04-26T03:39:46.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019471029999999999 2024-04-26T03:39:46.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for blocked object", 2024-04-26T03:39:46.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.538849+0000", 2024-04-26T03:39:46.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2901000000000001e-05 2024-04-26T03:39:46.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:46.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.557853+0000", 2024-04-26T03:39:46.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019003955999999999 2024-04-26T03:39:46.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:46.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.557954+0000", 2024-04-26T03:39:46.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010153399999999999 2024-04-26T03:39:46.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:46.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:59.558078+0000", 2024-04-26T03:39:46.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012363599999999999 2024-04-26T03:39:46.948 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:46.948 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:46.948 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:46.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.949 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:454 3.as0 3:5930de0f:::smithi09235267-50:head [append 0~737280] snapc 8=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:46.949 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:05.710804+0000", 2024-04-26T03:39:46.949 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.176752088999997, 2024-04-26T03:39:46.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017146651999999998, 2024-04-26T03:39:46.949 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:46.949 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:46.949 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:46.949 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:46.949 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:46.949 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:46.949 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 454 2024-04-26T03:39:46.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.950 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:46.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:46.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.710804+0000", 2024-04-26T03:39:46.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:46.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:46.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.710804+0000", 2024-04-26T03:39:46.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:46.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:46.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.710805+0000", 2024-04-26T03:39:46.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8509999999999999e-06 2024-04-26T03:39:46.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:46.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.711242+0000", 2024-04-26T03:39:46.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000436091 2024-04-26T03:39:46.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:46.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.711259+0000", 2024-04-26T03:39:46.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7533e-05 2024-04-26T03:39:46.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.711269+0000", 2024-04-26T03:39:46.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6760000000000002e-06 2024-04-26T03:39:46.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:46.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.711382+0000", 2024-04-26T03:39:46.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011296 2024-04-26T03:39:46.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:46.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.711483+0000", 2024-04-26T03:39:46.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101048 2024-04-26T03:39:46.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:46.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.713492+0000", 2024-04-26T03:39:46.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020094900000000001 2024-04-26T03:39:46.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:46.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.727613+0000", 2024-04-26T03:39:46.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014120819 2024-04-26T03:39:46.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:46.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.727890+0000", 2024-04-26T03:39:46.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027681299999999999 2024-04-26T03:39:46.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:46.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.727950+0000", 2024-04-26T03:39:46.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0371000000000002e-05 2024-04-26T03:39:46.955 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:46.955 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:46.955 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:46.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.955 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:457 3.as0 3:5930de0f:::smithi09235267-50:head [setxattr _header (56) in=7b] snapc 8=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:46.955 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:05.716766+0000", 2024-04-26T03:39:46.955 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.170789892000002, 2024-04-26T03:39:46.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017528595000000001, 2024-04-26T03:39:46.956 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:46.956 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:46.956 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:46.956 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:46.956 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:46.956 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:46.956 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 457 2024-04-26T03:39:46.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.956 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:46.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:46.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716766+0000", 2024-04-26T03:39:46.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:46.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:46.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716766+0000", 2024-04-26T03:39:46.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:46.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:46.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716766+0000", 2024-04-26T03:39:46.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6300000000000005e-07 2024-04-26T03:39:46.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:46.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716770+0000", 2024-04-26T03:39:46.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.377e-06 2024-04-26T03:39:46.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:46.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716776+0000", 2024-04-26T03:39:46.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6850000000000004e-06 2024-04-26T03:39:46.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716782+0000", 2024-04-26T03:39:46.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4779999999999999e-06 2024-04-26T03:39:46.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:46.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.730442+0000", 2024-04-26T03:39:46.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013660284 2024-04-26T03:39:46.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:46.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.730571+0000", 2024-04-26T03:39:46.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012850700000000001 2024-04-26T03:39:46.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:46.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.731245+0000", 2024-04-26T03:39:46.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00067431000000000001 2024-04-26T03:39:46.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:46.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.732606+0000", 2024-04-26T03:39:46.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013605850000000001 2024-04-26T03:39:46.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:46.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.734232+0000", 2024-04-26T03:39:46.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0016260809999999999 2024-04-26T03:39:46.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:46.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.734294+0000", 2024-04-26T03:39:46.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2725000000000007e-05 2024-04-26T03:39:46.961 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:46.961 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:46.961 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:46.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.962 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:458 3.as0 3:5930de0f:::smithi09235267-50:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:46.962 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:05.716790+0000", 2024-04-26T03:39:46.962 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.170766137999998, 2024-04-26T03:39:46.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019815810999999999, 2024-04-26T03:39:46.962 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:46.962 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:46.962 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:46.962 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:46.963 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:46.963 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:46.963 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 458 2024-04-26T03:39:46.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.963 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:46.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:46.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716790+0000", 2024-04-26T03:39:46.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:46.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:46.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716790+0000", 2024-04-26T03:39:46.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:46.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:46.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716790+0000", 2024-04-26T03:39:46.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-04-26T03:39:46.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:46.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716792+0000", 2024-04-26T03:39:46.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3379999999999999e-06 2024-04-26T03:39:46.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:46.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716798+0000", 2024-04-26T03:39:46.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.066e-06 2024-04-26T03:39:46.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716803+0000", 2024-04-26T03:39:46.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.673e-06 2024-04-26T03:39:46.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:46.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.732363+0000", 2024-04-26T03:39:46.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015560218000000001 2024-04-26T03:39:46.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:46.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.732421+0000", 2024-04-26T03:39:46.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7649000000000003e-05 2024-04-26T03:39:46.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:46.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.734770+0000", 2024-04-26T03:39:46.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002348761 2024-04-26T03:39:46.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:46.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.734838+0000", 2024-04-26T03:39:46.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8471000000000002e-05 2024-04-26T03:39:46.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:46.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.736605+0000", 2024-04-26T03:39:46.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017672250000000001 2024-04-26T03:39:46.968 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:46.968 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:46.968 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:46.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.968 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:760 3.as0 3:58923ed4:::smithi09235267-39:head [append 0~704512] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:46.968 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:20.745444+0000", 2024-04-26T03:39:46.968 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.142111659000001, 2024-04-26T03:39:46.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021214666, 2024-04-26T03:39:46.968 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:46.968 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:46.969 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:46.969 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:46.969 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:46.969 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:46.969 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 760 2024-04-26T03:39:46.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.969 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:46.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:46.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.745444+0000", 2024-04-26T03:39:46.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:46.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:46.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.745444+0000", 2024-04-26T03:39:46.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:46.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:46.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.745446+0000", 2024-04-26T03:39:46.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.858e-06 2024-04-26T03:39:46.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:46.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.745741+0000", 2024-04-26T03:39:46.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029482400000000001 2024-04-26T03:39:46.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:46.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.745774+0000", 2024-04-26T03:39:46.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3124000000000003e-05 2024-04-26T03:39:46.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.745798+0000", 2024-04-26T03:39:46.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4528999999999998e-05 2024-04-26T03:39:46.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:46.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.745885+0000", 2024-04-26T03:39:46.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6878000000000004e-05 2024-04-26T03:39:46.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-04-26T03:39:46.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.745910+0000", 2024-04-26T03:39:46.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4935000000000001e-05 2024-04-26T03:39:46.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:46.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.748635+0000", 2024-04-26T03:39:46.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0027248569999999998 2024-04-26T03:39:46.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:46.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.748702+0000", 2024-04-26T03:39:46.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7076000000000006e-05 2024-04-26T03:39:46.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:46.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.749617+0000", 2024-04-26T03:39:46.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00091456800000000004 2024-04-26T03:39:46.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:46.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.765879+0000", 2024-04-26T03:39:46.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016262566999999999 2024-04-26T03:39:46.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:46.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.766500+0000", 2024-04-26T03:39:46.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00062033100000000001 2024-04-26T03:39:46.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:46.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.766659+0000", 2024-04-26T03:39:46.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015911899999999999 2024-04-26T03:39:46.975 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:46.975 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:46.975 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:46.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.975 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:761 3.as0 3:58923ed4:::smithi09235267-39:head [append 0~475136] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:46.976 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:20.748282+0000", 2024-04-26T03:39:46.976 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.139274067999999, 2024-04-26T03:39:46.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019043616999999999, 2024-04-26T03:39:46.976 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:46.976 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:46.976 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:46.976 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:46.976 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:46.976 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:46.976 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 761 2024-04-26T03:39:46.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.976 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:46.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:46.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.748282+0000", 2024-04-26T03:39:46.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:46.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:46.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.748282+0000", 2024-04-26T03:39:46.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:46.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:46.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.748283+0000", 2024-04-26T03:39:46.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.133e-06 2024-04-26T03:39:46.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:46.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.748542+0000", 2024-04-26T03:39:46.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00025954400000000001 2024-04-26T03:39:46.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:46.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.748555+0000", 2024-04-26T03:39:46.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2483000000000001e-05 2024-04-26T03:39:46.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.748564+0000", 2024-04-26T03:39:46.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8109999999999992e-06 2024-04-26T03:39:46.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:46.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.753862+0000", 2024-04-26T03:39:46.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052979580000000002 2024-04-26T03:39:46.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:46.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.753977+0000", 2024-04-26T03:39:46.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115072 2024-04-26T03:39:46.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:46.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.754660+0000", 2024-04-26T03:39:46.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00068356000000000005 2024-04-26T03:39:46.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:46.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.765984+0000", 2024-04-26T03:39:46.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011324021 2024-04-26T03:39:46.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:46.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.767207+0000", 2024-04-26T03:39:46.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0012224340000000001 2024-04-26T03:39:46.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:46.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.767325+0000", 2024-04-26T03:39:46.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000118601 2024-04-26T03:39:46.981 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:46.982 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:46.982 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:46.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.982 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:763 3.as0 3:58923ed4:::smithi09235267-39:head [append 0~507904] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:46.982 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:20.754960+0000", 2024-04-26T03:39:46.982 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.132595224999999, 2024-04-26T03:39:46.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020013485000000001, 2024-04-26T03:39:46.982 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:46.982 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:46.982 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:46.982 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:46.983 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:46.983 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:46.983 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 763 2024-04-26T03:39:46.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.983 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:46.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:46.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.754960+0000", 2024-04-26T03:39:46.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:46.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:46.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.754960+0000", 2024-04-26T03:39:46.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:46.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:46.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.754962+0000", 2024-04-26T03:39:46.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.66e-06 2024-04-26T03:39:46.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:46.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.755204+0000", 2024-04-26T03:39:46.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024158200000000001 2024-04-26T03:39:46.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:46.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.755216+0000", 2024-04-26T03:39:46.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1888999999999999e-05 2024-04-26T03:39:46.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.755223+0000", 2024-04-26T03:39:46.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6529999999999994e-06 2024-04-26T03:39:46.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:46.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.767902+0000", 2024-04-26T03:39:46.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012678833 2024-04-26T03:39:46.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:46.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.768010+0000", 2024-04-26T03:39:46.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107639 2024-04-26T03:39:46.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:46.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.768777+0000", 2024-04-26T03:39:46.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00076683800000000005 2024-04-26T03:39:46.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:46.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.774647+0000", 2024-04-26T03:39:46.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058705609999999998 2024-04-26T03:39:46.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:46.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.774790+0000", 2024-04-26T03:39:46.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000143017 2024-04-26T03:39:46.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:46.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.774974+0000", 2024-04-26T03:39:46.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018381300000000001 2024-04-26T03:39:46.988 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:46.988 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:46.988 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:46.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.988 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:764 3.as0 3:58923ed4:::smithi09235267-39:head [setxattr _header (57) in=7b] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:46.989 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:20.755241+0000", 2024-04-26T03:39:46.989 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.132314964999999, 2024-04-26T03:39:46.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020359479, 2024-04-26T03:39:46.989 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:46.989 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:46.989 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:46.989 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:46.989 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:46.989 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:46.989 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 764 2024-04-26T03:39:46.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.989 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:46.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:46.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.755241+0000", 2024-04-26T03:39:46.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:46.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:46.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.755241+0000", 2024-04-26T03:39:46.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:46.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:46.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.755241+0000", 2024-04-26T03:39:46.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3399999999999999e-07 2024-04-26T03:39:46.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:46.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.755245+0000", 2024-04-26T03:39:46.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5470000000000001e-06 2024-04-26T03:39:46.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:46.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.755251+0000", 2024-04-26T03:39:46.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2389999999999999e-06 2024-04-26T03:39:46.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.755257+0000", 2024-04-26T03:39:46.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3680000000000001e-06 2024-04-26T03:39:46.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:46.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.773298+0000", 2024-04-26T03:39:46.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018041303000000002 2024-04-26T03:39:46.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:46.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.773404+0000", 2024-04-26T03:39:46.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010583500000000001 2024-04-26T03:39:46.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:46.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.773767+0000", 2024-04-26T03:39:46.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000363786 2024-04-26T03:39:46.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:46.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.774979+0000", 2024-04-26T03:39:46.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0012112360000000001 2024-04-26T03:39:46.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:46.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.775421+0000", 2024-04-26T03:39:46.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00044187799999999999 2024-04-26T03:39:46.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:46.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.775600+0000", 2024-04-26T03:39:46.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000179653 2024-04-26T03:39:46.995 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:46.995 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:46.995 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:46.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.995 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:765 3.as0 3:58923ed4:::smithi09235267-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:46.995 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:20.755268+0000", 2024-04-26T03:39:46.995 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.132287516000002, 2024-04-26T03:39:46.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021987618, 2024-04-26T03:39:46.995 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:46.996 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:46.996 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:46.996 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:46.996 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:46.996 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:46.996 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 765 2024-04-26T03:39:46.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.996 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:46.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:46.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.755268+0000", 2024-04-26T03:39:46.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:46.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:46.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.755268+0000", 2024-04-26T03:39:46.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:46.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:46.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.755269+0000", 2024-04-26T03:39:46.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5400000000000002e-07 2024-04-26T03:39:46.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:46.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.755271+0000", 2024-04-26T03:39:46.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7070000000000001e-06 2024-04-26T03:39:46.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:46.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.755280+0000", 2024-04-26T03:39:46.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4519999999999997e-06 2024-04-26T03:39:46.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:46.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.755285+0000", 2024-04-26T03:39:46.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6459999999999998e-06 2024-04-26T03:39:46.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:46.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.774477+0000", 2024-04-26T03:39:46.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019191144 2024-04-26T03:39:46.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:46.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.774591+0000", 2024-04-26T03:39:46.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000114602 2024-04-26T03:39:46.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:46.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:46.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.776170+0000", 2024-04-26T03:39:47.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0015785549999999999 2024-04-26T03:39:47.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:47.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.776262+0000", 2024-04-26T03:39:47.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1823000000000005e-05 2024-04-26T03:39:47.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:47.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.777256+0000", 2024-04-26T03:39:47.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000994235 2024-04-26T03:39:47.001 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.001 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.001 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.001 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:790 3.as0 3:50cd80b4:::smithi09235267-25:head [append 0~786432] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:47.001 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:20.850132+0000", 2024-04-26T03:39:47.001 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.037423230000002, 2024-04-26T03:39:47.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019444847000000001, 2024-04-26T03:39:47.001 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:47.001 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:47.001 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:47.001 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:47.002 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:47.002 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:47.002 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 790 2024-04-26T03:39:47.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.002 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:47.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:47.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.850132+0000", 2024-04-26T03:39:47.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:47.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.850132+0000", 2024-04-26T03:39:47.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:47.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.850134+0000", 2024-04-26T03:39:47.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7409999999999999e-06 2024-04-26T03:39:47.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:47.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.850299+0000", 2024-04-26T03:39:47.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000165119 2024-04-26T03:39:47.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:47.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.850316+0000", 2024-04-26T03:39:47.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6419999999999999e-05 2024-04-26T03:39:47.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:47.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.850324+0000", 2024-04-26T03:39:47.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6670000000000002e-06 2024-04-26T03:39:47.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.850375+0000", 2024-04-26T03:39:47.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0498999999999999e-05 2024-04-26T03:39:47.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:47.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.850439+0000", 2024-04-26T03:39:47.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4017999999999996e-05 2024-04-26T03:39:47.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:47.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.851224+0000", 2024-04-26T03:39:47.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00078535300000000003 2024-04-26T03:39:47.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:47.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.869292+0000", 2024-04-26T03:39:47.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018067295000000001 2024-04-26T03:39:47.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:47.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.869498+0000", 2024-04-26T03:39:47.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020635599999999999 2024-04-26T03:39:47.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:47.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.869577+0000", 2024-04-26T03:39:47.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9379e-05 2024-04-26T03:39:47.007 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.007 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.007 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.008 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:854 3.as0 3:50cd80b4:::smithi09235267-25:head [read 0~5373952,checksum] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-04-26T03:39:47.008 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:27.699567+0000", 2024-04-26T03:39:47.008 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.187988297, 2024-04-26T03:39:47.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017245364999999999, 2024-04-26T03:39:47.008 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:47.008 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:47.008 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:47.008 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:47.008 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:47.008 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:47.008 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 854 2024-04-26T03:39:47.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.009 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:47.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:47.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:27.699567+0000", 2024-04-26T03:39:47.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:47.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:27.699567+0000", 2024-04-26T03:39:47.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:47.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:27.699568+0000", 2024-04-26T03:39:47.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-07 2024-04-26T03:39:47.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:47.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:27.699572+0000", 2024-04-26T03:39:47.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.45e-06 2024-04-26T03:39:47.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:47.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:27.699579+0000", 2024-04-26T03:39:47.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5050000000000001e-06 2024-04-26T03:39:47.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:47.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:27.699585+0000", 2024-04-26T03:39:47.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7450000000000001e-06 2024-04-26T03:39:47.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:27.699634+0000", 2024-04-26T03:39:47.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9103000000000001e-05 2024-04-26T03:39:47.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-04-26T03:39:47.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:27.699643+0000", 2024-04-26T03:39:47.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1730000000000001e-06 2024-04-26T03:39:47.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:27.701305+0000", 2024-04-26T03:39:47.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001661654 2024-04-26T03:39:47.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:47.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:27.701372+0000", 2024-04-26T03:39:47.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7296000000000001e-05 2024-04-26T03:39:47.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:47.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:27.716813+0000", 2024-04-26T03:39:47.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015440729 2024-04-26T03:39:47.013 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.014 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.014 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.014 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:855 3.as0 3:50cd80b4:::smithi09235267-25:head [read 0~5373952,checksum] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e72)", 2024-04-26T03:39:47.014 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:27.699595+0000", 2024-04-26T03:39:47.014 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.187960407999999, 2024-04-26T03:39:47.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020112682, 2024-04-26T03:39:47.014 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:47.014 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:47.014 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:47.015 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:47.015 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:47.015 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:47.015 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 855 2024-04-26T03:39:47.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.015 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:47.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:47.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:27.699595+0000", 2024-04-26T03:39:47.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:47.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:27.699595+0000", 2024-04-26T03:39:47.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:47.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:27.699596+0000", 2024-04-26T03:39:47.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0600000000000001e-07 2024-04-26T03:39:47.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:47.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:27.699599+0000", 2024-04-26T03:39:47.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.813e-06 2024-04-26T03:39:47.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:47.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:27.699605+0000", 2024-04-26T03:39:47.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1410000000000002e-06 2024-04-26T03:39:47.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:47.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:27.699610+0000", 2024-04-26T03:39:47.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0259999999999998e-06 2024-04-26T03:39:47.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:27.699660+0000", 2024-04-26T03:39:47.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9972999999999997e-05 2024-04-26T03:39:47.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-04-26T03:39:47.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:27.699663+0000", 2024-04-26T03:39:47.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1669999999999999e-06 2024-04-26T03:39:47.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:27.704528+0000", 2024-04-26T03:39:47.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0048650680000000002 2024-04-26T03:39:47.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:47.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:27.704594+0000", 2024-04-26T03:39:47.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5790999999999994e-05 2024-04-26T03:39:47.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:47.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:27.719708+0000", 2024-04-26T03:39:47.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015114297 2024-04-26T03:39:47.020 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.020 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.021 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.021 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.021 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.021 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:47.022 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-26T03:39:47.165 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:47.165 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-26T03:39:47.165 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-26T03:39:47.165 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:47.174 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-26T03:39:47.277 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:47.277 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-26T03:39:47.277 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-26T03:39:47.277 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-26T03:39:47.277 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:47.285 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-26T03:39:47.387 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:47.388 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-26T03:39:47.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-26T03:39:47.388 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-26T03:39:47.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.388 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:45 3.ds0 3:b4337c88:::smithi09235267-15:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:47.388 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.498959+0000", 2024-04-26T03:39:47.388 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.886262228, 2024-04-26T03:39:47.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.028116775, 2024-04-26T03:39:47.388 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:47.388 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:47.388 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:47.388 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:47.389 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:47.389 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:47.389 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 45 2024-04-26T03:39:47.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.389 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:47.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:47.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498959+0000", 2024-04-26T03:39:47.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:47.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498959+0000", 2024-04-26T03:39:47.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:47.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498959+0000", 2024-04-26T03:39:47.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.58e-07 2024-04-26T03:39:47.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:47.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498963+0000", 2024-04-26T03:39:47.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2449999999999998e-06 2024-04-26T03:39:47.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:47.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498970+0000", 2024-04-26T03:39:47.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5780000000000001e-06 2024-04-26T03:39:47.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:47.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498976+0000", 2024-04-26T03:39:47.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6400000000000002e-06 2024-04-26T03:39:47.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.504199+0000", 2024-04-26T03:39:47.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052229190000000003 2024-04-26T03:39:47.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:47.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.504274+0000", 2024-04-26T03:39:47.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5187999999999999e-05 2024-04-26T03:39:47.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.512941+0000", 2024-04-26T03:39:47.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0086671879999999993 2024-04-26T03:39:47.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:47.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.513001+0000", 2024-04-26T03:39:47.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0204000000000002e-05 2024-04-26T03:39:47.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:47.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.527076+0000", 2024-04-26T03:39:47.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014074355 2024-04-26T03:39:47.393 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.393 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.393 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.393 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:52 3.ds0 3:b895fe71:::smithi09235267-18:head [append 0~294912,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:47.393 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.505560+0000", 2024-04-26T03:39:47.393 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.879661229, 2024-04-26T03:39:47.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016117464000000001, 2024-04-26T03:39:47.394 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:47.394 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:47.394 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:47.394 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:47.394 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:47.394 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:47.394 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 52 2024-04-26T03:39:47.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.394 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:47.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:47.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505560+0000", 2024-04-26T03:39:47.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:47.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505560+0000", 2024-04-26T03:39:47.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:47.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505562+0000", 2024-04-26T03:39:47.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0729999999999999e-06 2024-04-26T03:39:47.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:47.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505753+0000", 2024-04-26T03:39:47.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019100500000000001 2024-04-26T03:39:47.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:47.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505768+0000", 2024-04-26T03:39:47.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5213e-05 2024-04-26T03:39:47.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:47.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505776+0000", 2024-04-26T03:39:47.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7279999999999995e-06 2024-04-26T03:39:47.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505830+0000", 2024-04-26T03:39:47.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3749999999999999e-05 2024-04-26T03:39:47.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:47.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505998+0000", 2024-04-26T03:39:47.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000168259 2024-04-26T03:39:47.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:47.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.506585+0000", 2024-04-26T03:39:47.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00058713299999999999 2024-04-26T03:39:47.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:47.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.511525+0000", 2024-04-26T03:39:47.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004939789 2024-04-26T03:39:47.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:47.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.521601+0000", 2024-04-26T03:39:47.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010075766 2024-04-26T03:39:47.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:47.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.521677+0000", 2024-04-26T03:39:47.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6748000000000004e-05 2024-04-26T03:39:47.400 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.400 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.400 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.401 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:53 3.ds0 3:b895fe71:::smithi09235267-18:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:47.401 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.505792+0000", 2024-04-26T03:39:47.401 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.879428789999999, 2024-04-26T03:39:47.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016119725000000001, 2024-04-26T03:39:47.401 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:47.401 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:47.401 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:47.402 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:47.402 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:47.402 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:47.402 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 53 2024-04-26T03:39:47.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.402 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:47.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:47.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505792+0000", 2024-04-26T03:39:47.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:47.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505792+0000", 2024-04-26T03:39:47.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:47.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505793+0000", 2024-04-26T03:39:47.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5400000000000001e-07 2024-04-26T03:39:47.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:47.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505806+0000", 2024-04-26T03:39:47.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2733e-05 2024-04-26T03:39:47.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:47.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505814+0000", 2024-04-26T03:39:47.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0260000000000007e-06 2024-04-26T03:39:47.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:47.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505820+0000", 2024-04-26T03:39:47.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8769999999999998e-06 2024-04-26T03:39:47.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.509659+0000", 2024-04-26T03:39:47.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0038395149999999999 2024-04-26T03:39:47.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:47.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.509764+0000", 2024-04-26T03:39:47.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105159 2024-04-26T03:39:47.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:47.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.510189+0000", 2024-04-26T03:39:47.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00042430899999999999 2024-04-26T03:39:47.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:47.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.511526+0000", 2024-04-26T03:39:47.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001337934 2024-04-26T03:39:47.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:47.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.521837+0000", 2024-04-26T03:39:47.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010310503 2024-04-26T03:39:47.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:47.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.521912+0000", 2024-04-26T03:39:47.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5014999999999999e-05 2024-04-26T03:39:47.409 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.409 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.409 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.410 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:54 3.ds0 3:b895fe71:::smithi09235267-18:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:47.410 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.505829+0000", 2024-04-26T03:39:47.410 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.879391949000002, 2024-04-26T03:39:47.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021406122, 2024-04-26T03:39:47.410 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:47.410 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:47.410 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:47.410 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:47.410 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:47.410 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:47.411 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 54 2024-04-26T03:39:47.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.411 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:47.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:47.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505829+0000", 2024-04-26T03:39:47.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:47.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505829+0000", 2024-04-26T03:39:47.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:47.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505830+0000", 2024-04-26T03:39:47.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2e-07 2024-04-26T03:39:47.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:47.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505832+0000", 2024-04-26T03:39:47.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.509e-06 2024-04-26T03:39:47.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:47.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505839+0000", 2024-04-26T03:39:47.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9110000000000004e-06 2024-04-26T03:39:47.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:47.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.505844+0000", 2024-04-26T03:39:47.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2109999999999997e-06 2024-04-26T03:39:47.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.510372+0000", 2024-04-26T03:39:47.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045275990000000002 2024-04-26T03:39:47.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:47.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.510419+0000", 2024-04-26T03:39:47.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7524000000000001e-05 2024-04-26T03:39:47.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.522258+0000", 2024-04-26T03:39:47.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011838448999999999 2024-04-26T03:39:47.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:47.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.522334+0000", 2024-04-26T03:39:47.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6134e-05 2024-04-26T03:39:47.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:47.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.527235+0000", 2024-04-26T03:39:47.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049012650000000001 2024-04-26T03:39:47.417 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.417 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.417 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.417 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:64 3.ds0 3:b3f958c3:::smithi09235267-22:head [append 0~360448,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:47.418 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.515665+0000", 2024-04-26T03:39:47.418 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.869555753999997, 2024-04-26T03:39:47.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.023538667999999999, 2024-04-26T03:39:47.418 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:47.418 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:47.418 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:47.418 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:47.418 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:47.418 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:47.418 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 64 2024-04-26T03:39:47.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.418 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:47.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:47.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515665+0000", 2024-04-26T03:39:47.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:47.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515665+0000", 2024-04-26T03:39:47.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:47.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515667+0000", 2024-04-26T03:39:47.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.652e-06 2024-04-26T03:39:47.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:47.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515857+0000", 2024-04-26T03:39:47.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000190447 2024-04-26T03:39:47.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:47.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515869+0000", 2024-04-26T03:39:47.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1450999999999999e-05 2024-04-26T03:39:47.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:47.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515876+0000", 2024-04-26T03:39:47.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2024-04-26T03:39:47.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.527263+0000", 2024-04-26T03:39:47.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011387061 2024-04-26T03:39:47.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:47.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.527389+0000", 2024-04-26T03:39:47.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000126172 2024-04-26T03:39:47.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:47.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.528593+0000", 2024-04-26T03:39:47.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00120386 2024-04-26T03:39:47.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:47.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.533773+0000", 2024-04-26T03:39:47.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005179973 2024-04-26T03:39:47.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:47.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.539131+0000", 2024-04-26T03:39:47.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053582279999999996 2024-04-26T03:39:47.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:47.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.539204+0000", 2024-04-26T03:39:47.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2809e-05 2024-04-26T03:39:47.425 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.425 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.425 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.426 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:65 3.ds0 3:b3f958c3:::smithi09235267-22:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:47.426 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.515889+0000", 2024-04-26T03:39:47.426 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.869332114000002, 2024-04-26T03:39:47.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.023486671000000001, 2024-04-26T03:39:47.426 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:47.426 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:47.426 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:47.426 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:47.427 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:47.427 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:47.427 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 65 2024-04-26T03:39:47.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.427 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:47.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:47.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515889+0000", 2024-04-26T03:39:47.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:47.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515889+0000", 2024-04-26T03:39:47.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:47.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515890+0000", 2024-04-26T03:39:47.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.13e-07 2024-04-26T03:39:47.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:47.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515894+0000", 2024-04-26T03:39:47.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.386e-06 2024-04-26T03:39:47.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:47.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515903+0000", 2024-04-26T03:39:47.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.166e-06 2024-04-26T03:39:47.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:47.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515909+0000", 2024-04-26T03:39:47.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.412e-06 2024-04-26T03:39:47.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.532799+0000", 2024-04-26T03:39:47.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016889990000000001 2024-04-26T03:39:47.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:47.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.532885+0000", 2024-04-26T03:39:47.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6336999999999997e-05 2024-04-26T03:39:47.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:47.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.533334+0000", 2024-04-26T03:39:47.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00044950799999999999 2024-04-26T03:39:47.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:47.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.538492+0000", 2024-04-26T03:39:47.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051571389999999998 2024-04-26T03:39:47.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:47.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.539303+0000", 2024-04-26T03:39:47.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000811325 2024-04-26T03:39:47.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:47.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.539376+0000", 2024-04-26T03:39:47.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2794999999999997e-05 2024-04-26T03:39:47.434 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.434 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.434 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.435 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:66 3.ds0 3:b3f958c3:::smithi09235267-22:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:47.435 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.515917+0000", 2024-04-26T03:39:47.435 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.869304538999998, 2024-04-26T03:39:47.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.024757049, 2024-04-26T03:39:47.435 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:47.435 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:47.435 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:47.435 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:47.435 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:47.436 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:47.436 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 66 2024-04-26T03:39:47.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.436 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:47.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:47.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515917+0000", 2024-04-26T03:39:47.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:47.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515917+0000", 2024-04-26T03:39:47.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:47.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515917+0000", 2024-04-26T03:39:47.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8699999999999995e-07 2024-04-26T03:39:47.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:47.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515920+0000", 2024-04-26T03:39:47.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.055e-06 2024-04-26T03:39:47.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:47.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515927+0000", 2024-04-26T03:39:47.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5150000000000003e-06 2024-04-26T03:39:47.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:47.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515936+0000", 2024-04-26T03:39:47.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7920000000000004e-06 2024-04-26T03:39:47.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.533517+0000", 2024-04-26T03:39:47.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017580535000000001 2024-04-26T03:39:47.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:47.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.533598+0000", 2024-04-26T03:39:47.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1354999999999996e-05 2024-04-26T03:39:47.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.539706+0000", 2024-04-26T03:39:47.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061075569999999996 2024-04-26T03:39:47.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:47.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.539758+0000", 2024-04-26T03:39:47.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2500000000000002e-05 2024-04-26T03:39:47.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:47.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.540674+0000", 2024-04-26T03:39:47.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00091525299999999999 2024-04-26T03:39:47.442 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.442 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.442 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.443 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:607 3.ds0 3:b4337c88:::smithi09235267-15:head [append 0~737280] snapc d=[d,c,b,a] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-26T03:39:47.443 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:11.722465+0000", 2024-04-26T03:39:47.443 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.662755961000002, 2024-04-26T03:39:47.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019589842999999999, 2024-04-26T03:39:47.443 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:47.443 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:47.443 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:47.443 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:47.443 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:47.444 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:47.444 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 607 2024-04-26T03:39:47.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.444 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:47.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:47.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.722465+0000", 2024-04-26T03:39:47.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:47.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.722465+0000", 2024-04-26T03:39:47.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:47.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.722467+0000", 2024-04-26T03:39:47.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.553e-06 2024-04-26T03:39:47.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:47.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.722788+0000", 2024-04-26T03:39:47.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00032146600000000002 2024-04-26T03:39:47.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:47.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.722805+0000", 2024-04-26T03:39:47.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6534000000000001e-05 2024-04-26T03:39:47.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:47.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.722815+0000", 2024-04-26T03:39:47.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0062e-05 2024-04-26T03:39:47.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.722890+0000", 2024-04-26T03:39:47.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5012000000000006e-05 2024-04-26T03:39:47.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:47.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.723013+0000", 2024-04-26T03:39:47.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000123693 2024-04-26T03:39:47.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:47.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.724487+0000", 2024-04-26T03:39:47.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001473427 2024-04-26T03:39:47.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:47.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.741651+0000", 2024-04-26T03:39:47.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017163754999999999 2024-04-26T03:39:47.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:47.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.741886+0000", 2024-04-26T03:39:47.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0002354 2024-04-26T03:39:47.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:47.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.742055+0000", 2024-04-26T03:39:47.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016894100000000001 2024-04-26T03:39:47.450 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.451 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.451 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.451 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:608 3.ds0 3:b4337c88:::smithi09235267-15:head [append 0~557056] snapc d=[d,c,b,a] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-26T03:39:47.451 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:11.726593+0000", 2024-04-26T03:39:47.451 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.658627725999999, 2024-04-26T03:39:47.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019055236999999999, 2024-04-26T03:39:47.451 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:47.451 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:47.451 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:47.452 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:47.452 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:47.452 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:47.452 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 608 2024-04-26T03:39:47.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.452 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:47.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:47.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.726593+0000", 2024-04-26T03:39:47.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:47.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.726593+0000", 2024-04-26T03:39:47.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:47.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.726596+0000", 2024-04-26T03:39:47.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3649999999999998e-06 2024-04-26T03:39:47.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:47.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.726927+0000", 2024-04-26T03:39:47.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00033128999999999998 2024-04-26T03:39:47.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:47.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.726950+0000", 2024-04-26T03:39:47.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.317e-05 2024-04-26T03:39:47.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:47.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.726961+0000", 2024-04-26T03:39:47.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0387999999999999e-05 2024-04-26T03:39:47.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.730805+0000", 2024-04-26T03:39:47.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003843993 2024-04-26T03:39:47.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:47.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.730917+0000", 2024-04-26T03:39:47.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011274599999999999 2024-04-26T03:39:47.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:47.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.731823+0000", 2024-04-26T03:39:47.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00090538700000000001 2024-04-26T03:39:47.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:47.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.742998+0000", 2024-04-26T03:39:47.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011175144 2024-04-26T03:39:47.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:47.458 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.745560+0000", 2024-04-26T03:39:47.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025625660000000001 2024-04-26T03:39:47.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.458 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:47.458 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.745649+0000", 2024-04-26T03:39:47.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8188000000000003e-05 2024-04-26T03:39:47.458 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.458 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.458 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.459 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:610 3.ds0 3:b4337c88:::smithi09235267-15:head [setxattr _header (57) in=7b] snapc d=[d,c,b,a] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-26T03:39:47.459 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:11.731469+0000", 2024-04-26T03:39:47.459 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.653752281000003, 2024-04-26T03:39:47.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015326482000000001, 2024-04-26T03:39:47.459 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:47.459 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:47.459 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:47.459 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:47.459 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:47.460 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:47.460 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 610 2024-04-26T03:39:47.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.460 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:47.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:47.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.731469+0000", 2024-04-26T03:39:47.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:47.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.731469+0000", 2024-04-26T03:39:47.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:47.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.731469+0000", 2024-04-26T03:39:47.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2800000000000002e-07 2024-04-26T03:39:47.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:47.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.731473+0000", 2024-04-26T03:39:47.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.456e-06 2024-04-26T03:39:47.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:47.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.731478+0000", 2024-04-26T03:39:47.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9860000000000002e-06 2024-04-26T03:39:47.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:47.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.731481+0000", 2024-04-26T03:39:47.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2590000000000001e-06 2024-04-26T03:39:47.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.742103+0000", 2024-04-26T03:39:47.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010621680999999999 2024-04-26T03:39:47.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:47.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.742180+0000", 2024-04-26T03:39:47.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7392999999999996e-05 2024-04-26T03:39:47.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:47.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.742694+0000", 2024-04-26T03:39:47.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000513654 2024-04-26T03:39:47.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:47.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.745660+0000", 2024-04-26T03:39:47.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002966836 2024-04-26T03:39:47.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:47.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.746708+0000", 2024-04-26T03:39:47.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010471619999999999 2024-04-26T03:39:47.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:47.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.746795+0000", 2024-04-26T03:39:47.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7627000000000007e-05 2024-04-26T03:39:47.466 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.466 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.466 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.467 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:611 3.ds0 3:b4337c88:::smithi09235267-15:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-04-26T03:39:47.467 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:11.731485+0000", 2024-04-26T03:39:47.467 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.653735580999999, 2024-04-26T03:39:47.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017052068, 2024-04-26T03:39:47.467 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:47.467 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:47.467 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:47.467 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:47.467 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:47.467 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:47.468 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 611 2024-04-26T03:39:47.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.468 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:47.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:47.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.731485+0000", 2024-04-26T03:39:47.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:47.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.731485+0000", 2024-04-26T03:39:47.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:47.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.731486+0000", 2024-04-26T03:39:47.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6899999999999999e-07 2024-04-26T03:39:47.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:47.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.731487+0000", 2024-04-26T03:39:47.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.44e-06 2024-04-26T03:39:47.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:47.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.731492+0000", 2024-04-26T03:39:47.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9930000000000003e-06 2024-04-26T03:39:47.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:47.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.731495+0000", 2024-04-26T03:39:47.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.022e-06 2024-04-26T03:39:47.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.743270+0000", 2024-04-26T03:39:47.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011774316 2024-04-26T03:39:47.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:47.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.743323+0000", 2024-04-26T03:39:47.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3463e-05 2024-04-26T03:39:47.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.747168+0000", 2024-04-26T03:39:47.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003844859 2024-04-26T03:39:47.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:47.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.747255+0000", 2024-04-26T03:39:47.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7052000000000006e-05 2024-04-26T03:39:47.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:47.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.748538+0000", 2024-04-26T03:39:47.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001282654 2024-04-26T03:39:47.473 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.473 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.473 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.474 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:690 3.ds0 3:b895fe71:::smithi09235267-18:head [append 0~720896] snapc f=[f,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-04-26T03:39:47.474 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:15.944310+0000", 2024-04-26T03:39:47.474 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.440911501999999, 2024-04-26T03:39:47.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016189867, 2024-04-26T03:39:47.474 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:47.474 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:47.474 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:47.474 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:47.474 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:47.474 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:47.475 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 690 2024-04-26T03:39:47.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.475 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:47.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:47.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.944310+0000", 2024-04-26T03:39:47.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:47.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.944310+0000", 2024-04-26T03:39:47.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:47.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.944311+0000", 2024-04-26T03:39:47.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2500000000000001e-06 2024-04-26T03:39:47.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:47.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.944635+0000", 2024-04-26T03:39:47.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000324271 2024-04-26T03:39:47.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:47.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.944644+0000", 2024-04-26T03:39:47.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1670000000000005e-06 2024-04-26T03:39:47.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:47.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.944651+0000", 2024-04-26T03:39:47.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4300000000000003e-06 2024-04-26T03:39:47.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.944689+0000", 2024-04-26T03:39:47.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8272000000000002e-05 2024-04-26T03:39:47.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:47.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.944788+0000", 2024-04-26T03:39:47.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8916e-05 2024-04-26T03:39:47.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:47.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.945668+0000", 2024-04-26T03:39:47.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00088055000000000002 2024-04-26T03:39:47.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:47.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.960314+0000", 2024-04-26T03:39:47.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014645467000000001 2024-04-26T03:39:47.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:47.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.960414+0000", 2024-04-26T03:39:47.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.98e-05 2024-04-26T03:39:47.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:47.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.960499+0000", 2024-04-26T03:39:47.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5743999999999998e-05 2024-04-26T03:39:47.481 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.481 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.481 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.482 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:694 3.ds0 3:b895fe71:::smithi09235267-18:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e60)", 2024-04-26T03:39:47.482 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:15.950694+0000", 2024-04-26T03:39:47.482 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.434527201000002, 2024-04-26T03:39:47.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018244219999999998, 2024-04-26T03:39:47.482 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:47.482 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:47.482 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:47.482 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:47.482 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:47.483 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:47.483 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 694 2024-04-26T03:39:47.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.483 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:47.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:47.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.950694+0000", 2024-04-26T03:39:47.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:47.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.950694+0000", 2024-04-26T03:39:47.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:47.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.950694+0000", 2024-04-26T03:39:47.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.82e-07 2024-04-26T03:39:47.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:47.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.950697+0000", 2024-04-26T03:39:47.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.559e-06 2024-04-26T03:39:47.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:47.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.950704+0000", 2024-04-26T03:39:47.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.917e-06 2024-04-26T03:39:47.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.486 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:47.486 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.950713+0000", 2024-04-26T03:39:47.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5419999999999994e-06 2024-04-26T03:39:47.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.486 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.486 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.962925+0000", 2024-04-26T03:39:47.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012211926999999999 2024-04-26T03:39:47.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.487 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:47.487 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.963013+0000", 2024-04-26T03:39:47.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7791e-05 2024-04-26T03:39:47.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.487 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.966400+0000", 2024-04-26T03:39:47.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003386816 2024-04-26T03:39:47.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:47.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.966566+0000", 2024-04-26T03:39:47.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000166525 2024-04-26T03:39:47.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.489 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:47.489 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.968938+0000", 2024-04-26T03:39:47.489 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023716610000000002 2024-04-26T03:39:47.489 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.489 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.489 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.490 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.10s2 60/48 ECSubWrite(tid=272, reqid=client.4556.0:703, at_version=60'20, trim_to=0'0, roll_forward_to=60'18))", 2024-04-26T03:39:47.490 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:15.958431+0000", 2024-04-26T03:39:47.490 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.426790257, 2024-04-26T03:39:47.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016173328000000001, 2024-04-26T03:39:47.490 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:47.490 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:47.490 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:47.491 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:47.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.491 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:47.491 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.958431+0000", 2024-04-26T03:39:47.491 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.491 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:47.491 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.958431+0000", 2024-04-26T03:39:47.492 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.492 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:47.492 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.958432+0000", 2024-04-26T03:39:47.492 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.031e-06 2024-04-26T03:39:47.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.493 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:47.493 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.962281+0000", 2024-04-26T03:39:47.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0038488630000000001 2024-04-26T03:39:47.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.493 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.493 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:47.493 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.962301+0000", 2024-04-26T03:39:47.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9828999999999999e-05 2024-04-26T03:39:47.494 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.494 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.494 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:47.494 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.962308+0000", 2024-04-26T03:39:47.494 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2620000000000001e-06 2024-04-26T03:39:47.494 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.494 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.494 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.495 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.962375+0000", 2024-04-26T03:39:47.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6729000000000004e-05 2024-04-26T03:39:47.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.495 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:47.495 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.962384+0000", 2024-04-26T03:39:47.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6390000000000004e-06 2024-04-26T03:39:47.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:47.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.974567+0000", 2024-04-26T03:39:47.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012182767000000001 2024-04-26T03:39:47.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:47.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.974604+0000", 2024-04-26T03:39:47.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7208e-05 2024-04-26T03:39:47.496 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.497 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.497 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.497 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:774 3.ds0 3:b3f958c3:::smithi09235267-22:head [append 0~737280] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:47.497 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:20.780689+0000", 2024-04-26T03:39:47.497 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.604532278000001, 2024-04-26T03:39:47.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021054691, 2024-04-26T03:39:47.498 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:47.498 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:47.498 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:47.498 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:47.498 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:47.498 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:47.498 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 774 2024-04-26T03:39:47.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.499 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:47.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:47.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.780689+0000", 2024-04-26T03:39:47.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:47.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.780689+0000", 2024-04-26T03:39:47.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.500 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:47.500 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.780690+0000", 2024-04-26T03:39:47.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3400000000000001e-06 2024-04-26T03:39:47.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:47.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.780841+0000", 2024-04-26T03:39:47.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000150743 2024-04-26T03:39:47.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:47.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.780853+0000", 2024-04-26T03:39:47.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1681e-05 2024-04-26T03:39:47.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:47.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.780858+0000", 2024-04-26T03:39:47.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8390000000000003e-06 2024-04-26T03:39:47.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.780915+0000", 2024-04-26T03:39:47.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6338000000000002e-05 2024-04-26T03:39:47.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:47.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.781021+0000", 2024-04-26T03:39:47.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106498 2024-04-26T03:39:47.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:47.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.782158+0000", 2024-04-26T03:39:47.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001136977 2024-04-26T03:39:47.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:47.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.795362+0000", 2024-04-26T03:39:47.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013204052000000001 2024-04-26T03:39:47.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:47.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.801661+0000", 2024-04-26T03:39:47.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0062988540000000004 2024-04-26T03:39:47.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:47.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.801743+0000", 2024-04-26T03:39:47.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2368999999999997e-05 2024-04-26T03:39:47.506 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.506 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.506 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.506 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:775 3.ds0 3:b3f958c3:::smithi09235267-22:head [append 0~770048] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:47.507 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:20.785167+0000", 2024-04-26T03:39:47.507 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.600054241999999, 2024-04-26T03:39:47.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017555833, 2024-04-26T03:39:47.507 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:47.507 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:47.507 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:47.507 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:47.507 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:47.508 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:47.508 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 775 2024-04-26T03:39:47.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.508 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:47.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:47.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.785167+0000", 2024-04-26T03:39:47.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:47.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.785167+0000", 2024-04-26T03:39:47.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:47.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.785168+0000", 2024-04-26T03:39:47.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.339e-06 2024-04-26T03:39:47.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:47.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.785488+0000", 2024-04-26T03:39:47.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031973100000000002 2024-04-26T03:39:47.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:47.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.785499+0000", 2024-04-26T03:39:47.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1409999999999999e-05 2024-04-26T03:39:47.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:47.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.785506+0000", 2024-04-26T03:39:47.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.224e-06 2024-04-26T03:39:47.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.789235+0000", 2024-04-26T03:39:47.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0037290069999999999 2024-04-26T03:39:47.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:47.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.789350+0000", 2024-04-26T03:39:47.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115327 2024-04-26T03:39:47.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:47.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.790178+0000", 2024-04-26T03:39:47.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00082851799999999998 2024-04-26T03:39:47.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:47.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.801460+0000", 2024-04-26T03:39:47.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011282003000000001 2024-04-26T03:39:47.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:47.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.802652+0000", 2024-04-26T03:39:47.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011913889999999999 2024-04-26T03:39:47.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:47.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.802723+0000", 2024-04-26T03:39:47.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0884999999999996e-05 2024-04-26T03:39:47.515 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.515 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.515 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.515 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:776 3.ds0 3:b3f958c3:::smithi09235267-22:head [append 0~540672] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:47.515 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:20.788324+0000", 2024-04-26T03:39:47.516 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.596897303999999, 2024-04-26T03:39:47.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016617261000000001, 2024-04-26T03:39:47.516 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:47.516 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:47.516 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:47.516 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:47.516 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:47.516 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:47.516 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 776 2024-04-26T03:39:47.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.517 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:47.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:47.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.788324+0000", 2024-04-26T03:39:47.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:47.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.788324+0000", 2024-04-26T03:39:47.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:47.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.788325+0000", 2024-04-26T03:39:47.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7010000000000001e-06 2024-04-26T03:39:47.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:47.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.788621+0000", 2024-04-26T03:39:47.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029502800000000002 2024-04-26T03:39:47.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:47.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.788635+0000", 2024-04-26T03:39:47.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4741e-05 2024-04-26T03:39:47.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:47.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.788643+0000", 2024-04-26T03:39:47.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9610000000000005e-06 2024-04-26T03:39:47.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.795417+0000", 2024-04-26T03:39:47.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067738520000000003 2024-04-26T03:39:47.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:47.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.795523+0000", 2024-04-26T03:39:47.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010549599999999999 2024-04-26T03:39:47.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:47.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.796141+0000", 2024-04-26T03:39:47.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00061875999999999999 2024-04-26T03:39:47.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.522 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:47.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.801534+0000", 2024-04-26T03:39:47.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053927899999999997 2024-04-26T03:39:47.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.522 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:47.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.804742+0000", 2024-04-26T03:39:47.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003207522 2024-04-26T03:39:47.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:47.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.804941+0000", 2024-04-26T03:39:47.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019940999999999999 2024-04-26T03:39:47.523 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.523 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.523 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.523 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:778 3.ds0 3:b3f958c3:::smithi09235267-22:head [setxattr _header (57) in=7b] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:47.523 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:20.789998+0000", 2024-04-26T03:39:47.524 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.595223076, 2024-04-26T03:39:47.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015643300999999998, 2024-04-26T03:39:47.524 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:47.524 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:47.524 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:47.524 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:47.524 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:47.524 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:47.524 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 778 2024-04-26T03:39:47.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.524 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:47.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:47.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.789998+0000", 2024-04-26T03:39:47.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:47.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.789998+0000", 2024-04-26T03:39:47.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:47.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.789999+0000", 2024-04-26T03:39:47.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9800000000000003e-07 2024-04-26T03:39:47.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:47.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.790002+0000", 2024-04-26T03:39:47.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.597e-06 2024-04-26T03:39:47.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:47.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.790009+0000", 2024-04-26T03:39:47.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9700000000000002e-06 2024-04-26T03:39:47.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:47.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.790015+0000", 2024-04-26T03:39:47.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6350000000000003e-06 2024-04-26T03:39:47.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.801932+0000", 2024-04-26T03:39:47.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011917669000000001 2024-04-26T03:39:47.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:47.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.801990+0000", 2024-04-26T03:39:47.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7822000000000003e-05 2024-04-26T03:39:47.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:47.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.802279+0000", 2024-04-26T03:39:47.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028822199999999998 2024-04-26T03:39:47.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:47.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.803223+0000", 2024-04-26T03:39:47.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00094406100000000005 2024-04-26T03:39:47.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:47.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.805407+0000", 2024-04-26T03:39:47.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0021844360000000001 2024-04-26T03:39:47.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:47.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.805641+0000", 2024-04-26T03:39:47.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000234291 2024-04-26T03:39:47.530 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.530 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.530 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.531 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:779 3.ds0 3:b3f958c3:::smithi09235267-22:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:47.531 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:20.790023+0000", 2024-04-26T03:39:47.531 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.595198178, 2024-04-26T03:39:47.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017774486999999999, 2024-04-26T03:39:47.531 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:47.531 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:47.531 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:47.531 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:47.531 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:47.531 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:47.532 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 779 2024-04-26T03:39:47.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.532 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:47.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:47.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.790023+0000", 2024-04-26T03:39:47.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:47.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.790023+0000", 2024-04-26T03:39:47.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:47.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.790023+0000", 2024-04-26T03:39:47.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4200000000000001e-07 2024-04-26T03:39:47.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:47.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.790026+0000", 2024-04-26T03:39:47.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1159999999999999e-06 2024-04-26T03:39:47.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:47.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.790033+0000", 2024-04-26T03:39:47.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6100000000000002e-06 2024-04-26T03:39:47.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:47.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.790038+0000", 2024-04-26T03:39:47.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9439999999999996e-06 2024-04-26T03:39:47.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.802971+0000", 2024-04-26T03:39:47.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012933445 2024-04-26T03:39:47.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:47.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.803177+0000", 2024-04-26T03:39:47.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020553499999999999 2024-04-26T03:39:47.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.806365+0000", 2024-04-26T03:39:47.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003187615 2024-04-26T03:39:47.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:47.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.806453+0000", 2024-04-26T03:39:47.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8240999999999998e-05 2024-04-26T03:39:47.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:47.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.807797+0000", 2024-04-26T03:39:47.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013445390000000001 2024-04-26T03:39:47.538 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.538 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.538 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.538 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.10s2 80/48 ECSubWrite(tid=322, reqid=client.4556.0:895, at_version=80'35, trim_to=0'0, roll_forward_to=75'33))", 2024-04-26T03:39:47.539 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:35.740651+0000", 2024-04-26T03:39:47.539 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 11.644570562, 2024-04-26T03:39:47.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.028220485999999999, 2024-04-26T03:39:47.539 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:47.539 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:47.539 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:47.539 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:47.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:47.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.740651+0000", 2024-04-26T03:39:47.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:47.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.740651+0000", 2024-04-26T03:39:47.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:47.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.740651+0000", 2024-04-26T03:39:47.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4699999999999998e-07 2024-04-26T03:39:47.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:47.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.745849+0000", 2024-04-26T03:39:47.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051984409999999998 2024-04-26T03:39:47.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:47.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.745862+0000", 2024-04-26T03:39:47.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2369e-05 2024-04-26T03:39:47.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:47.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.745868+0000", 2024-04-26T03:39:47.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6930000000000001e-06 2024-04-26T03:39:47.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.745920+0000", 2024-04-26T03:39:47.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1579000000000002e-05 2024-04-26T03:39:47.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:47.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.745928+0000", 2024-04-26T03:39:47.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0369999999999995e-06 2024-04-26T03:39:47.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:47.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.768824+0000", 2024-04-26T03:39:47.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022896162000000001 2024-04-26T03:39:47.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:47.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.768871+0000", 2024-04-26T03:39:47.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6958e-05 2024-04-26T03:39:47.543 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.543 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.543 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.543 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.544 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.544 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:47.545 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-04-26T03:39:47.648 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:47.648 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-26T03:39:47.648 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-26T03:39:47.648 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:47.657 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-04-26T03:39:47.758 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:47.759 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-26T03:39:47.759 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-26T03:39:47.759 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-26T03:39:47.759 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:47.767 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-04-26T03:39:47.832 INFO:teuthology.orchestra.run.smithi028.stderr:noscrub is set 2024-04-26T03:39:47.872 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:47.872 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-26T03:39:47.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-26T03:39:47.872 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-26T03:39:47.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.873 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:12 3.cs0 3:3946a5f1:::smithi09235267-4:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:47.873 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.481722+0000", 2024-04-26T03:39:47.873 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 56.388333865, 2024-04-26T03:39:47.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039281273999999998, 2024-04-26T03:39:47.873 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:47.873 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:47.873 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:47.873 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:47.873 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:47.873 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:47.873 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 12 2024-04-26T03:39:47.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.874 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:47.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:47.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.481722+0000", 2024-04-26T03:39:47.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:47.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.481722+0000", 2024-04-26T03:39:47.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:47.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.481723+0000", 2024-04-26T03:39:47.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.75e-07 2024-04-26T03:39:47.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:47.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.481725+0000", 2024-04-26T03:39:47.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6979999999999998e-06 2024-04-26T03:39:47.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:47.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.481733+0000", 2024-04-26T03:39:47.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2899999999999997e-06 2024-04-26T03:39:47.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:47.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.481743+0000", 2024-04-26T03:39:47.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0845999999999999e-05 2024-04-26T03:39:47.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.484203+0000", 2024-04-26T03:39:47.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024598889999999998 2024-04-26T03:39:47.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:47.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.484353+0000", 2024-04-26T03:39:47.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015005199999999999 2024-04-26T03:39:47.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.497880+0000", 2024-04-26T03:39:47.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013526589 2024-04-26T03:39:47.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:47.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.497957+0000", 2024-04-26T03:39:47.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7126000000000007e-05 2024-04-26T03:39:47.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:47.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.521003+0000", 2024-04-26T03:39:47.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.023046309000000001 2024-04-26T03:39:47.878 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.878 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.878 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.878 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:31 3.cs0 3:3edefb40:::smithi09235267-11:head [append 0~180224,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:47.878 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.494104+0000", 2024-04-26T03:39:47.878 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 56.375951561000001, 2024-04-26T03:39:47.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025578053999999999, 2024-04-26T03:39:47.878 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:47.878 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:47.878 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:47.879 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:47.879 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:47.879 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:47.879 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 31 2024-04-26T03:39:47.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.879 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:47.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:47.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.494104+0000", 2024-04-26T03:39:47.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:47.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.494104+0000", 2024-04-26T03:39:47.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:47.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.494106+0000", 2024-04-26T03:39:47.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2100000000000001e-06 2024-04-26T03:39:47.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:47.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.494218+0000", 2024-04-26T03:39:47.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000112398 2024-04-26T03:39:47.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:47.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.494230+0000", 2024-04-26T03:39:47.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1688999999999999e-05 2024-04-26T03:39:47.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:47.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.494237+0000", 2024-04-26T03:39:47.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2049999999999996e-06 2024-04-26T03:39:47.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.494745+0000", 2024-04-26T03:39:47.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00050836900000000005 2024-04-26T03:39:47.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:47.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.494857+0000", 2024-04-26T03:39:47.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000112045 2024-04-26T03:39:47.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:47.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495764+0000", 2024-04-26T03:39:47.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00090692400000000003 2024-04-26T03:39:47.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:47.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519293+0000", 2024-04-26T03:39:47.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.023528830000000001 2024-04-26T03:39:47.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:47.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519597+0000", 2024-04-26T03:39:47.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030426100000000001 2024-04-26T03:39:47.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:47.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519682+0000", 2024-04-26T03:39:47.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5123000000000005e-05 2024-04-26T03:39:47.883 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.883 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.884 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.884 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:32 3.cs0 3:3edefb40:::smithi09235267-11:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:47.884 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.495390+0000", 2024-04-26T03:39:47.884 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 56.374665645, 2024-04-26T03:39:47.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027633438, 2024-04-26T03:39:47.884 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:47.884 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:47.884 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:47.884 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:47.884 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:47.884 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:47.884 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 32 2024-04-26T03:39:47.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.885 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:47.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:47.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495390+0000", 2024-04-26T03:39:47.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:47.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495390+0000", 2024-04-26T03:39:47.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:47.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495391+0000", 2024-04-26T03:39:47.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8999999999999995e-07 2024-04-26T03:39:47.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:47.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495396+0000", 2024-04-26T03:39:47.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4360000000000001e-06 2024-04-26T03:39:47.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:47.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495410+0000", 2024-04-26T03:39:47.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3482000000000001e-05 2024-04-26T03:39:47.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:47.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495419+0000", 2024-04-26T03:39:47.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9600000000000006e-06 2024-04-26T03:39:47.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518418+0000", 2024-04-26T03:39:47.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022999116999999999 2024-04-26T03:39:47.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:47.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518517+0000", 2024-04-26T03:39:47.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8542999999999995e-05 2024-04-26T03:39:47.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:47.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519020+0000", 2024-04-26T03:39:47.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00050364099999999998 2024-04-26T03:39:47.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:47.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519837+0000", 2024-04-26T03:39:47.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00081666699999999998 2024-04-26T03:39:47.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:47.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.522936+0000", 2024-04-26T03:39:47.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030991170000000002 2024-04-26T03:39:47.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:47.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.523024+0000", 2024-04-26T03:39:47.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7684999999999998e-05 2024-04-26T03:39:47.890 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.891 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.891 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.891 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:33 3.cs0 3:3edefb40:::smithi09235267-11:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:47.891 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.495433+0000", 2024-04-26T03:39:47.891 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 56.374622723000002, 2024-04-26T03:39:47.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032718572000000001, 2024-04-26T03:39:47.891 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:47.891 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:47.891 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:47.891 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:47.891 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:47.891 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:47.892 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 33 2024-04-26T03:39:47.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.892 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:47.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:47.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495433+0000", 2024-04-26T03:39:47.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:47.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495433+0000", 2024-04-26T03:39:47.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:47.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495434+0000", 2024-04-26T03:39:47.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2300000000000001e-07 2024-04-26T03:39:47.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:47.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495437+0000", 2024-04-26T03:39:47.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2399999999999999e-06 2024-04-26T03:39:47.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:47.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495444+0000", 2024-04-26T03:39:47.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3869999999999997e-06 2024-04-26T03:39:47.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:47.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.495450+0000", 2024-04-26T03:39:47.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.412e-06 2024-04-26T03:39:47.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.520715+0000", 2024-04-26T03:39:47.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.02526488 2024-04-26T03:39:47.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:47.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.520783+0000", 2024-04-26T03:39:47.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8553000000000006e-05 2024-04-26T03:39:47.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.523315+0000", 2024-04-26T03:39:47.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025314299999999999 2024-04-26T03:39:47.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:47.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.523384+0000", 2024-04-26T03:39:47.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9503000000000002e-05 2024-04-26T03:39:47.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:47.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.528152+0000", 2024-04-26T03:39:47.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047676439999999997 2024-04-26T03:39:47.896 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.896 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.896 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.896 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.cs0 35/32 ECSubReadReply(tid=5, attrs_read=0))", 2024-04-26T03:39:47.896 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.498822+0000", 2024-04-26T03:39:47.896 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 56.371234078999997, 2024-04-26T03:39:47.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021016639, 2024-04-26T03:39:47.896 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:47.896 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:47.896 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:47.896 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:47.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:47.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498822+0000", 2024-04-26T03:39:47.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:47.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498822+0000", 2024-04-26T03:39:47.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:47.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498822+0000", 2024-04-26T03:39:47.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3099999999999997e-07 2024-04-26T03:39:47.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:47.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498834+0000", 2024-04-26T03:39:47.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1331e-05 2024-04-26T03:39:47.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:47.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498878+0000", 2024-04-26T03:39:47.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4092999999999999e-05 2024-04-26T03:39:47.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:47.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.498888+0000", 2024-04-26T03:39:47.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0049e-05 2024-04-26T03:39:47.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519785+0000", 2024-04-26T03:39:47.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020896656 2024-04-26T03:39:47.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:47.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519838+0000", 2024-04-26T03:39:47.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3878999999999997e-05 2024-04-26T03:39:47.899 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.899 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.899 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.900 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 35/32 ECSubWriteReply(tid=4, last_complete=35'3, committed=1, applied=1))", 2024-04-26T03:39:47.900 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.499205+0000", 2024-04-26T03:39:47.900 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 56.370851072000001, 2024-04-26T03:39:47.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020089540999999999, 2024-04-26T03:39:47.900 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:47.900 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:47.900 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:47.900 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:47.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:47.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499205+0000", 2024-04-26T03:39:47.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:47.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499205+0000", 2024-04-26T03:39:47.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:47.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499205+0000", 2024-04-26T03:39:47.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5199999999999996e-07 2024-04-26T03:39:47.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:47.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499210+0000", 2024-04-26T03:39:47.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0690000000000003e-06 2024-04-26T03:39:47.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:47.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499223+0000", 2024-04-26T03:39:47.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3783e-05 2024-04-26T03:39:47.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:47.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499230+0000", 2024-04-26T03:39:47.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7970000000000003e-06 2024-04-26T03:39:47.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519273+0000", 2024-04-26T03:39:47.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020043068000000001 2024-04-26T03:39:47.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:47.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519294+0000", 2024-04-26T03:39:47.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1172e-05 2024-04-26T03:39:47.903 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.903 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.903 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.904 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.cs0 35/32 ECSubReadReply(tid=5, attrs_read=0))", 2024-04-26T03:39:47.904 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.500849+0000", 2024-04-26T03:39:47.904 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 56.369207222999997, 2024-04-26T03:39:47.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019810352, 2024-04-26T03:39:47.904 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:47.904 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:47.904 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:47.904 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:47.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:47.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500849+0000", 2024-04-26T03:39:47.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:47.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500849+0000", 2024-04-26T03:39:47.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:47.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500849+0000", 2024-04-26T03:39:47.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9999999999999999e-07 2024-04-26T03:39:47.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:47.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500860+0000", 2024-04-26T03:39:47.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0811e-05 2024-04-26T03:39:47.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:47.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500867+0000", 2024-04-26T03:39:47.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9670000000000004e-06 2024-04-26T03:39:47.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:47.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500871+0000", 2024-04-26T03:39:47.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3109999999999999e-06 2024-04-26T03:39:47.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.520619+0000", 2024-04-26T03:39:47.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019747602 2024-04-26T03:39:47.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:47.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.520659+0000", 2024-04-26T03:39:47.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0361000000000002e-05 2024-04-26T03:39:47.907 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.907 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.907 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.907 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:604 3.13s0 3:cad73b03:::smithi09235267-37:head [copy-from ver 50] snapc d=[d,c,b,a] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-26T03:39:47.907 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:11.710211+0000", 2024-04-26T03:39:47.907 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.159845167999997, 2024-04-26T03:39:47.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.082214682999999997, 2024-04-26T03:39:47.907 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:47.908 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:47.908 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:47.908 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:47.908 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:47.908 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:47.908 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 604 2024-04-26T03:39:47.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.908 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:47.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:47.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.710211+0000", 2024-04-26T03:39:47.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:47.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.710211+0000", 2024-04-26T03:39:47.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:47.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.710211+0000", 2024-04-26T03:39:47.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9100000000000004e-07 2024-04-26T03:39:47.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:47.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.710218+0000", 2024-04-26T03:39:47.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6880000000000002e-06 2024-04-26T03:39:47.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:47.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.710225+0000", 2024-04-26T03:39:47.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2019999999999998e-06 2024-04-26T03:39:47.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:47.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.710232+0000", 2024-04-26T03:39:47.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8059999999999997e-06 2024-04-26T03:39:47.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.712246+0000", 2024-04-26T03:39:47.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020136189999999999 2024-04-26T03:39:47.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-26T03:39:47.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.712248+0000", 2024-04-26T03:39:47.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1179999999999999e-06 2024-04-26T03:39:47.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.743111+0000", 2024-04-26T03:39:47.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.030862865999999999 2024-04-26T03:39:47.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for missing object", 2024-04-26T03:39:47.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.743951+0000", 2024-04-26T03:39:47.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00084010900000000002 2024-04-26T03:39:47.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.759857+0000", 2024-04-26T03:39:47.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015906699 2024-04-26T03:39:47.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:47.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.760063+0000", 2024-04-26T03:39:47.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020573299999999999 2024-04-26T03:39:47.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:47.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.774271+0000", 2024-04-26T03:39:47.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014208346 2024-04-26T03:39:47.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:47.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.790920+0000", 2024-04-26T03:39:47.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016649048 2024-04-26T03:39:47.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:47.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.791986+0000", 2024-04-26T03:39:47.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010652299999999999 2024-04-26T03:39:47.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:47.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.792425+0000", 2024-04-26T03:39:47.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00043972800000000002 2024-04-26T03:39:47.914 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.914 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.914 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.914 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:605 3.13s0 3:cad73b03:::smithi09235267-37:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-04-26T03:39:47.914 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:11.710239+0000", 2024-04-26T03:39:47.914 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.159816513000003, 2024-04-26T03:39:47.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080074244000000003, 2024-04-26T03:39:47.914 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:47.914 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:47.914 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:47.914 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:47.915 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:47.915 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:47.915 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 605 2024-04-26T03:39:47.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.915 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:47.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:47.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.710239+0000", 2024-04-26T03:39:47.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:47.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.710239+0000", 2024-04-26T03:39:47.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:47.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.710240+0000", 2024-04-26T03:39:47.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.32e-07 2024-04-26T03:39:47.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:47.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.710243+0000", 2024-04-26T03:39:47.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5319999999999998e-06 2024-04-26T03:39:47.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:47.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.710250+0000", 2024-04-26T03:39:47.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5889999999999999e-06 2024-04-26T03:39:47.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:47.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.710256+0000", 2024-04-26T03:39:47.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5670000000000002e-06 2024-04-26T03:39:47.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.712312+0000", 2024-04-26T03:39:47.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020565190000000001 2024-04-26T03:39:47.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-26T03:39:47.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.712314+0000", 2024-04-26T03:39:47.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1880000000000001e-06 2024-04-26T03:39:47.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.746793+0000", 2024-04-26T03:39:47.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034479224000000003 2024-04-26T03:39:47.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for missing object", 2024-04-26T03:39:47.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.746907+0000", 2024-04-26T03:39:47.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113739 2024-04-26T03:39:47.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.919 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.919 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.762064+0000", 2024-04-26T03:39:47.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015157096 2024-04-26T03:39:47.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.919 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for blocked object", 2024-04-26T03:39:47.919 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.762162+0000", 2024-04-26T03:39:47.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8133000000000004e-05 2024-04-26T03:39:47.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.919 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.919 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.789483+0000", 2024-04-26T03:39:47.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027321055 2024-04-26T03:39:47.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:47.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.789695+0000", 2024-04-26T03:39:47.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021135500000000001 2024-04-26T03:39:47.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:47.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.790314+0000", 2024-04-26T03:39:47.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00061881499999999999 2024-04-26T03:39:47.920 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.920 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.920 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.921 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:619 3.13s0 3:c944ba3d:::smithi09235267-6:a [read 0~4669440,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-04-26T03:39:47.921 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:11.732770+0000", 2024-04-26T03:39:47.921 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.137286162999999, 2024-04-26T03:39:47.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.12874891899999999, 2024-04-26T03:39:47.921 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:47.921 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:47.921 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:47.921 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:47.921 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:47.921 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:47.921 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 619 2024-04-26T03:39:47.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.921 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:47.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:47.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732770+0000", 2024-04-26T03:39:47.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:47.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732770+0000", 2024-04-26T03:39:47.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:47.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732772+0000", 2024-04-26T03:39:47.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.266e-06 2024-04-26T03:39:47.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:47.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732781+0000", 2024-04-26T03:39:47.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0350000000000007e-06 2024-04-26T03:39:47.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:47.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732803+0000", 2024-04-26T03:39:47.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2140000000000001e-05 2024-04-26T03:39:47.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:47.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732815+0000", 2024-04-26T03:39:47.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1637000000000001e-05 2024-04-26T03:39:47.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732928+0000", 2024-04-26T03:39:47.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000112717 2024-04-26T03:39:47.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-26T03:39:47.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732936+0000", 2024-04-26T03:39:47.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5439999999999999e-06 2024-04-26T03:39:47.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.746980+0000", 2024-04-26T03:39:47.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014043959999999999 2024-04-26T03:39:47.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for missing object", 2024-04-26T03:39:47.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.748272+0000", 2024-04-26T03:39:47.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0012919139999999999 2024-04-26T03:39:47.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.852799+0000", 2024-04-26T03:39:47.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.104527289 2024-04-26T03:39:47.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:47.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.852915+0000", 2024-04-26T03:39:47.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115395 2024-04-26T03:39:47.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:47.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.861519+0000", 2024-04-26T03:39:47.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0086040219999999994 2024-04-26T03:39:47.926 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.926 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.926 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.926 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:620 3.13s0 3:c944ba3d:::smithi09235267-6:a [read 0~4669440,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-04-26T03:39:47.927 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:11.732848+0000", 2024-04-26T03:39:47.927 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.137207752999998, 2024-04-26T03:39:47.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.13189979199999999, 2024-04-26T03:39:47.927 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:47.927 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:47.927 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:47.927 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:47.927 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:47.927 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:47.927 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 620 2024-04-26T03:39:47.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.927 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:47.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:47.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732848+0000", 2024-04-26T03:39:47.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:47.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732848+0000", 2024-04-26T03:39:47.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:47.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732849+0000", 2024-04-26T03:39:47.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9800000000000003e-07 2024-04-26T03:39:47.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:47.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732860+0000", 2024-04-26T03:39:47.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0942e-05 2024-04-26T03:39:47.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:47.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732869+0000", 2024-04-26T03:39:47.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1800000000000002e-06 2024-04-26T03:39:47.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:47.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732880+0000", 2024-04-26T03:39:47.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1021e-05 2024-04-26T03:39:47.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732987+0000", 2024-04-26T03:39:47.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107095 2024-04-26T03:39:47.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-26T03:39:47.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732989+0000", 2024-04-26T03:39:47.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1919999999999999e-06 2024-04-26T03:39:47.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.748328+0000", 2024-04-26T03:39:47.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015338312 2024-04-26T03:39:47.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for missing object", 2024-04-26T03:39:47.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.748430+0000", 2024-04-26T03:39:47.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102669 2024-04-26T03:39:47.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.853182+0000", 2024-04-26T03:39:47.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10475184799999999 2024-04-26T03:39:47.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:47.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.853309+0000", 2024-04-26T03:39:47.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012699099999999999 2024-04-26T03:39:47.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:47.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.864748+0000", 2024-04-26T03:39:47.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011438844 2024-04-26T03:39:47.932 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.932 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.932 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.932 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:621 3.13s0 3:c944ba3d:::smithi09235267-6:a [read 0~4669440,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-04-26T03:39:47.932 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:11.732920+0000", 2024-04-26T03:39:47.933 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.137135717, 2024-04-26T03:39:47.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.134463005, 2024-04-26T03:39:47.933 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:47.933 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:47.933 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:47.933 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:47.933 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:47.933 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:47.933 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 621 2024-04-26T03:39:47.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.933 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:47.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:47.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732920+0000", 2024-04-26T03:39:47.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:47.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732920+0000", 2024-04-26T03:39:47.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:47.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732921+0000", 2024-04-26T03:39:47.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5400000000000001e-07 2024-04-26T03:39:47.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:47.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732925+0000", 2024-04-26T03:39:47.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0180000000000003e-06 2024-04-26T03:39:47.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:47.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732938+0000", 2024-04-26T03:39:47.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2928999999999999e-05 2024-04-26T03:39:47.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:47.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.733001+0000", 2024-04-26T03:39:47.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3621999999999995e-05 2024-04-26T03:39:47.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.733061+0000", 2024-04-26T03:39:47.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9905000000000003e-05 2024-04-26T03:39:47.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-26T03:39:47.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.733063+0000", 2024-04-26T03:39:47.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3990000000000001e-06 2024-04-26T03:39:47.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.749893+0000", 2024-04-26T03:39:47.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016830254999999999 2024-04-26T03:39:47.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for missing object", 2024-04-26T03:39:47.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.750023+0000", 2024-04-26T03:39:47.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000130264 2024-04-26T03:39:47.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.853768+0000", 2024-04-26T03:39:47.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.103745171 2024-04-26T03:39:47.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:47.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.853937+0000", 2024-04-26T03:39:47.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016834 2024-04-26T03:39:47.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:47.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.867383+0000", 2024-04-26T03:39:47.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013446448 2024-04-26T03:39:47.938 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.938 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.938 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.938 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDPGPushReply(3.13s0 56/55 [PushReplyOp(3:c944ba3d:::smithi09235267-6:head)])", 2024-04-26T03:39:47.938 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:11.768451+0000", 2024-04-26T03:39:47.938 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.101605376000002, 2024-04-26T03:39:47.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.023073083000000001, 2024-04-26T03:39:47.938 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:47.939 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:47.939 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:47.939 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:47.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:47.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.768451+0000", 2024-04-26T03:39:47.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:47.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.768451+0000", 2024-04-26T03:39:47.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:47.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.768451+0000", 2024-04-26T03:39:47.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7400000000000002e-07 2024-04-26T03:39:47.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:47.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.768460+0000", 2024-04-26T03:39:47.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0450000000000006e-06 2024-04-26T03:39:47.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:47.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.768471+0000", 2024-04-26T03:39:47.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0776e-05 2024-04-26T03:39:47.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:47.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.768482+0000", 2024-04-26T03:39:47.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1131999999999999e-05 2024-04-26T03:39:47.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.791177+0000", 2024-04-26T03:39:47.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022694269 2024-04-26T03:39:47.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:47.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.791524+0000", 2024-04-26T03:39:47.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00034708699999999999 2024-04-26T03:39:47.941 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.941 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.942 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.942 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDPGPushReply(3.13s0 56/55 [PushReplyOp(3:c944ba3d:::smithi09235267-6:head)])", 2024-04-26T03:39:47.942 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:11.768915+0000", 2024-04-26T03:39:47.942 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.101141280999997, 2024-04-26T03:39:47.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022008744, 2024-04-26T03:39:47.942 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:47.942 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:47.942 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:47.942 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:47.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:47.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.768915+0000", 2024-04-26T03:39:47.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:47.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.768915+0000", 2024-04-26T03:39:47.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:47.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.768915+0000", 2024-04-26T03:39:47.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0600000000000001e-07 2024-04-26T03:39:47.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:47.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.768920+0000", 2024-04-26T03:39:47.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9740000000000001e-06 2024-04-26T03:39:47.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:47.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.768928+0000", 2024-04-26T03:39:47.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2609999999999993e-06 2024-04-26T03:39:47.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:47.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.768941+0000", 2024-04-26T03:39:47.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2480000000000001e-05 2024-04-26T03:39:47.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.790794+0000", 2024-04-26T03:39:47.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021853289000000001 2024-04-26T03:39:47.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:47.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.790923+0000", 2024-04-26T03:39:47.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012933399999999999 2024-04-26T03:39:47.945 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.945 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.945 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.945 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDPGPushReply(3.13s0 56/55 [PushReplyOp(3:c944ba3d:::smithi09235267-6:head)])", 2024-04-26T03:39:47.946 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:11.769116+0000", 2024-04-26T03:39:47.946 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.100940303999998, 2024-04-26T03:39:47.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021803037000000001, 2024-04-26T03:39:47.946 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:47.946 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:47.946 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:47.946 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:47.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:47.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769116+0000", 2024-04-26T03:39:47.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:47.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769116+0000", 2024-04-26T03:39:47.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:47.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769116+0000", 2024-04-26T03:39:47.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9400000000000005e-07 2024-04-26T03:39:47.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:47.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769120+0000", 2024-04-26T03:39:47.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0740000000000003e-06 2024-04-26T03:39:47.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:47.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769136+0000", 2024-04-26T03:39:47.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5532e-05 2024-04-26T03:39:47.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:47.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769144+0000", 2024-04-26T03:39:47.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7549999999999994e-06 2024-04-26T03:39:47.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.790869+0000", 2024-04-26T03:39:47.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021725661 2024-04-26T03:39:47.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:47.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.790919+0000", 2024-04-26T03:39:47.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9320999999999998e-05 2024-04-26T03:39:47.949 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.949 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.949 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.949 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDPGPushReply(3.13s0 56/55 [PushReplyOp(3:c944ba3d:::smithi09235267-6:head)])", 2024-04-26T03:39:47.949 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:11.769260+0000", 2024-04-26T03:39:47.949 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.100795969000004, 2024-04-26T03:39:47.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021509337999999999, 2024-04-26T03:39:47.950 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:47.950 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:47.950 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:47.950 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:47.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:47.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769260+0000", 2024-04-26T03:39:47.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:47.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769260+0000", 2024-04-26T03:39:47.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:47.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769260+0000", 2024-04-26T03:39:47.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9200000000000002e-07 2024-04-26T03:39:47.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:47.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769266+0000", 2024-04-26T03:39:47.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2024-04-26T03:39:47.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:47.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769274+0000", 2024-04-26T03:39:47.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3289999999999996e-06 2024-04-26T03:39:47.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:47.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769279+0000", 2024-04-26T03:39:47.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3600000000000004e-06 2024-04-26T03:39:47.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.790713+0000", 2024-04-26T03:39:47.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021433612000000001 2024-04-26T03:39:47.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:47.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.790769+0000", 2024-04-26T03:39:47.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6536999999999998e-05 2024-04-26T03:39:47.953 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.953 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.953 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.953 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDPGPushReply(3.13s0 56/55 [PushReplyOp(3:c944ba3d:::smithi09235267-6:head)])", 2024-04-26T03:39:47.953 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:11.769267+0000", 2024-04-26T03:39:47.953 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.100789161999998, 2024-04-26T03:39:47.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021747525, 2024-04-26T03:39:47.953 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:47.953 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:47.954 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:47.954 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:47.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:47.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769267+0000", 2024-04-26T03:39:47.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:47.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769267+0000", 2024-04-26T03:39:47.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:47.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769268+0000", 2024-04-26T03:39:47.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9400000000000004e-07 2024-04-26T03:39:47.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:47.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769271+0000", 2024-04-26T03:39:47.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5090000000000002e-06 2024-04-26T03:39:47.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:47.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769285+0000", 2024-04-26T03:39:47.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3566e-05 2024-04-26T03:39:47.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:47.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.769290+0000", 2024-04-26T03:39:47.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8420000000000001e-06 2024-04-26T03:39:47.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.790968+0000", 2024-04-26T03:39:47.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021677923000000002 2024-04-26T03:39:47.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:47.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.791014+0000", 2024-04-26T03:39:47.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5890999999999998e-05 2024-04-26T03:39:47.957 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.957 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.957 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.957 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:710 3.13s0 3:c8283ddb:::smithi09235267-32:head [append 0~573440] snapc f=[f,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-04-26T03:39:47.957 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:15.970859+0000", 2024-04-26T03:39:47.957 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.899197381, 2024-04-26T03:39:47.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019548756, 2024-04-26T03:39:47.957 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:47.957 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:47.957 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:47.957 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:47.958 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:47.958 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:47.958 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 710 2024-04-26T03:39:47.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.958 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:47.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:47.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.970859+0000", 2024-04-26T03:39:47.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:47.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.970859+0000", 2024-04-26T03:39:47.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:47.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.970860+0000", 2024-04-26T03:39:47.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9649999999999998e-06 2024-04-26T03:39:47.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:47.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.971054+0000", 2024-04-26T03:39:47.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000193413 2024-04-26T03:39:47.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:47.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.971071+0000", 2024-04-26T03:39:47.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.694e-05 2024-04-26T03:39:47.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:47.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.971080+0000", 2024-04-26T03:39:47.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.414e-06 2024-04-26T03:39:47.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.971155+0000", 2024-04-26T03:39:47.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4482999999999998e-05 2024-04-26T03:39:47.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:47.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.971280+0000", 2024-04-26T03:39:47.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012498899999999999 2024-04-26T03:39:47.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:47.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.971925+0000", 2024-04-26T03:39:47.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000645003 2024-04-26T03:39:47.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:47.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.982227+0000", 2024-04-26T03:39:47.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01030181 2024-04-26T03:39:47.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:47.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.990344+0000", 2024-04-26T03:39:47.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0081176319999999996 2024-04-26T03:39:47.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:47.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:15.990407+0000", 2024-04-26T03:39:47.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3107000000000004e-05 2024-04-26T03:39:47.962 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.962 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.962 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.963 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:782 3.fs0 3:f81f96ba:::smithi09235267-46:head [append 0~540672] snapc 12=[12,10,e,d,c] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-04-26T03:39:47.963 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:20.821620+0000", 2024-04-26T03:39:47.963 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.048435484999999, 2024-04-26T03:39:47.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020779720000000002, 2024-04-26T03:39:47.963 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:47.963 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:47.963 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:47.963 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:47.963 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:47.963 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:47.963 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 782 2024-04-26T03:39:47.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.963 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:47.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:47.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.821620+0000", 2024-04-26T03:39:47.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:47.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.821620+0000", 2024-04-26T03:39:47.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:47.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.821621+0000", 2024-04-26T03:39:47.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6200000000000006e-07 2024-04-26T03:39:47.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:47.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.821861+0000", 2024-04-26T03:39:47.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000240063 2024-04-26T03:39:47.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:47.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.821873+0000", 2024-04-26T03:39:47.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1525999999999999e-05 2024-04-26T03:39:47.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:47.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.821882+0000", 2024-04-26T03:39:47.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9290000000000004e-06 2024-04-26T03:39:47.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.828072+0000", 2024-04-26T03:39:47.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061898659999999996 2024-04-26T03:39:47.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:47.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.828171+0000", 2024-04-26T03:39:47.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9437000000000004e-05 2024-04-26T03:39:47.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:47.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.828799+0000", 2024-04-26T03:39:47.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00062785900000000003 2024-04-26T03:39:47.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:47.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.841267+0000", 2024-04-26T03:39:47.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012467914 2024-04-26T03:39:47.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:47.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.842350+0000", 2024-04-26T03:39:47.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010830099999999999 2024-04-26T03:39:47.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:47.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:20.842400+0000", 2024-04-26T03:39:47.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0154000000000001e-05 2024-04-26T03:39:47.968 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.968 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.968 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.968 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.10s6 80/48 ECSubWrite(tid=322, reqid=client.4556.0:895, at_version=80'35, trim_to=0'0, roll_forward_to=75'33))", 2024-04-26T03:39:47.968 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:35.741160+0000", 2024-04-26T03:39:47.968 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.128895495, 2024-04-26T03:39:47.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029831673999999999, 2024-04-26T03:39:47.968 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:47.968 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:47.968 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:47.968 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:47.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:47.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.741160+0000", 2024-04-26T03:39:47.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:47.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.741160+0000", 2024-04-26T03:39:47.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:47.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:47.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.741161+0000", 2024-04-26T03:39:47.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4999999999999998e-07 2024-04-26T03:39:47.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:47.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.745641+0000", 2024-04-26T03:39:47.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00447978 2024-04-26T03:39:47.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:47.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.745652+0000", 2024-04-26T03:39:47.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1596999999999999e-05 2024-04-26T03:39:47.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:47.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.745675+0000", 2024-04-26T03:39:47.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2960999999999999e-05 2024-04-26T03:39:47.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:47.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.745736+0000", 2024-04-26T03:39:47.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1106999999999996e-05 2024-04-26T03:39:47.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:47.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.745747+0000", 2024-04-26T03:39:47.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0190000000000001e-05 2024-04-26T03:39:47.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:47.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.770911+0000", 2024-04-26T03:39:47.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025164907 2024-04-26T03:39:47.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:47.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:47.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:47.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.770992+0000", 2024-04-26T03:39:47.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0681999999999998e-05 2024-04-26T03:39:47.972 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.972 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.972 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.972 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:47.972 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:47.972 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:47.974 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-04-26T03:39:48.077 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:48.077 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-26T03:39:48.077 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-04-26T03:39:48.077 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:48.085 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-04-26T03:39:48.189 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:48.189 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-04-26T03:39:48.189 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-04-26T03:39:48.189 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-04-26T03:39:48.189 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:48.198 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-04-26T03:39:48.275 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T03:39:48.302 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-04-26T03:39:48.303 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-04-26T03:39:48.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-04-26T03:39:48.303 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-04-26T03:39:48.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.303 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:30 3.5s0 3:a64abe59:::smithi09235267-10:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-04-26T03:39:48.303 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.491045+0000", 2024-04-26T03:39:48.303 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 56.809082359999998, 2024-04-26T03:39:48.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.024868352, 2024-04-26T03:39:48.303 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:48.303 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:48.303 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:48.303 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:48.304 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:48.304 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:48.304 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 30 2024-04-26T03:39:48.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.304 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:48.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:48.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.491045+0000", 2024-04-26T03:39:48.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:48.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:48.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.491045+0000", 2024-04-26T03:39:48.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:48.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:48.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.491045+0000", 2024-04-26T03:39:48.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3799999999999997e-07 2024-04-26T03:39:48.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:48.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.491049+0000", 2024-04-26T03:39:48.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3359999999999999e-06 2024-04-26T03:39:48.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:48.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.491056+0000", 2024-04-26T03:39:48.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7549999999999994e-06 2024-04-26T03:39:48.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:48.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.491062+0000", 2024-04-26T03:39:48.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3730000000000001e-06 2024-04-26T03:39:48.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:48.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.493431+0000", 2024-04-26T03:39:48.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023688490000000001 2024-04-26T03:39:48.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:48.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.493515+0000", 2024-04-26T03:39:48.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4342e-05 2024-04-26T03:39:48.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:48.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499490+0000", 2024-04-26T03:39:48.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059748809999999996 2024-04-26T03:39:48.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:48.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.499562+0000", 2024-04-26T03:39:48.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1942999999999999e-05 2024-04-26T03:39:48.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:48.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515913+0000", 2024-04-26T03:39:48.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016351335000000002 2024-04-26T03:39:48.308 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:48.308 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:48.308 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:48.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.308 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.5s0 35/32 ECSubReadReply(tid=3, attrs_read=0))", 2024-04-26T03:39:48.308 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.500548+0000", 2024-04-26T03:39:48.308 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 56.799579152, 2024-04-26T03:39:48.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014243212999999999, 2024-04-26T03:39:48.308 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:48.308 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:48.308 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:48.309 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:48.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:48.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500548+0000", 2024-04-26T03:39:48.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:48.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:48.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500548+0000", 2024-04-26T03:39:48.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:48.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:48.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500549+0000", 2024-04-26T03:39:48.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3300000000000002e-07 2024-04-26T03:39:48.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:48.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500560+0000", 2024-04-26T03:39:48.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1316e-05 2024-04-26T03:39:48.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:48.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500573+0000", 2024-04-26T03:39:48.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2853999999999999e-05 2024-04-26T03:39:48.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:48.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500579+0000", 2024-04-26T03:39:48.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3090000000000001e-06 2024-04-26T03:39:48.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:48.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.514618+0000", 2024-04-26T03:39:48.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014038861 2024-04-26T03:39:48.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:48.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.514791+0000", 2024-04-26T03:39:48.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017323999999999999 2024-04-26T03:39:48.312 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:48.312 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:48.312 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:48.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.312 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.5s0 35/32 ECSubReadReply(tid=3, attrs_read=0))", 2024-04-26T03:39:48.312 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.500609+0000", 2024-04-26T03:39:48.312 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 56.799518001000003, 2024-04-26T03:39:48.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014706643, 2024-04-26T03:39:48.313 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:48.313 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:48.313 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:48.313 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:48.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:48.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500609+0000", 2024-04-26T03:39:48.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:48.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:48.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500609+0000", 2024-04-26T03:39:48.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:48.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:48.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500610+0000", 2024-04-26T03:39:48.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1900000000000002e-07 2024-04-26T03:39:48.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:48.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500621+0000", 2024-04-26T03:39:48.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1102e-05 2024-04-26T03:39:48.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:48.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500631+0000", 2024-04-26T03:39:48.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0197000000000001e-05 2024-04-26T03:39:48.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:48.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.500638+0000", 2024-04-26T03:39:48.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3770000000000001e-06 2024-04-26T03:39:48.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:48.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515285+0000", 2024-04-26T03:39:48.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014647004 2024-04-26T03:39:48.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:48.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515316+0000", 2024-04-26T03:39:48.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1244000000000001e-05 2024-04-26T03:39:48.316 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:48.316 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:48.317 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:48.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.317 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 35/32 ECSubWriteReply(tid=2, last_complete=34'2, committed=1, applied=1))", 2024-04-26T03:39:48.317 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.501067+0000", 2024-04-26T03:39:48.317 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 56.799060552999997, 2024-04-26T03:39:48.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013961199000000001, 2024-04-26T03:39:48.317 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:48.317 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:48.317 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:48.317 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:48.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:48.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501067+0000", 2024-04-26T03:39:48.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:48.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:48.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501067+0000", 2024-04-26T03:39:48.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:48.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:48.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501067+0000", 2024-04-26T03:39:48.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7599999999999996e-07 2024-04-26T03:39:48.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:48.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501072+0000", 2024-04-26T03:39:48.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7700000000000001e-06 2024-04-26T03:39:48.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:48.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501083+0000", 2024-04-26T03:39:48.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0652000000000001e-05 2024-04-26T03:39:48.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:48.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501090+0000", 2024-04-26T03:39:48.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5299999999999999e-06 2024-04-26T03:39:48.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:48.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515013+0000", 2024-04-26T03:39:48.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013922436999999999 2024-04-26T03:39:48.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:48.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515028+0000", 2024-04-26T03:39:48.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5034e-05 2024-04-26T03:39:48.320 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:48.320 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:48.320 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:48.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.320 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.5s0 35/32 ECSubReadReply(tid=3, attrs_read=0))", 2024-04-26T03:39:48.321 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.501907+0000", 2024-04-26T03:39:48.321 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 56.798219736, 2024-04-26T03:39:48.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013462257, 2024-04-26T03:39:48.321 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:48.321 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:48.321 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:48.321 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:48.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:48.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501907+0000", 2024-04-26T03:39:48.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:48.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:48.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501907+0000", 2024-04-26T03:39:48.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:48.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:48.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501908+0000", 2024-04-26T03:39:48.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2200000000000003e-07 2024-04-26T03:39:48.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:48.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501918+0000", 2024-04-26T03:39:48.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0329999999999999e-05 2024-04-26T03:39:48.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:48.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501945+0000", 2024-04-26T03:39:48.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6047999999999999e-05 2024-04-26T03:39:48.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:48.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.501951+0000", 2024-04-26T03:39:48.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7560000000000002e-06 2024-04-26T03:39:48.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:48.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515339+0000", 2024-04-26T03:39:48.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013387731 2024-04-26T03:39:48.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:48.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.515370+0000", 2024-04-26T03:39:48.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0670000000000003e-05 2024-04-26T03:39:48.324 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:48.324 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:48.324 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:48.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.324 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:67 3.5s0 3:a79a63ac:::smithi09235267-23:head [append 0~376832,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:48.324 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.518499+0000", 2024-04-26T03:39:48.324 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 56.781627858999997, 2024-04-26T03:39:48.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01578007, 2024-04-26T03:39:48.325 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:48.325 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:48.325 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:48.325 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:48.325 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:48.325 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:48.325 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 67 2024-04-26T03:39:48.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.325 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:48.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:48.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518499+0000", 2024-04-26T03:39:48.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:48.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:48.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518499+0000", 2024-04-26T03:39:48.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:48.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:48.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518501+0000", 2024-04-26T03:39:48.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9010000000000001e-06 2024-04-26T03:39:48.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:48.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518725+0000", 2024-04-26T03:39:48.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000224183 2024-04-26T03:39:48.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:48.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518740+0000", 2024-04-26T03:39:48.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4966e-05 2024-04-26T03:39:48.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:48.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518748+0000", 2024-04-26T03:39:48.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6399999999999997e-06 2024-04-26T03:39:48.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:48.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518798+0000", 2024-04-26T03:39:48.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0263000000000003e-05 2024-04-26T03:39:48.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:48.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518927+0000", 2024-04-26T03:39:48.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012898900000000001 2024-04-26T03:39:48.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:48.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.519605+0000", 2024-04-26T03:39:48.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00067774899999999995 2024-04-26T03:39:48.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:48.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.524447+0000", 2024-04-26T03:39:48.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0048424319999999998 2024-04-26T03:39:48.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:48.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.534195+0000", 2024-04-26T03:39:48.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097473149999999995 2024-04-26T03:39:48.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:48.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.534279+0000", 2024-04-26T03:39:48.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4631999999999999e-05 2024-04-26T03:39:48.329 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:48.329 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:48.329 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:48.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.330 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:68 3.5s0 3:a79a63ac:::smithi09235267-23:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:48.330 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.518766+0000", 2024-04-26T03:39:48.330 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 56.781360937999999, 2024-04-26T03:39:48.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015709772, 2024-04-26T03:39:48.330 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:48.330 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:48.330 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:48.330 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:48.330 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:48.330 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:48.330 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 68 2024-04-26T03:39:48.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.330 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:48.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:48.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518766+0000", 2024-04-26T03:39:48.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:48.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:48.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518766+0000", 2024-04-26T03:39:48.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:48.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:48.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518767+0000", 2024-04-26T03:39:48.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6899999999999997e-07 2024-04-26T03:39:48.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:48.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518771+0000", 2024-04-26T03:39:48.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-06 2024-04-26T03:39:48.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:48.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518777+0000", 2024-04-26T03:39:48.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3600000000000004e-06 2024-04-26T03:39:48.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:48.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518783+0000", 2024-04-26T03:39:48.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9490000000000001e-06 2024-04-26T03:39:48.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:48.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.522888+0000", 2024-04-26T03:39:48.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004105577 2024-04-26T03:39:48.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:48.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.522976+0000", 2024-04-26T03:39:48.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400999999999999e-05 2024-04-26T03:39:48.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:48.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.523380+0000", 2024-04-26T03:39:48.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00040403599999999998 2024-04-26T03:39:48.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:48.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.524449+0000", 2024-04-26T03:39:48.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00106897 2024-04-26T03:39:48.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:48.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.534390+0000", 2024-04-26T03:39:48.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099415249999999997 2024-04-26T03:39:48.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:48.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.534476+0000", 2024-04-26T03:39:48.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5884999999999995e-05 2024-04-26T03:39:48.335 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:48.335 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:48.335 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:48.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.335 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:69 3.5s0 3:a79a63ac:::smithi09235267-23:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-04-26T03:39:48.335 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:51.518790+0000", 2024-04-26T03:39:48.335 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 56.781336705999998, 2024-04-26T03:39:48.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018453847999999998, 2024-04-26T03:39:48.335 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:48.335 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:48.335 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:48.336 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:48.336 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:48.336 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:48.336 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 69 2024-04-26T03:39:48.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.336 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:48.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:48.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518790+0000", 2024-04-26T03:39:48.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:48.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:48.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518790+0000", 2024-04-26T03:39:48.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:48.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:48.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518791+0000", 2024-04-26T03:39:48.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9999999999999998e-07 2024-04-26T03:39:48.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:48.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518793+0000", 2024-04-26T03:39:48.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0949999999999998e-06 2024-04-26T03:39:48.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:48.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518799+0000", 2024-04-26T03:39:48.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.117e-06 2024-04-26T03:39:48.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:48.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.518804+0000", 2024-04-26T03:39:48.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8670000000000002e-06 2024-04-26T03:39:48.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:48.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.523575+0000", 2024-04-26T03:39:48.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004771155 2024-04-26T03:39:48.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:48.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.523646+0000", 2024-04-26T03:39:48.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0992000000000001e-05 2024-04-26T03:39:48.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:48.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.534812+0000", 2024-04-26T03:39:48.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011166099000000001 2024-04-26T03:39:48.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:48.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.534887+0000", 2024-04-26T03:39:48.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5094999999999998e-05 2024-04-26T03:39:48.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:48.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:51.537244+0000", 2024-04-26T03:39:48.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023569279999999999 2024-04-26T03:39:48.340 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:48.340 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:48.340 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:48.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.340 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:256 3.5s0 3:a64abe59:::smithi09235267-10:head [append 0~0] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T03:39:48.340 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:56.508641+0000", 2024-04-26T03:39:48.340 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.791486235000001, 2024-04-26T03:39:48.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013570576000000001, 2024-04-26T03:39:48.341 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:48.341 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:48.341 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:48.341 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:48.341 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:48.341 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:48.341 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 256 2024-04-26T03:39:48.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.341 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:48.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:48.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508641+0000", 2024-04-26T03:39:48.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:48.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:48.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508641+0000", 2024-04-26T03:39:48.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:48.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:48.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508644+0000", 2024-04-26T03:39:48.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7760000000000002e-06 2024-04-26T03:39:48.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:48.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508655+0000", 2024-04-26T03:39:48.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1049e-05 2024-04-26T03:39:48.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:48.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508727+0000", 2024-04-26T03:39:48.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1907000000000001e-05 2024-04-26T03:39:48.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:48.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508737+0000", 2024-04-26T03:39:48.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0691e-05 2024-04-26T03:39:48.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:48.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508889+0000", 2024-04-26T03:39:48.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015123099999999999 2024-04-26T03:39:48.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-04-26T03:39:48.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508905+0000", 2024-04-26T03:39:48.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6042999999999999e-05 2024-04-26T03:39:48.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:48.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.511938+0000", 2024-04-26T03:39:48.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030333119999999998 2024-04-26T03:39:48.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:48.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.512054+0000", 2024-04-26T03:39:48.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115742 2024-04-26T03:39:48.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:48.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.512811+0000", 2024-04-26T03:39:48.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00075763100000000004 2024-04-26T03:39:48.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:48.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.521838+0000", 2024-04-26T03:39:48.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0090266489999999994 2024-04-26T03:39:48.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:48.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.522144+0000", 2024-04-26T03:39:48.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030574600000000002 2024-04-26T03:39:48.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:48.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.522211+0000", 2024-04-26T03:39:48.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7799000000000006e-05 2024-04-26T03:39:48.346 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:48.346 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:48.346 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:48.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.346 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:257 3.5s0 3:a64abe59:::smithi09235267-10:head [rollback 2] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T03:39:48.346 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:56.508783+0000", 2024-04-26T03:39:48.346 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.791344215000002, 2024-04-26T03:39:48.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014341474999999999, 2024-04-26T03:39:48.347 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:48.347 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:48.347 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:48.347 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:48.347 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:48.347 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:48.347 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 257 2024-04-26T03:39:48.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.347 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:48.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:48.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508783+0000", 2024-04-26T03:39:48.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:48.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:48.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508783+0000", 2024-04-26T03:39:48.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:48.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:48.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508784+0000", 2024-04-26T03:39:48.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9999999999999995e-07 2024-04-26T03:39:48.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:48.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508789+0000", 2024-04-26T03:39:48.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4280000000000004e-06 2024-04-26T03:39:48.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:48.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508800+0000", 2024-04-26T03:39:48.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.076e-05 2024-04-26T03:39:48.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:48.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508808+0000", 2024-04-26T03:39:48.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.593e-06 2024-04-26T03:39:48.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:48.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508993+0000", 2024-04-26T03:39:48.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018527999999999999 2024-04-26T03:39:48.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-04-26T03:39:48.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.509001+0000", 2024-04-26T03:39:48.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6529999999999994e-06 2024-04-26T03:39:48.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:48.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.513192+0000", 2024-04-26T03:39:48.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041913009999999997 2024-04-26T03:39:48.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:48.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.513348+0000", 2024-04-26T03:39:48.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000155654 2024-04-26T03:39:48.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:48.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.514509+0000", 2024-04-26T03:39:48.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011615169999999999 2024-04-26T03:39:48.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:48.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.521845+0000", 2024-04-26T03:39:48.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0073361140000000003 2024-04-26T03:39:48.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:48.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.523044+0000", 2024-04-26T03:39:48.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001198939 2024-04-26T03:39:48.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:48.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.523124+0000", 2024-04-26T03:39:48.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0235999999999995e-05 2024-04-26T03:39:48.352 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:48.352 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:48.352 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:48.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.352 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:258 3.5s0 3:a64abe59:::smithi09235267-10:head [append 0~0] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T03:39:48.353 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:38:56.508917+0000", 2024-04-26T03:39:48.353 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.791210370999998, 2024-04-26T03:39:48.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014562304999999999, 2024-04-26T03:39:48.353 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:48.353 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:48.353 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:48.353 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:48.353 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:48.353 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:48.353 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 258 2024-04-26T03:39:48.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.353 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:48.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:48.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508917+0000", 2024-04-26T03:39:48.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:48.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:48.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508917+0000", 2024-04-26T03:39:48.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:48.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:48.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508918+0000", 2024-04-26T03:39:48.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5600000000000004e-07 2024-04-26T03:39:48.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:48.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508922+0000", 2024-04-26T03:39:48.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4730000000000002e-06 2024-04-26T03:39:48.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:48.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508937+0000", 2024-04-26T03:39:48.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5114e-05 2024-04-26T03:39:48.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:48.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.508948+0000", 2024-04-26T03:39:48.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1222e-05 2024-04-26T03:39:48.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:48.356 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.509035+0000", 2024-04-26T03:39:48.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7032000000000003e-05 2024-04-26T03:39:48.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.356 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-04-26T03:39:48.356 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.509040+0000", 2024-04-26T03:39:48.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6299999999999997e-06 2024-04-26T03:39:48.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.356 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:48.356 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.515348+0000", 2024-04-26T03:39:48.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0063082959999999997 2024-04-26T03:39:48.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.356 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.515613+0000", 2024-04-26T03:39:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026497100000000002 2024-04-26T03:39:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.516295+0000", 2024-04-26T03:39:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00068196499999999998 2024-04-26T03:39:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.521977+0000", 2024-04-26T03:39:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056815759999999998 2024-04-26T03:39:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:48.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.523407+0000", 2024-04-26T03:39:48.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014299409999999999 2024-04-26T03:39:48.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:48.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:38:56.523479+0000", 2024-04-26T03:39:48.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2229000000000003e-05 2024-04-26T03:39:48.358 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:48.358 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:48.358 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:48.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.358 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:318 3.5s0 3:a79a63ac:::smithi09235267-23:head [delete] snapc 7=[7,6,5] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e44)", 2024-04-26T03:39:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:00.528087+0000", 2024-04-26T03:39:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.772039681999999, 2024-04-26T03:39:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.020004624, 2024-04-26T03:39:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 318 2024-04-26T03:39:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:00.528087+0000", 2024-04-26T03:39:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:00.528087+0000", 2024-04-26T03:39:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:00.528089+0000", 2024-04-26T03:39:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9120000000000001e-06 2024-04-26T03:39:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:00.528096+0000", 2024-04-26T03:39:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0829999999999998e-06 2024-04-26T03:39:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:00.528112+0000", 2024-04-26T03:39:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5619000000000001e-05 2024-04-26T03:39:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:00.528121+0000", 2024-04-26T03:39:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6789999999999994e-06 2024-04-26T03:39:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:48.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:00.528185+0000", 2024-04-26T03:39:48.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4156e-05 2024-04-26T03:39:48.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-04-26T03:39:48.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:00.528190+0000", 2024-04-26T03:39:48.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9010000000000002e-06 2024-04-26T03:39:48.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:48.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:00.534892+0000", 2024-04-26T03:39:48.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067026239999999999 2024-04-26T03:39:48.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-04-26T03:39:48.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:00.534897+0000", 2024-04-26T03:39:48.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9389999999999996e-06 2024-04-26T03:39:48.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:48.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.540918+0000", 2024-04-26T03:39:48.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0060203679999999 2024-04-26T03:39:48.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:48.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.541113+0000", 2024-04-26T03:39:48.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019529200000000001 2024-04-26T03:39:48.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:48.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.541689+0000", 2024-04-26T03:39:48.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00057578299999999996 2024-04-26T03:39:48.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:48.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.544502+0000", 2024-04-26T03:39:48.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028131950000000001 2024-04-26T03:39:48.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:48.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.547870+0000", 2024-04-26T03:39:48.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0033683039999999999 2024-04-26T03:39:48.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:48.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:01.548092+0000", 2024-04-26T03:39:48.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000221769 2024-04-26T03:39:48.365 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:48.365 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:48.365 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:48.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.365 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.13s3 50/44 ECSubWrite(tid=74, reqid=client.4556.0:446, at_version=50'37, trim_to=0'0, roll_forward_to=50'36))", 2024-04-26T03:39:48.365 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:05.702348+0000", 2024-04-26T03:39:48.365 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.597779086000003, 2024-04-26T03:39:48.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013733293000000001, 2024-04-26T03:39:48.366 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:48.366 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:48.366 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:48.366 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:48.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:48.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.702348+0000", 2024-04-26T03:39:48.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:48.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:48.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.702348+0000", 2024-04-26T03:39:48.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:48.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:48.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.702349+0000", 2024-04-26T03:39:48.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5400000000000001e-07 2024-04-26T03:39:48.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:48.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.702837+0000", 2024-04-26T03:39:48.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00048857299999999996 2024-04-26T03:39:48.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:48.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.702851+0000", 2024-04-26T03:39:48.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3723999999999999e-05 2024-04-26T03:39:48.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:48.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.702862+0000", 2024-04-26T03:39:48.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1571e-05 2024-04-26T03:39:48.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:48.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.702925+0000", 2024-04-26T03:39:48.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2890000000000003e-05 2024-04-26T03:39:48.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:48.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.702942+0000", 2024-04-26T03:39:48.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6857000000000001e-05 2024-04-26T03:39:48.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:48.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716045+0000", 2024-04-26T03:39:48.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013102744 2024-04-26T03:39:48.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:48.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.716081+0000", 2024-04-26T03:39:48.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6380000000000001e-05 2024-04-26T03:39:48.369 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:48.369 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:48.369 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:48.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.370 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:462 3.5s0 3:a64abe59:::smithi09235267-10:head [append 0~753664] snapc 8=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:48.370 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:05.749661+0000", 2024-04-26T03:39:48.370 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.550466176999997, 2024-04-26T03:39:48.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014030855, 2024-04-26T03:39:48.370 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:48.370 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:48.370 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:48.370 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:48.370 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:48.370 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:48.370 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 462 2024-04-26T03:39:48.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.370 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:48.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:48.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.749661+0000", 2024-04-26T03:39:48.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:48.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:48.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.749661+0000", 2024-04-26T03:39:48.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:48.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:48.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.749663+0000", 2024-04-26T03:39:48.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.722e-06 2024-04-26T03:39:48.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:48.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.749971+0000", 2024-04-26T03:39:48.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000308475 2024-04-26T03:39:48.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:48.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.749985+0000", 2024-04-26T03:39:48.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4191e-05 2024-04-26T03:39:48.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:48.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.749995+0000", 2024-04-26T03:39:48.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0020000000000001e-05 2024-04-26T03:39:48.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:48.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.750050+0000", 2024-04-26T03:39:48.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5000000000000002e-05 2024-04-26T03:39:48.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:48.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.750121+0000", 2024-04-26T03:39:48.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0669999999999999e-05 2024-04-26T03:39:48.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:48.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.750769+0000", 2024-04-26T03:39:48.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00064844000000000002 2024-04-26T03:39:48.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:48.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.763254+0000", 2024-04-26T03:39:48.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012484215999999999 2024-04-26T03:39:48.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:48.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.763622+0000", 2024-04-26T03:39:48.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00036791699999999999 2024-04-26T03:39:48.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:48.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.763692+0000", 2024-04-26T03:39:48.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0203999999999994e-05 2024-04-26T03:39:48.375 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:48.375 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:48.375 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:48.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.375 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:466 3.5s0 3:a64abe59:::smithi09235267-10:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:48.375 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:05.755663+0000", 2024-04-26T03:39:48.375 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.54446368, 2024-04-26T03:39:48.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021406392999999999, 2024-04-26T03:39:48.375 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:48.375 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:48.375 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:48.376 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:48.376 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:48.376 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:48.376 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 466 2024-04-26T03:39:48.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.376 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:48.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:48.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755663+0000", 2024-04-26T03:39:48.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:48.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:48.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755663+0000", 2024-04-26T03:39:48.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:48.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:48.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755664+0000", 2024-04-26T03:39:48.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6600000000000003e-07 2024-04-26T03:39:48.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:48.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755665+0000", 2024-04-26T03:39:48.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.389e-06 2024-04-26T03:39:48.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:48.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755669+0000", 2024-04-26T03:39:48.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2024-04-26T03:39:48.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:48.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755672+0000", 2024-04-26T03:39:48.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7190000000000001e-06 2024-04-26T03:39:48.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:48.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.764511+0000", 2024-04-26T03:39:48.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0088397730000000004 2024-04-26T03:39:48.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-04-26T03:39:48.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.764598+0000", 2024-04-26T03:39:48.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6438000000000001e-05 2024-04-26T03:39:48.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:48.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.766650+0000", 2024-04-26T03:39:48.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020521430000000002 2024-04-26T03:39:48.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:48.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.766698+0000", 2024-04-26T03:39:48.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8133000000000001e-05 2024-04-26T03:39:48.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:48.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.777070+0000", 2024-04-26T03:39:48.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010371672 2024-04-26T03:39:48.380 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:48.380 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:48.380 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:48.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.380 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:467 3.5s0 3:a79a63ac:::smithi09235267-23:head [stat,delete] snapc 8=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T03:39:48.380 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:05.755675+0000", 2024-04-26T03:39:48.380 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.544451924999997, 2024-04-26T03:39:48.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020820686000000001, 2024-04-26T03:39:48.380 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:48.381 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:48.381 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:48.381 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:48.381 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:48.381 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:48.381 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 467 2024-04-26T03:39:48.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.381 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:48.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:48.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755675+0000", 2024-04-26T03:39:48.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:48.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:48.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755675+0000", 2024-04-26T03:39:48.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:48.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:48.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755675+0000", 2024-04-26T03:39:48.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0900000000000001e-07 2024-04-26T03:39:48.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:48.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755677+0000", 2024-04-26T03:39:48.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.406e-06 2024-04-26T03:39:48.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:48.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755680+0000", 2024-04-26T03:39:48.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2329999999999998e-06 2024-04-26T03:39:48.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:48.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.755684+0000", 2024-04-26T03:39:48.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.472e-06 2024-04-26T03:39:48.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:48.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.764647+0000", 2024-04-26T03:39:48.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0089629900000000005 2024-04-26T03:39:48.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:48.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.764702+0000", 2024-04-26T03:39:48.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5748000000000002e-05 2024-04-26T03:39:48.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:48.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.765004+0000", 2024-04-26T03:39:48.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030170000000000002 2024-04-26T03:39:48.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:48.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.776390+0000", 2024-04-26T03:39:48.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011386454000000001 2024-04-26T03:39:48.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:48.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.776455+0000", 2024-04-26T03:39:48.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4304e-05 2024-04-26T03:39:48.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:48.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:05.776496+0000", 2024-04-26T03:39:48.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1170000000000001e-05 2024-04-26T03:39:48.385 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:48.385 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:48.385 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:48.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.386 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:615 3.5s0 3:a79a63ac:::smithi09235267-23:head [create,setxattr Z]ca\\SkkXhsYi]s_mecUZQfPbbjaqacZrSObr_`a (734) in=40b,setxattr i (1998) in=1b,setxattr OQosgYUVOPebZdfit`Xtcc]XX_pdNqdkSof_lrck (1822) in=40b,setxattr mrQaYTllqfj^q]UYaoR`drdgY]bVfYQ\\SgOe_gca (1789) in=40b,setxattr ZaeONfchPlXaOYt[dQNtaR[NNiaSangpfQNVhVY[ (1646) in=40b,setxattr epskr[spqr^j^XqcosqPQ (1998) in=21b,setxattr WarqdipUlkSQhTbQR[NhrYRrda[QmOgj^VqWWero (313) in=40b,setxattr f\\fnrpSa`gWXj`Yo`dQY]Vdr[Z^`gkVjb^enmUig (1645) in=40b,setxattr bjWaN_RafX`the (1998) in=14b,setxattr nmdNljVWb`aR_`Oeb[l_OntR_YgidWjRabRf[]Ye (10596) in=40b,setxattr [nhisfmkVd_n_YV\\efebsW_pQaiWle_]o]rhc[bO (14456) in=40b,setxattr []hmkY]V]hkrrd]gr\\c\\hPq_smnTfakPZRNg^bY[ (1649) in=40b,setxattr ZZ\\\\jZnU`jtY\\[k]hnm`r_OYQeWTg\\rnQdX^rbhg (807) in=40b,setxattr QT_d^gX[dNZPP\\VapoetR^[gpVcZQXfhp^`l_Vh[ (66) in=40b,setxattr ee[ (1998) in=3b,setxattr RY\\sp]UifT[Zt\\[VenNWarqiiNSXsdnktf^QhT[U (487) in=40b,setxattr hUnl\\fnak]U``QN^rs`a^kfaQ`NoTVNnPXYbdmre (13680) in=40b,setxattr gha[S`afeloOcnfbeOkUV]T]ZTYWaOeaS]bbVa\\Q (1226) in=40b,setxattr NaraXbinjWQraPh`Pp]qnPWgWfZf[WaUb]oOY]^r (1611) in=40b,setxattr tpfZiddi]a^RVWpgOZormUeqYosdY_UsSpiQQRgj (1225) in=40b] snapc d=[d,c,b,a] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-26T03:39:48.386 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:11.732685+0000", 2024-04-26T03:39:48.386 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.567441817000002, 2024-04-26T03:39:48.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013378048999999999, 2024-04-26T03:39:48.386 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:48.386 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:48.386 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T03:39:48.386 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:48.386 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:48.386 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:48.386 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 615 2024-04-26T03:39:48.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.386 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:48.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:48.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732685+0000", 2024-04-26T03:39:48.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:48.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:48.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732685+0000", 2024-04-26T03:39:48.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:48.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:48.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732688+0000", 2024-04-26T03:39:48.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8320000000000002e-06 2024-04-26T03:39:48.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:48.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732753+0000", 2024-04-26T03:39:48.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4678999999999995e-05 2024-04-26T03:39:48.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:48.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732774+0000", 2024-04-26T03:39:48.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1637999999999998e-05 2024-04-26T03:39:48.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:48.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732784+0000", 2024-04-26T03:39:48.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5410000000000006e-06 2024-04-26T03:39:48.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:48.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.732856+0000", 2024-04-26T03:39:48.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2020000000000005e-05 2024-04-26T03:39:48.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:48.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.733002+0000", 2024-04-26T03:39:48.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000145697 2024-04-26T03:39:48.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.733773+0000", 2024-04-26T03:39:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000770774 2024-04-26T03:39:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.735028+0000", 2024-04-26T03:39:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0012551089999999999 2024-04-26T03:39:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-04-26T03:39:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.745960+0000", 2024-04-26T03:39:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010932737999999999 2024-04-26T03:39:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:48.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:11.746063+0000", 2024-04-26T03:39:48.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103021 2024-04-26T03:39:48.391 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:48.391 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:48.391 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:48.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.391 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:818 3.5s0 3:a64abe59:::smithi09235267-10:head [read 0~2605056,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-04-26T03:39:48.391 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:22.692385+0000", 2024-04-26T03:39:48.391 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 25.607742518999999, 2024-04-26T03:39:48.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018880436, 2024-04-26T03:39:48.391 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:48.391 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:48.392 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:48.392 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:48.392 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:48.392 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:48.392 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 818 2024-04-26T03:39:48.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.392 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:48.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:48.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692385+0000", 2024-04-26T03:39:48.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:48.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692385+0000", 2024-04-26T03:39:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692385+0000", 2024-04-26T03:39:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2900000000000003e-07 2024-04-26T03:39:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692390+0000", 2024-04-26T03:39:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7380000000000002e-06 2024-04-26T03:39:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692400+0000", 2024-04-26T03:39:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5880000000000004e-06 2024-04-26T03:39:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692406+0000", 2024-04-26T03:39:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5529999999999998e-06 2024-04-26T03:39:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692508+0000", 2024-04-26T03:39:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101698 2024-04-26T03:39:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-04-26T03:39:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692515+0000", 2024-04-26T03:39:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1629999999999999e-06 2024-04-26T03:39:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.697431+0000", 2024-04-26T03:39:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049163280000000002 2024-04-26T03:39:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.697602+0000", 2024-04-26T03:39:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017039999999999999 2024-04-26T03:39:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.711265+0000", 2024-04-26T03:39:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013663339 2024-04-26T03:39:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4556.0:819 3.5s0 3:a64abe59:::smithi09235267-10:head [read 0~2605056,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-04-26T03:39:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:22.692428+0000", 2024-04-26T03:39:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 25.607699478000001, 2024-04-26T03:39:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020633177999999999, 2024-04-26T03:39:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-04-26T03:39:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-04-26T03:39:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4556", 2024-04-26T03:39:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1482977591", 2024-04-26T03:39:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 819 2024-04-26T03:39:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692428+0000", 2024-04-26T03:39:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692428+0000", 2024-04-26T03:39:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692428+0000", 2024-04-26T03:39:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2024-04-26T03:39:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692432+0000", 2024-04-26T03:39:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6320000000000001e-06 2024-04-26T03:39:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692437+0000", 2024-04-26T03:39:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.164e-06 2024-04-26T03:39:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692441+0000", 2024-04-26T03:39:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7459999999999998e-06 2024-04-26T03:39:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692537+0000", 2024-04-26T03:39:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6175000000000007e-05 2024-04-26T03:39:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-04-26T03:39:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.692541+0000", 2024-04-26T03:39:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6279999999999998e-06 2024-04-26T03:39:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.699526+0000", 2024-04-26T03:39:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0069851940000000001 2024-04-26T03:39:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-04-26T03:39:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.699637+0000", 2024-04-26T03:39:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000111437 2024-04-26T03:39:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:22.713061+0000", 2024-04-26T03:39:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013423554000000001 2024-04-26T03:39:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.8s5 80/63 ECSubWrite(tid=321, reqid=client.4556.0:897, at_version=80'111, trim_to=0'0, roll_forward_to=80'110))", 2024-04-26T03:39:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-04-26T03:39:35.717514+0000", 2024-04-26T03:39:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.582612823, 2024-04-26T03:39:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018440938, 2024-04-26T03:39:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-04-26T03:39:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-04-26T03:39:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-04-26T03:39:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-04-26T03:39:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-04-26T03:39:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.717514+0000", 2024-04-26T03:39:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-04-26T03:39:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.717514+0000", 2024-04-26T03:39:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-04-26T03:39:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-04-26T03:39:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.717515+0000", 2024-04-26T03:39:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1900000000000002e-07 2024-04-26T03:39:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-04-26T03:39:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.719503+0000", 2024-04-26T03:39:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019876220000000001 2024-04-26T03:39:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-04-26T03:39:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.719513+0000", 2024-04-26T03:39:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0638e-05 2024-04-26T03:39:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-04-26T03:39:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.719519+0000", 2024-04-26T03:39:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9190000000000004e-06 2024-04-26T03:39:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-04-26T03:39:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.719572+0000", 2024-04-26T03:39:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3158999999999997e-05 2024-04-26T03:39:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-04-26T03:39:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.719579+0000", 2024-04-26T03:39:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4420000000000003e-06 2024-04-26T03:39:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-04-26T03:39:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.735911+0000", 2024-04-26T03:39:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016332583000000001 2024-04-26T03:39:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-04-26T03:39:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-04-26T03:39:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-04-26T03:39:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-04-26T03:39:35.735955+0000", 2024-04-26T03:39:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3856000000000002e-05 2024-04-26T03:39:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:48.406 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:48.406 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-04-26T03:39:48.406 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-04-26T03:39:48.406 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-04-26T03:39:48.582 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:39:48.582 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-04-26T03:39:48.595 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":116,"stamp":"2024-04-26T03:39:48.064480+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":146,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":753048,"kb_used_data":204116,"kb_used_omap":24,"kb_used_meta":548647,"kb_avail":1509196392,"statfs":{"total":1546188226560,"available":1545417105408,"internally_reserved":0,"allocated":209014784,"data_stored":267441733,"data_compressed":2152919,"data_compressed_allocated":4386816,"data_compressed_original":9666560,"omap_allocated":25522,"internal_metadata":561814606},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-133054464,"num_objects":-85,"num_object_clones":-41,"num_object_copies":-595,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-85,"num_whiteouts":-7,"num_read":-336,"num_read_kb":-236694,"num_write":-727,"num_write_kb":-123488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-23,"num_bytes_recovered":-31670272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000810"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":136,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T03:39:45.771642+0000","last_change":"2024-04-26T03:38:37.359252+0000","last_active":"2024-04-26T03:39:45.771642+0000","last_peered":"2024-04-26T03:39:45.771642+0000","last_clean":"2024-04-26T03:39:45.771642+0000","last_became_active":"2024-04-26T03:38:37.358910+0000","last_became_peered":"2024-04-26T03:38:37.358910+0000","last_unstale":"2024-04-26T03:39:45.771642+0000","last_undegraded":"2024-04-26T03:39:45.771642+0000","last_fullsized":"2024-04-26T03:39:45.771642+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:04.186001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":146,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T03:39:47.176377+0000","last_change":"2024-04-26T03:39:45.393243+0000","last_active":"2024-04-26T03:39:47.176377+0000","last_peered":"2024-04-26T03:39:47.176377+0000","last_clean":"2024-04-26T03:39:47.176377+0000","last_became_active":"2024-04-26T03:38:36.484388+0000","last_became_peered":"2024-04-26T03:38:36.484388+0000","last_unstale":"2024-04-26T03:39:47.176377+0000","last_undegraded":"2024-04-26T03:39:47.176377+0000","last_fullsized":"2024-04-26T03:39:47.176377+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:39:45.393105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:39:45.393105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:50.701189+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,13],"acting":[8,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":137,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T03:39:47.149313+0000","last_change":"2024-04-26T03:38:38.187714+0000","last_active":"2024-04-26T03:39:47.149313+0000","last_peered":"2024-04-26T03:39:47.149313+0000","last_clean":"2024-04-26T03:39:47.149313+0000","last_became_active":"2024-04-26T03:38:38.187383+0000","last_became_peered":"2024-04-26T03:38:38.187383+0000","last_unstale":"2024-04-26T03:39:47.149313+0000","last_undegraded":"2024-04-26T03:39:47.149313+0000","last_fullsized":"2024-04-26T03:39:47.149313+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:59.062623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4],"acting":[9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":129,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T03:39:42.815417+0000","last_change":"2024-04-26T03:38:37.255389+0000","last_active":"2024-04-26T03:39:42.815417+0000","last_peered":"2024-04-26T03:39:42.815417+0000","last_clean":"2024-04-26T03:39:42.815417+0000","last_became_active":"2024-04-26T03:38:37.255066+0000","last_became_peered":"2024-04-26T03:38:37.255066+0000","last_unstale":"2024-04-26T03:39:42.815417+0000","last_undegraded":"2024-04-26T03:39:42.815417+0000","last_fullsized":"2024-04-26T03:39:42.815417+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:43.860480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":134,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T03:39:45.702558+0000","last_change":"2024-04-26T03:39:45.538938+0000","last_active":"2024-04-26T03:39:45.702558+0000","last_peered":"2024-04-26T03:39:45.702558+0000","last_clean":"2024-04-26T03:39:45.702558+0000","last_became_active":"2024-04-26T03:38:48.475155+0000","last_became_peered":"2024-04-26T03:38:48.475155+0000","last_unstale":"2024-04-26T03:39:45.702558+0000","last_undegraded":"2024-04-26T03:39:45.702558+0000","last_fullsized":"2024-04-26T03:39:45.702558+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:39:45.538842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:39:45.538842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:48.125744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,12],"acting":[11,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":138,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T03:39:46.833770+0000","last_change":"2024-04-26T03:38:37.255450+0000","last_active":"2024-04-26T03:39:46.833770+0000","last_peered":"2024-04-26T03:39:46.833770+0000","last_clean":"2024-04-26T03:39:46.833770+0000","last_became_active":"2024-04-26T03:38:37.255181+0000","last_became_peered":"2024-04-26T03:38:37.255181+0000","last_unstale":"2024-04-26T03:39:46.833770+0000","last_undegraded":"2024-04-26T03:39:46.833770+0000","last_fullsized":"2024-04-26T03:39:46.833770+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:00.449388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0],"acting":[14,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":131,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T03:39:43.827128+0000","last_change":"2024-04-26T03:38:36.431325+0000","last_active":"2024-04-26T03:39:43.827128+0000","last_peered":"2024-04-26T03:39:43.827128+0000","last_clean":"2024-04-26T03:39:43.827128+0000","last_became_active":"2024-04-26T03:38:36.431121+0000","last_became_peered":"2024-04-26T03:38:36.431121+0000","last_unstale":"2024-04-26T03:39:43.827128+0000","last_undegraded":"2024-04-26T03:39:43.827128+0000","last_fullsized":"2024-04-26T03:39:43.827128+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:53.294619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":138,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T03:39:46.833143+0000","last_change":"2024-04-26T03:38:36.431667+0000","last_active":"2024-04-26T03:39:46.833143+0000","last_peered":"2024-04-26T03:39:46.833143+0000","last_clean":"2024-04-26T03:39:46.833143+0000","last_became_active":"2024-04-26T03:38:36.431449+0000","last_became_peered":"2024-04-26T03:38:36.431449+0000","last_unstale":"2024-04-26T03:39:46.833143+0000","last_undegraded":"2024-04-26T03:39:46.833143+0000","last_fullsized":"2024-04-26T03:39:46.833143+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:38:35.319452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:47.562648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"27'157","reported_seq":347,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T03:39:47.178486+0000","last_change":"2024-04-26T03:39:42.110436+0000","last_active":"2024-04-26T03:39:47.178486+0000","last_peered":"2024-04-26T03:39:47.178486+0000","last_clean":"2024-04-26T03:39:47.178486+0000","last_became_active":"2024-04-26T03:38:34.389099+0000","last_became_peered":"2024-04-26T03:38:34.389099+0000","last_unstale":"2024-04-26T03:39:47.178486+0000","last_undegraded":"2024-04-26T03:39:47.178486+0000","last_fullsized":"2024-04-26T03:39:47.178486+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'157","last_scrub_stamp":"2024-04-26T03:39:42.110300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:32.306810+0000","last_clean_scrub_stamp":"2024-04-26T03:39:42.110300+0000","objects_scrubbed":2,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:11.925418+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":13852,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":24,"seq":103079215123,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50620,"kb_used_data":15276,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94321220,"statfs":{"total":96636764160,"available":96584929280,"internally_reserved":0,"allocated":15642624,"data_stored":19199959,"data_compressed":136565,"data_compressed_allocated":294912,"data_compressed_original":1036288,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,4,6,7,8,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215123,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43232,"kb_used_data":12052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94328608,"statfs":{"total":96636764160,"available":96592494592,"internally_reserved":0,"allocated":12341248,"data_stored":16788667,"data_compressed":131739,"data_compressed_allocated":266240,"data_compressed_original":532480,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":23,"seq":98784247829,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53232,"kb_used_data":17892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94318608,"statfs":{"total":96636764160,"available":96582254592,"internally_reserved":0,"allocated":18321408,"data_stored":24597943,"data_compressed":129639,"data_compressed_allocated":262144,"data_compressed_original":524288,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280530,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47400,"kb_used_data":16216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94324440,"statfs":{"total":96636764160,"available":96588226560,"internally_reserved":0,"allocated":16605184,"data_stored":20746225,"data_compressed":142857,"data_compressed_allocated":307200,"data_compressed_original":1060864,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,4,5,7,8,9,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280530,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1044480,"data_stored":767411,"data_compressed":137972,"data_compressed_allocated":278528,"data_compressed_original":557056,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,3,4,5,6,8,9,10,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":22,"seq":94489280531,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39556,"kb_used_data":12528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94332284,"statfs":{"total":96636764160,"available":96596258816,"internally_reserved":0,"allocated":12828672,"data_stored":13909457,"data_compressed":135931,"data_compressed_allocated":274432,"data_compressed_original":548864,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[2,4,5,6,7,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280529,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41848,"kb_used_data":10660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94329992,"statfs":{"total":96636764160,"available":96593911808,"internally_reserved":0,"allocated":10915840,"data_stored":13061585,"data_compressed":135931,"data_compressed_allocated":274432,"data_compressed_original":548864,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,2,4,7,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280530,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55280,"kb_used_data":24092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94316560,"statfs":{"total":96636764160,"available":96580157440,"internally_reserved":0,"allocated":24670208,"data_stored":34250193,"data_compressed":135931,"data_compressed_allocated":274432,"data_compressed_original":548864,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[1,2,3,4,7,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313235,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66924,"kb_used_data":17692,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94304916,"statfs":{"total":96636764160,"available":96568233984,"internally_reserved":0,"allocated":18116608,"data_stored":22373758,"data_compressed":133839,"data_compressed_allocated":270336,"data_compressed_original":540672,"omap_allocated":1596,"internal_metadata":50395588},"hb_peers":[0,1,2,4,5,6,7,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":12,"up_from":22,"seq":94489280529,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343800,"statfs":{"total":96636764160,"available":96608051200,"internally_reserved":0,"allocated":1044480,"data_stored":767411,"data_compressed":137972,"data_compressed_allocated":278528,"data_compressed_original":557056,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,7,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280529,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343800,"statfs":{"total":96636764160,"available":96608051200,"internally_reserved":0,"allocated":1044480,"data_stored":767411,"data_compressed":137972,"data_compressed_allocated":278528,"data_compressed_original":557056,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[2,4,6,7,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247826,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68944,"kb_used_data":15548,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94302896,"statfs":{"total":96636764160,"available":96566165504,"internally_reserved":0,"allocated":15921152,"data_stored":21609812,"data_compressed":127535,"data_compressed_allocated":258048,"data_compressed_original":516096,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,6,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280529,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38484,"kb_used_data":11456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94333356,"statfs":{"total":96636764160,"available":96597356544,"internally_reserved":0,"allocated":11730944,"data_stored":17608145,"data_compressed":135931,"data_compressed_allocated":274432,"data_compressed_original":548864,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,3,5,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":23,"seq":98784247828,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74824,"kb_used_data":21436,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94297016,"statfs":{"total":96636764160,"available":96560144384,"internally_reserved":0,"allocated":21950464,"data_stored":31823287,"data_compressed":129639,"data_compressed_allocated":262144,"data_compressed_original":524288,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,4,6,7,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280529,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40544,"kb_used_data":13516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94331296,"statfs":{"total":96636764160,"available":96595247104,"internally_reserved":0,"allocated":13840384,"data_stored":15330769,"data_compressed":135931,"data_compressed_allocated":274432,"data_compressed_original":548864,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,4,5,7,8,9,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247826,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43880,"kb_used_data":12692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94327960,"statfs":{"total":96636764160,"available":96591831040,"internally_reserved":0,"allocated":12996608,"data_stored":13839700,"data_compressed":127535,"data_compressed_allocated":258048,"data_compressed_original":516096,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T03:39:48.596 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-04-26T03:39:48.596 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-26T03:39:49.851 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-26T03:39:50.850 INFO:teuthology.orchestra.run.smithi028.stderr:noscrub is unset 2024-04-26T03:39:50.867 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-26T03:39:51.851 INFO:teuthology.orchestra.run.smithi028.stderr:nodeep-scrub is unset 2024-04-26T03:39:53.597 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-04-26T03:39:53.727 INFO:teuthology.orchestra.run.smithi028.stdout:mon.a: {} 2024-04-26T03:39:53.727 INFO:teuthology.orchestra.run.smithi028.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-26T03:39:53.737 INFO:teuthology.orchestra.run.smithi028.stdout:mon.b: {} 2024-04-26T03:39:53.737 INFO:teuthology.orchestra.run.smithi028.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-26T03:39:53.747 INFO:teuthology.orchestra.run.smithi028.stdout:mon.c: {} 2024-04-26T03:39:53.748 INFO:teuthology.orchestra.run.smithi028.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-26T03:39:53.758 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-04-26T03:39:54.868 INFO:teuthology.orchestra.run.smithi028.stderr:reweighted osd.0 to 1 (10000) 2024-04-26T03:39:54.885 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-04-26T03:39:55.875 INFO:teuthology.orchestra.run.smithi028.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-04-26T03:39:55.892 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-04-26T03:39:56.883 INFO:teuthology.orchestra.run.smithi028.stderr:reweighted osd.4 to 1 (10000) 2024-04-26T03:39:56.900 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-04-26T03:39:57.888 INFO:teuthology.orchestra.run.smithi028.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-04-26T03:39:57.906 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 8 1 2024-04-26T03:39:58.895 INFO:teuthology.orchestra.run.smithi028.stderr:reweighted osd.8 to 1 (10000) 2024-04-26T03:39:58.913 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 8 1 2024-04-26T03:39:59.901 INFO:teuthology.orchestra.run.smithi028.stderr:set osd.8 primary-affinity to 1 (8655362) 2024-04-26T03:39:59.919 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 12 1 2024-04-26T03:40:00.907 INFO:teuthology.orchestra.run.smithi028.stderr:reweighted osd.12 to 1 (10000) 2024-04-26T03:40:00.924 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 12 1 2024-04-26T03:40:01.912 INFO:teuthology.orchestra.run.smithi028.stderr:set osd.12 primary-affinity to 1 (8655362) 2024-04-26T03:40:01.929 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-04-26T03:40:02.920 INFO:teuthology.orchestra.run.smithi028.stderr:reweighted osd.1 to 1 (10000) 2024-04-26T03:40:02.937 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-04-26T03:40:03.926 INFO:teuthology.orchestra.run.smithi028.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-04-26T03:40:03.943 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-04-26T03:40:04.932 INFO:teuthology.orchestra.run.smithi028.stderr:reweighted osd.5 to 1 (10000) 2024-04-26T03:40:04.948 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-04-26T03:40:05.937 INFO:teuthology.orchestra.run.smithi028.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-04-26T03:40:05.954 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 9 1 2024-04-26T03:40:06.942 INFO:teuthology.orchestra.run.smithi028.stderr:reweighted osd.9 to 1 (10000) 2024-04-26T03:40:06.959 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 9 1 2024-04-26T03:40:07.947 INFO:teuthology.orchestra.run.smithi028.stderr:set osd.9 primary-affinity to 1 (8655362) 2024-04-26T03:40:07.964 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 13 1 2024-04-26T03:40:08.954 INFO:teuthology.orchestra.run.smithi028.stderr:reweighted osd.13 to 1 (10000) 2024-04-26T03:40:08.971 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 13 1 2024-04-26T03:40:09.959 INFO:teuthology.orchestra.run.smithi028.stderr:set osd.13 primary-affinity to 1 (8655362) 2024-04-26T03:40:09.976 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-04-26T03:40:10.965 INFO:teuthology.orchestra.run.smithi028.stderr:reweighted osd.2 to 1 (10000) 2024-04-26T03:40:10.983 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-04-26T03:40:11.973 INFO:teuthology.orchestra.run.smithi028.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-04-26T03:40:11.989 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-04-26T03:40:12.980 INFO:teuthology.orchestra.run.smithi028.stderr:reweighted osd.6 to 1 (10000) 2024-04-26T03:40:12.997 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-04-26T03:40:13.988 INFO:teuthology.orchestra.run.smithi028.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-04-26T03:40:14.004 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 10 1 2024-04-26T03:40:14.993 INFO:teuthology.orchestra.run.smithi028.stderr:reweighted osd.10 to 1 (10000) 2024-04-26T03:40:15.011 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 10 1 2024-04-26T03:40:15.999 INFO:teuthology.orchestra.run.smithi028.stderr:set osd.10 primary-affinity to 1 (8655362) 2024-04-26T03:40:16.016 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 14 1 2024-04-26T03:40:17.006 INFO:teuthology.orchestra.run.smithi028.stderr:reweighted osd.14 to 1 (10000) 2024-04-26T03:40:17.023 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 14 1 2024-04-26T03:40:18.009 INFO:teuthology.orchestra.run.smithi028.stderr:set osd.14 primary-affinity to 1 (8655362) 2024-04-26T03:40:18.025 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-04-26T03:40:19.014 INFO:teuthology.orchestra.run.smithi028.stderr:reweighted osd.3 to 1 (10000) 2024-04-26T03:40:19.030 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-04-26T03:40:20.020 INFO:teuthology.orchestra.run.smithi028.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-04-26T03:40:20.038 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-04-26T03:40:21.027 INFO:teuthology.orchestra.run.smithi028.stderr:reweighted osd.7 to 1 (10000) 2024-04-26T03:40:21.044 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-04-26T03:40:22.033 INFO:teuthology.orchestra.run.smithi028.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-04-26T03:40:22.050 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 11 1 2024-04-26T03:40:23.038 INFO:teuthology.orchestra.run.smithi028.stderr:reweighted osd.11 to 1 (10000) 2024-04-26T03:40:23.055 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 11 1 2024-04-26T03:40:24.043 INFO:teuthology.orchestra.run.smithi028.stderr:set osd.11 primary-affinity to 1 (8655362) 2024-04-26T03:40:24.059 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 15 1 2024-04-26T03:40:25.049 INFO:teuthology.orchestra.run.smithi028.stderr:reweighted osd.15 to 1 (10000) 2024-04-26T03:40:25.065 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 15 1 2024-04-26T03:40:26.054 INFO:teuthology.orchestra.run.smithi028.stderr:set osd.15 primary-affinity to 1 (8655362) 2024-04-26T03:40:26.071 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-04-26T03:40:26.072 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-04-26T03:40:26.072 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-04-26T03:40:26.072 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-04-26T03:40:26.072 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-26T03:40:26.072 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T03:40:26.377 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:40:26.377 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":129,"fsid":"c5a13c99-5c9a-4b8b-a893-4f6f4211677a","created":"2024-04-26T03:38:19.896648+0000","modified":"2024-04-26T03:40:26.051686+0000","last_up_change":"2024-04-26T03:38:33.308561+0000","last_in_change":"2024-04-26T03:38:24.150336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T03:38:32.306696+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-26T03:38:34.592633+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.75,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"31b0e7a9-455d-4874-9c60-26efac8d0b1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2377656090},{"type":"v1","addr":"172.21.15.28:6801","nonce":2377656090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2377656090},{"type":"v1","addr":"172.21.15.28:6803","nonce":2377656090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2377656090},{"type":"v1","addr":"172.21.15.28:6807","nonce":2377656090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2377656090},{"type":"v1","addr":"172.21.15.28:6805","nonce":2377656090}]},"public_addr":"172.21.15.28:6801/2377656090","cluster_addr":"172.21.15.28:6803/2377656090","heartbeat_back_addr":"172.21.15.28:6807/2377656090","heartbeat_front_addr":"172.21.15.28:6805/2377656090","state":["exists","up"]},{"osd":1,"uuid":"52819ace-a070-4822-b33e-3c54466e61ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":1678150206},{"type":"v1","addr":"172.21.15.42:6801","nonce":1678150206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1678150206},{"type":"v1","addr":"172.21.15.42:6803","nonce":1678150206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1678150206},{"type":"v1","addr":"172.21.15.42:6807","nonce":1678150206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1678150206},{"type":"v1","addr":"172.21.15.42:6805","nonce":1678150206}]},"public_addr":"172.21.15.42:6801/1678150206","cluster_addr":"172.21.15.42:6803/1678150206","heartbeat_back_addr":"172.21.15.42:6807/1678150206","heartbeat_front_addr":"172.21.15.42:6805/1678150206","state":["exists","up"]},{"osd":2,"uuid":"858f8ac1-2edc-43d5-914c-c1914bdc0c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":714535415},{"type":"v1","addr":"172.21.15.66:6801","nonce":714535415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":714535415},{"type":"v1","addr":"172.21.15.66:6803","nonce":714535415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":714535415},{"type":"v1","addr":"172.21.15.66:6807","nonce":714535415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":714535415},{"type":"v1","addr":"172.21.15.66:6805","nonce":714535415}]},"public_addr":"172.21.15.66:6801/714535415","cluster_addr":"172.21.15.66:6803/714535415","heartbeat_back_addr":"172.21.15.66:6807/714535415","heartbeat_front_addr":"172.21.15.66:6805/714535415","state":["exists","up"]},{"osd":3,"uuid":"531a2e30-3351-4f84-804b-5fb5050b84ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2377983625},{"type":"v1","addr":"172.21.15.92:6811","nonce":2377983625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2377983625},{"type":"v1","addr":"172.21.15.92:6813","nonce":2377983625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2377983625},{"type":"v1","addr":"172.21.15.92:6817","nonce":2377983625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2377983625},{"type":"v1","addr":"172.21.15.92:6815","nonce":2377983625}]},"public_addr":"172.21.15.92:6811/2377983625","cluster_addr":"172.21.15.92:6813/2377983625","heartbeat_back_addr":"172.21.15.92:6817/2377983625","heartbeat_front_addr":"172.21.15.92:6815/2377983625","state":["exists","up"]},{"osd":4,"uuid":"6d38ffda-40eb-4fd7-9300-2a09f2673e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":183752455},{"type":"v1","addr":"172.21.15.28:6809","nonce":183752455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":183752455},{"type":"v1","addr":"172.21.15.28:6811","nonce":183752455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":183752455},{"type":"v1","addr":"172.21.15.28:6815","nonce":183752455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":183752455},{"type":"v1","addr":"172.21.15.28:6813","nonce":183752455}]},"public_addr":"172.21.15.28:6809/183752455","cluster_addr":"172.21.15.28:6811/183752455","heartbeat_back_addr":"172.21.15.28:6815/183752455","heartbeat_front_addr":"172.21.15.28:6813/183752455","state":["exists","up"]},{"osd":5,"uuid":"361aa40d-e53b-42af-afd4-81fbcac28725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2012261939},{"type":"v1","addr":"172.21.15.42:6809","nonce":2012261939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2012261939},{"type":"v1","addr":"172.21.15.42:6811","nonce":2012261939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2012261939},{"type":"v1","addr":"172.21.15.42:6815","nonce":2012261939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2012261939},{"type":"v1","addr":"172.21.15.42:6813","nonce":2012261939}]},"public_addr":"172.21.15.42:6809/2012261939","cluster_addr":"172.21.15.42:6811/2012261939","heartbeat_back_addr":"172.21.15.42:6815/2012261939","heartbeat_front_addr":"172.21.15.42:6813/2012261939","state":["exists","up"]},{"osd":6,"uuid":"f610ad3e-1fd8-42bd-af4b-0d79c3bb9cf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":3533460882},{"type":"v1","addr":"172.21.15.66:6809","nonce":3533460882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":3533460882},{"type":"v1","addr":"172.21.15.66:6811","nonce":3533460882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":3533460882},{"type":"v1","addr":"172.21.15.66:6815","nonce":3533460882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":3533460882},{"type":"v1","addr":"172.21.15.66:6813","nonce":3533460882}]},"public_addr":"172.21.15.66:6809/3533460882","cluster_addr":"172.21.15.66:6811/3533460882","heartbeat_back_addr":"172.21.15.66:6815/3533460882","heartbeat_front_addr":"172.21.15.66:6813/3533460882","state":["exists","up"]},{"osd":7,"uuid":"a4bae6fb-e087-4563-a129-6938e18eed8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":1760163933},{"type":"v1","addr":"172.21.15.92:6803","nonce":1760163933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":1760163933},{"type":"v1","addr":"172.21.15.92:6805","nonce":1760163933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1760163933},{"type":"v1","addr":"172.21.15.92:6809","nonce":1760163933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":1760163933},{"type":"v1","addr":"172.21.15.92:6807","nonce":1760163933}]},"public_addr":"172.21.15.92:6803/1760163933","cluster_addr":"172.21.15.92:6805/1760163933","heartbeat_back_addr":"172.21.15.92:6809/1760163933","heartbeat_front_addr":"172.21.15.92:6807/1760163933","state":["exists","up"]},{"osd":8,"uuid":"479f3885-452e-4231-bdc2-b79668384ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":4127602407},{"type":"v1","addr":"172.21.15.28:6817","nonce":4127602407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":4127602407},{"type":"v1","addr":"172.21.15.28:6819","nonce":4127602407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":4127602407},{"type":"v1","addr":"172.21.15.28:6823","nonce":4127602407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":4127602407},{"type":"v1","addr":"172.21.15.28:6821","nonce":4127602407}]},"public_addr":"172.21.15.28:6817/4127602407","cluster_addr":"172.21.15.28:6819/4127602407","heartbeat_back_addr":"172.21.15.28:6823/4127602407","heartbeat_front_addr":"172.21.15.28:6821/4127602407","state":["exists","up"]},{"osd":9,"uuid":"9a603a47-7c5a-47d4-a298-56b3ac991892","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":4168729845},{"type":"v1","addr":"172.21.15.42:6818","nonce":4168729845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":4168729845},{"type":"v1","addr":"172.21.15.42:6822","nonce":4168729845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":4168729845},{"type":"v1","addr":"172.21.15.42:6830","nonce":4168729845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":4168729845},{"type":"v1","addr":"172.21.15.42:6826","nonce":4168729845}]},"public_addr":"172.21.15.42:6818/4168729845","cluster_addr":"172.21.15.42:6822/4168729845","heartbeat_back_addr":"172.21.15.42:6830/4168729845","heartbeat_front_addr":"172.21.15.42:6826/4168729845","state":["exists","up"]},{"osd":10,"uuid":"1cd42945-6ee1-49f0-b17b-4093dd36b4ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4180572718},{"type":"v1","addr":"172.21.15.66:6817","nonce":4180572718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":4180572718},{"type":"v1","addr":"172.21.15.66:6819","nonce":4180572718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":4180572718},{"type":"v1","addr":"172.21.15.66:6823","nonce":4180572718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":4180572718},{"type":"v1","addr":"172.21.15.66:6821","nonce":4180572718}]},"public_addr":"172.21.15.66:6817/4180572718","cluster_addr":"172.21.15.66:6819/4180572718","heartbeat_back_addr":"172.21.15.66:6823/4180572718","heartbeat_front_addr":"172.21.15.66:6821/4180572718","state":["exists","up"]},{"osd":11,"uuid":"b41e8100-1d4f-4b7a-991a-7a27bc29feec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":295713815},{"type":"v1","addr":"172.21.15.92:6819","nonce":295713815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":295713815},{"type":"v1","addr":"172.21.15.92:6821","nonce":295713815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":295713815},{"type":"v1","addr":"172.21.15.92:6825","nonce":295713815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":295713815},{"type":"v1","addr":"172.21.15.92:6823","nonce":295713815}]},"public_addr":"172.21.15.92:6819/295713815","cluster_addr":"172.21.15.92:6821/295713815","heartbeat_back_addr":"172.21.15.92:6825/295713815","heartbeat_front_addr":"172.21.15.92:6823/295713815","state":["exists","up"]},{"osd":12,"uuid":"458cf150-32e1-449e-b533-ac6ab5226f85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":2081141953},{"type":"v1","addr":"172.21.15.28:6825","nonce":2081141953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2081141953},{"type":"v1","addr":"172.21.15.28:6827","nonce":2081141953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2081141953},{"type":"v1","addr":"172.21.15.28:6831","nonce":2081141953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":2081141953},{"type":"v1","addr":"172.21.15.28:6829","nonce":2081141953}]},"public_addr":"172.21.15.28:6825/2081141953","cluster_addr":"172.21.15.28:6827/2081141953","heartbeat_back_addr":"172.21.15.28:6831/2081141953","heartbeat_front_addr":"172.21.15.28:6829/2081141953","state":["exists","up"]},{"osd":13,"uuid":"f008462e-a735-4845-9ecf-dd7d4a6941af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6817","nonce":301360893},{"type":"v1","addr":"172.21.15.42:6819","nonce":301360893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6821","nonce":301360893},{"type":"v1","addr":"172.21.15.42:6823","nonce":301360893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6829","nonce":301360893},{"type":"v1","addr":"172.21.15.42:6831","nonce":301360893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6825","nonce":301360893},{"type":"v1","addr":"172.21.15.42:6827","nonce":301360893}]},"public_addr":"172.21.15.42:6819/301360893","cluster_addr":"172.21.15.42:6823/301360893","heartbeat_back_addr":"172.21.15.42:6831/301360893","heartbeat_front_addr":"172.21.15.42:6827/301360893","state":["exists","up"]},{"osd":14,"uuid":"6f66ae97-eb8a-48eb-9bcc-20f65fbf690b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":1534905602},{"type":"v1","addr":"172.21.15.66:6825","nonce":1534905602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1534905602},{"type":"v1","addr":"172.21.15.66:6827","nonce":1534905602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1534905602},{"type":"v1","addr":"172.21.15.66:6831","nonce":1534905602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1534905602},{"type":"v1","addr":"172.21.15.66:6829","nonce":1534905602}]},"public_addr":"172.21.15.66:6825/1534905602","cluster_addr":"172.21.15.66:6827/1534905602","heartbeat_back_addr":"172.21.15.66:6831/1534905602","heartbeat_front_addr":"172.21.15.66:6829/1534905602","state":["exists","up"]},{"osd":15,"uuid":"b856bfc8-253b-4112-be5c-f67d61d01768","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3982634323},{"type":"v1","addr":"172.21.15.92:6827","nonce":3982634323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3982634323},{"type":"v1","addr":"172.21.15.92:6829","nonce":3982634323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3982634323},{"type":"v1","addr":"172.21.15.92:6833","nonce":3982634323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3982634323},{"type":"v1","addr":"172.21.15.92:6831","nonce":3982634323}]},"public_addr":"172.21.15.92:6827/3982634323","cluster_addr":"172.21.15.92:6829/3982634323","heartbeat_back_addr":"172.21.15.92:6833/3982634323","heartbeat_front_addr":"172.21.15.92:6831/3982634323","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:40.180887+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:34.060445+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:35.302133+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:34.066992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:52.275741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:33.955241+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:40.990020+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:53.221513+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:32.243723+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:49.481065+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:55.555809+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:35.430680+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:32.221433+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:34.790817+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:57.884579+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:36.409075+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.2","mappings":[{"from":3,"to":11}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"shecprofile":{"c":"2","crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","k":"4","m":"3","name":"shecprofile","plugin":"shec","technique":"multiple"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":25,"length":1},{"begin":28,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T03:40:26.390 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-26T03:40:26.390 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T03:40:26.690 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:40:26.691 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":129,"fsid":"c5a13c99-5c9a-4b8b-a893-4f6f4211677a","created":"2024-04-26T03:38:19.896648+0000","modified":"2024-04-26T03:40:26.051686+0000","last_up_change":"2024-04-26T03:38:33.308561+0000","last_in_change":"2024-04-26T03:38:24.150336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T03:38:32.306696+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-26T03:38:34.592633+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.75,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"31b0e7a9-455d-4874-9c60-26efac8d0b1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2377656090},{"type":"v1","addr":"172.21.15.28:6801","nonce":2377656090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2377656090},{"type":"v1","addr":"172.21.15.28:6803","nonce":2377656090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2377656090},{"type":"v1","addr":"172.21.15.28:6807","nonce":2377656090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2377656090},{"type":"v1","addr":"172.21.15.28:6805","nonce":2377656090}]},"public_addr":"172.21.15.28:6801/2377656090","cluster_addr":"172.21.15.28:6803/2377656090","heartbeat_back_addr":"172.21.15.28:6807/2377656090","heartbeat_front_addr":"172.21.15.28:6805/2377656090","state":["exists","up"]},{"osd":1,"uuid":"52819ace-a070-4822-b33e-3c54466e61ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":1678150206},{"type":"v1","addr":"172.21.15.42:6801","nonce":1678150206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1678150206},{"type":"v1","addr":"172.21.15.42:6803","nonce":1678150206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1678150206},{"type":"v1","addr":"172.21.15.42:6807","nonce":1678150206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1678150206},{"type":"v1","addr":"172.21.15.42:6805","nonce":1678150206}]},"public_addr":"172.21.15.42:6801/1678150206","cluster_addr":"172.21.15.42:6803/1678150206","heartbeat_back_addr":"172.21.15.42:6807/1678150206","heartbeat_front_addr":"172.21.15.42:6805/1678150206","state":["exists","up"]},{"osd":2,"uuid":"858f8ac1-2edc-43d5-914c-c1914bdc0c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":714535415},{"type":"v1","addr":"172.21.15.66:6801","nonce":714535415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":714535415},{"type":"v1","addr":"172.21.15.66:6803","nonce":714535415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":714535415},{"type":"v1","addr":"172.21.15.66:6807","nonce":714535415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":714535415},{"type":"v1","addr":"172.21.15.66:6805","nonce":714535415}]},"public_addr":"172.21.15.66:6801/714535415","cluster_addr":"172.21.15.66:6803/714535415","heartbeat_back_addr":"172.21.15.66:6807/714535415","heartbeat_front_addr":"172.21.15.66:6805/714535415","state":["exists","up"]},{"osd":3,"uuid":"531a2e30-3351-4f84-804b-5fb5050b84ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2377983625},{"type":"v1","addr":"172.21.15.92:6811","nonce":2377983625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2377983625},{"type":"v1","addr":"172.21.15.92:6813","nonce":2377983625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2377983625},{"type":"v1","addr":"172.21.15.92:6817","nonce":2377983625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2377983625},{"type":"v1","addr":"172.21.15.92:6815","nonce":2377983625}]},"public_addr":"172.21.15.92:6811/2377983625","cluster_addr":"172.21.15.92:6813/2377983625","heartbeat_back_addr":"172.21.15.92:6817/2377983625","heartbeat_front_addr":"172.21.15.92:6815/2377983625","state":["exists","up"]},{"osd":4,"uuid":"6d38ffda-40eb-4fd7-9300-2a09f2673e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":183752455},{"type":"v1","addr":"172.21.15.28:6809","nonce":183752455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":183752455},{"type":"v1","addr":"172.21.15.28:6811","nonce":183752455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":183752455},{"type":"v1","addr":"172.21.15.28:6815","nonce":183752455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":183752455},{"type":"v1","addr":"172.21.15.28:6813","nonce":183752455}]},"public_addr":"172.21.15.28:6809/183752455","cluster_addr":"172.21.15.28:6811/183752455","heartbeat_back_addr":"172.21.15.28:6815/183752455","heartbeat_front_addr":"172.21.15.28:6813/183752455","state":["exists","up"]},{"osd":5,"uuid":"361aa40d-e53b-42af-afd4-81fbcac28725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2012261939},{"type":"v1","addr":"172.21.15.42:6809","nonce":2012261939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2012261939},{"type":"v1","addr":"172.21.15.42:6811","nonce":2012261939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2012261939},{"type":"v1","addr":"172.21.15.42:6815","nonce":2012261939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2012261939},{"type":"v1","addr":"172.21.15.42:6813","nonce":2012261939}]},"public_addr":"172.21.15.42:6809/2012261939","cluster_addr":"172.21.15.42:6811/2012261939","heartbeat_back_addr":"172.21.15.42:6815/2012261939","heartbeat_front_addr":"172.21.15.42:6813/2012261939","state":["exists","up"]},{"osd":6,"uuid":"f610ad3e-1fd8-42bd-af4b-0d79c3bb9cf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":3533460882},{"type":"v1","addr":"172.21.15.66:6809","nonce":3533460882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":3533460882},{"type":"v1","addr":"172.21.15.66:6811","nonce":3533460882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":3533460882},{"type":"v1","addr":"172.21.15.66:6815","nonce":3533460882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":3533460882},{"type":"v1","addr":"172.21.15.66:6813","nonce":3533460882}]},"public_addr":"172.21.15.66:6809/3533460882","cluster_addr":"172.21.15.66:6811/3533460882","heartbeat_back_addr":"172.21.15.66:6815/3533460882","heartbeat_front_addr":"172.21.15.66:6813/3533460882","state":["exists","up"]},{"osd":7,"uuid":"a4bae6fb-e087-4563-a129-6938e18eed8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":1760163933},{"type":"v1","addr":"172.21.15.92:6803","nonce":1760163933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":1760163933},{"type":"v1","addr":"172.21.15.92:6805","nonce":1760163933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1760163933},{"type":"v1","addr":"172.21.15.92:6809","nonce":1760163933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":1760163933},{"type":"v1","addr":"172.21.15.92:6807","nonce":1760163933}]},"public_addr":"172.21.15.92:6803/1760163933","cluster_addr":"172.21.15.92:6805/1760163933","heartbeat_back_addr":"172.21.15.92:6809/1760163933","heartbeat_front_addr":"172.21.15.92:6807/1760163933","state":["exists","up"]},{"osd":8,"uuid":"479f3885-452e-4231-bdc2-b79668384ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":4127602407},{"type":"v1","addr":"172.21.15.28:6817","nonce":4127602407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":4127602407},{"type":"v1","addr":"172.21.15.28:6819","nonce":4127602407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":4127602407},{"type":"v1","addr":"172.21.15.28:6823","nonce":4127602407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":4127602407},{"type":"v1","addr":"172.21.15.28:6821","nonce":4127602407}]},"public_addr":"172.21.15.28:6817/4127602407","cluster_addr":"172.21.15.28:6819/4127602407","heartbeat_back_addr":"172.21.15.28:6823/4127602407","heartbeat_front_addr":"172.21.15.28:6821/4127602407","state":["exists","up"]},{"osd":9,"uuid":"9a603a47-7c5a-47d4-a298-56b3ac991892","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":4168729845},{"type":"v1","addr":"172.21.15.42:6818","nonce":4168729845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":4168729845},{"type":"v1","addr":"172.21.15.42:6822","nonce":4168729845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":4168729845},{"type":"v1","addr":"172.21.15.42:6830","nonce":4168729845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":4168729845},{"type":"v1","addr":"172.21.15.42:6826","nonce":4168729845}]},"public_addr":"172.21.15.42:6818/4168729845","cluster_addr":"172.21.15.42:6822/4168729845","heartbeat_back_addr":"172.21.15.42:6830/4168729845","heartbeat_front_addr":"172.21.15.42:6826/4168729845","state":["exists","up"]},{"osd":10,"uuid":"1cd42945-6ee1-49f0-b17b-4093dd36b4ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4180572718},{"type":"v1","addr":"172.21.15.66:6817","nonce":4180572718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":4180572718},{"type":"v1","addr":"172.21.15.66:6819","nonce":4180572718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":4180572718},{"type":"v1","addr":"172.21.15.66:6823","nonce":4180572718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":4180572718},{"type":"v1","addr":"172.21.15.66:6821","nonce":4180572718}]},"public_addr":"172.21.15.66:6817/4180572718","cluster_addr":"172.21.15.66:6819/4180572718","heartbeat_back_addr":"172.21.15.66:6823/4180572718","heartbeat_front_addr":"172.21.15.66:6821/4180572718","state":["exists","up"]},{"osd":11,"uuid":"b41e8100-1d4f-4b7a-991a-7a27bc29feec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":295713815},{"type":"v1","addr":"172.21.15.92:6819","nonce":295713815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":295713815},{"type":"v1","addr":"172.21.15.92:6821","nonce":295713815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":295713815},{"type":"v1","addr":"172.21.15.92:6825","nonce":295713815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":295713815},{"type":"v1","addr":"172.21.15.92:6823","nonce":295713815}]},"public_addr":"172.21.15.92:6819/295713815","cluster_addr":"172.21.15.92:6821/295713815","heartbeat_back_addr":"172.21.15.92:6825/295713815","heartbeat_front_addr":"172.21.15.92:6823/295713815","state":["exists","up"]},{"osd":12,"uuid":"458cf150-32e1-449e-b533-ac6ab5226f85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":2081141953},{"type":"v1","addr":"172.21.15.28:6825","nonce":2081141953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2081141953},{"type":"v1","addr":"172.21.15.28:6827","nonce":2081141953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2081141953},{"type":"v1","addr":"172.21.15.28:6831","nonce":2081141953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":2081141953},{"type":"v1","addr":"172.21.15.28:6829","nonce":2081141953}]},"public_addr":"172.21.15.28:6825/2081141953","cluster_addr":"172.21.15.28:6827/2081141953","heartbeat_back_addr":"172.21.15.28:6831/2081141953","heartbeat_front_addr":"172.21.15.28:6829/2081141953","state":["exists","up"]},{"osd":13,"uuid":"f008462e-a735-4845-9ecf-dd7d4a6941af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6817","nonce":301360893},{"type":"v1","addr":"172.21.15.42:6819","nonce":301360893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6821","nonce":301360893},{"type":"v1","addr":"172.21.15.42:6823","nonce":301360893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6829","nonce":301360893},{"type":"v1","addr":"172.21.15.42:6831","nonce":301360893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6825","nonce":301360893},{"type":"v1","addr":"172.21.15.42:6827","nonce":301360893}]},"public_addr":"172.21.15.42:6819/301360893","cluster_addr":"172.21.15.42:6823/301360893","heartbeat_back_addr":"172.21.15.42:6831/301360893","heartbeat_front_addr":"172.21.15.42:6827/301360893","state":["exists","up"]},{"osd":14,"uuid":"6f66ae97-eb8a-48eb-9bcc-20f65fbf690b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":1534905602},{"type":"v1","addr":"172.21.15.66:6825","nonce":1534905602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1534905602},{"type":"v1","addr":"172.21.15.66:6827","nonce":1534905602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1534905602},{"type":"v1","addr":"172.21.15.66:6831","nonce":1534905602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1534905602},{"type":"v1","addr":"172.21.15.66:6829","nonce":1534905602}]},"public_addr":"172.21.15.66:6825/1534905602","cluster_addr":"172.21.15.66:6827/1534905602","heartbeat_back_addr":"172.21.15.66:6831/1534905602","heartbeat_front_addr":"172.21.15.66:6829/1534905602","state":["exists","up"]},{"osd":15,"uuid":"b856bfc8-253b-4112-be5c-f67d61d01768","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3982634323},{"type":"v1","addr":"172.21.15.92:6827","nonce":3982634323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3982634323},{"type":"v1","addr":"172.21.15.92:6829","nonce":3982634323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3982634323},{"type":"v1","addr":"172.21.15.92:6833","nonce":3982634323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3982634323},{"type":"v1","addr":"172.21.15.92:6831","nonce":3982634323}]},"public_addr":"172.21.15.92:6827/3982634323","cluster_addr":"172.21.15.92:6829/3982634323","heartbeat_back_addr":"172.21.15.92:6833/3982634323","heartbeat_front_addr":"172.21.15.92:6831/3982634323","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:40.180887+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:34.060445+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:35.302133+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:34.066992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:52.275741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:33.955241+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:40.990020+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:53.221513+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:32.243723+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:49.481065+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:55.555809+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:35.430680+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:32.221433+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:34.790817+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:57.884579+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:36.409075+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.2","mappings":[{"from":3,"to":11}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"shecprofile":{"c":"2","crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","k":"4","m":"3","name":"shecprofile","plugin":"shec","technique":"multiple"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":25,"length":1},{"begin":28,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T03:40:26.704 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-26T03:40:26.704 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-26T03:40:26.705 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-26T03:40:26.705 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-26T03:40:26.706 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-26T03:40:26.706 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-26T03:40:26.706 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-26T03:40:26.707 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-26T03:40:26.707 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 flush_pg_stats 2024-04-26T03:40:26.707 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 flush_pg_stats 2024-04-26T03:40:26.708 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 flush_pg_stats 2024-04-26T03:40:26.708 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 flush_pg_stats 2024-04-26T03:40:26.709 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.12 flush_pg_stats 2024-04-26T03:40:26.709 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.13 flush_pg_stats 2024-04-26T03:40:26.710 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.14 flush_pg_stats 2024-04-26T03:40:26.710 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.15 flush_pg_stats 2024-04-26T03:40:27.139 INFO:teuthology.orchestra.run.smithi028.stdout:94489280538 2024-04-26T03:40:27.140 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2024-04-26T03:40:27.164 INFO:teuthology.orchestra.run.smithi028.stdout:103079215132 2024-04-26T03:40:27.165 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-26T03:40:27.168 INFO:teuthology.orchestra.run.smithi028.stdout:98784247837 2024-04-26T03:40:27.168 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-26T03:40:27.176 INFO:teuthology.orchestra.run.smithi028.stdout:103079215132 2024-04-26T03:40:27.177 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-26T03:40:27.187 INFO:teuthology.orchestra.run.smithi028.stdout:94489280539 2024-04-26T03:40:27.187 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2024-04-26T03:40:27.196 INFO:teuthology.orchestra.run.smithi028.stdout:94489280538 2024-04-26T03:40:27.196 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-26T03:40:27.203 INFO:teuthology.orchestra.run.smithi028.stdout:90194313244 2024-04-26T03:40:27.203 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2024-04-26T03:40:27.220 INFO:teuthology.orchestra.run.smithi028.stdout:98784247838 2024-04-26T03:40:27.220 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-26T03:40:27.225 INFO:teuthology.orchestra.run.smithi028.stdout:94489280537 2024-04-26T03:40:27.226 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.13 2024-04-26T03:40:27.227 INFO:teuthology.orchestra.run.smithi028.stdout:94489280537 2024-04-26T03:40:27.227 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.15 2024-04-26T03:40:27.228 INFO:teuthology.orchestra.run.smithi028.stdout:98784247835 2024-04-26T03:40:27.228 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-26T03:40:27.230 INFO:teuthology.orchestra.run.smithi028.stdout:94489280537 2024-04-26T03:40:27.230 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.14 2024-04-26T03:40:27.245 INFO:teuthology.orchestra.run.smithi028.stdout:98784247835 2024-04-26T03:40:27.245 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-26T03:40:27.263 INFO:teuthology.orchestra.run.smithi028.stdout:94489280537 2024-04-26T03:40:27.263 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.12 2024-04-26T03:40:27.277 INFO:teuthology.orchestra.run.smithi028.stdout:94489280538 2024-04-26T03:40:27.277 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-26T03:40:27.287 INFO:teuthology.orchestra.run.smithi028.stdout:94489280537 2024-04-26T03:40:27.287 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2024-04-26T03:40:28.062 INFO:teuthology.orchestra.run.smithi028.stdout:103079215131 2024-04-26T03:40:28.090 INFO:tasks.ceph.ceph_manager.ceph:need seq 103079215132 got 103079215131 for osd.5 2024-04-26T03:40:28.146 INFO:teuthology.orchestra.run.smithi028.stdout:90194313242 2024-04-26T03:40:28.167 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313244 got 90194313242 for osd.11 2024-04-26T03:40:28.280 INFO:teuthology.orchestra.run.smithi028.stdout:94489280538 2024-04-26T03:40:28.310 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280539 got 94489280538 for osd.8 2024-04-26T03:40:28.383 INFO:teuthology.orchestra.run.smithi028.stdout:98784247834 2024-04-26T03:40:28.386 INFO:teuthology.orchestra.run.smithi028.stdout:94489280537 2024-04-26T03:40:28.391 INFO:teuthology.orchestra.run.smithi028.stdout:98784247836 2024-04-26T03:40:28.409 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247835 got 98784247834 for osd.2 2024-04-26T03:40:28.409 INFO:teuthology.orchestra.run.smithi028.stdout:94489280536 2024-04-26T03:40:28.417 INFO:teuthology.orchestra.run.smithi028.stdout:94489280536 2024-04-26T03:40:28.424 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280538 got 94489280537 for osd.10 2024-04-26T03:40:28.437 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280537 got 94489280536 for osd.13 2024-04-26T03:40:28.452 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280537 got 94489280536 for osd.15 2024-04-26T03:40:28.452 INFO:teuthology.orchestra.run.smithi028.stdout:103079215131 2024-04-26T03:40:28.455 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247837 got 98784247836 for osd.1 2024-04-26T03:40:28.455 INFO:teuthology.orchestra.run.smithi028.stdout:98784247837 2024-04-26T03:40:28.472 INFO:tasks.ceph.ceph_manager.ceph:need seq 103079215132 got 103079215131 for osd.3 2024-04-26T03:40:28.475 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247838 got 98784247837 for osd.4 2024-04-26T03:40:28.491 INFO:teuthology.orchestra.run.smithi028.stdout:98784247834 2024-04-26T03:40:28.500 INFO:teuthology.orchestra.run.smithi028.stdout:94489280537 2024-04-26T03:40:28.507 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247835 got 98784247834 for osd.0 2024-04-26T03:40:28.510 INFO:teuthology.orchestra.run.smithi028.stdout:94489280536 2024-04-26T03:40:28.511 INFO:teuthology.orchestra.run.smithi028.stdout:94489280537 2024-04-26T03:40:28.514 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280538 got 94489280537 for osd.6 2024-04-26T03:40:28.520 INFO:teuthology.orchestra.run.smithi028.stdout:94489280536 2024-04-26T03:40:28.524 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280537 got 94489280536 for osd.12 2024-04-26T03:40:28.531 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280538 got 94489280537 for osd.7 2024-04-26T03:40:28.533 INFO:teuthology.orchestra.run.smithi028.stdout:94489280536 2024-04-26T03:40:28.534 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280537 got 94489280536 for osd.14 2024-04-26T03:40:28.546 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280537 got 94489280536 for osd.9 2024-04-26T03:40:29.090 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-26T03:40:29.167 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2024-04-26T03:40:29.311 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2024-04-26T03:40:29.409 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-26T03:40:29.424 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2024-04-26T03:40:29.437 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.13 2024-04-26T03:40:29.453 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.15 2024-04-26T03:40:29.455 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-26T03:40:29.473 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-26T03:40:29.476 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-26T03:40:29.494 INFO:teuthology.orchestra.run.smithi028.stdout:103079215132 2024-04-26T03:40:29.508 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-26T03:40:29.515 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-26T03:40:29.518 INFO:tasks.ceph.ceph_manager.ceph:need seq 103079215132 got 103079215132 for osd.5 2024-04-26T03:40:29.519 DEBUG:teuthology.parallel:result is None 2024-04-26T03:40:29.525 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.12 2024-04-26T03:40:29.532 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-26T03:40:29.534 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.14 2024-04-26T03:40:29.546 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2024-04-26T03:40:29.676 INFO:teuthology.orchestra.run.smithi028.stdout:90194313244 2024-04-26T03:40:29.730 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313244 got 90194313244 for osd.11 2024-04-26T03:40:29.731 DEBUG:teuthology.parallel:result is None 2024-04-26T03:40:29.896 INFO:teuthology.orchestra.run.smithi028.stdout:94489280540 2024-04-26T03:40:29.938 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280539 got 94489280540 for osd.8 2024-04-26T03:40:29.939 DEBUG:teuthology.parallel:result is None 2024-04-26T03:40:30.221 INFO:teuthology.orchestra.run.smithi028.stdout:98784247835 2024-04-26T03:40:30.264 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247835 got 98784247835 for osd.2 2024-04-26T03:40:30.265 DEBUG:teuthology.parallel:result is None 2024-04-26T03:40:30.340 INFO:teuthology.orchestra.run.smithi028.stdout:94489280538 2024-04-26T03:40:30.359 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280537 got 94489280538 for osd.13 2024-04-26T03:40:30.360 DEBUG:teuthology.parallel:result is None 2024-04-26T03:40:30.380 INFO:teuthology.orchestra.run.smithi028.stdout:94489280539 2024-04-26T03:40:30.419 INFO:teuthology.orchestra.run.smithi028.stdout:103079215132 2024-04-26T03:40:30.419 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280538 got 94489280539 for osd.10 2024-04-26T03:40:30.420 DEBUG:teuthology.parallel:result is None 2024-04-26T03:40:30.443 INFO:teuthology.orchestra.run.smithi028.stdout:94489280538 2024-04-26T03:40:30.449 INFO:tasks.ceph.ceph_manager.ceph:need seq 103079215132 got 103079215132 for osd.3 2024-04-26T03:40:30.449 DEBUG:teuthology.parallel:result is None 2024-04-26T03:40:30.451 INFO:teuthology.orchestra.run.smithi028.stdout:98784247837 2024-04-26T03:40:30.469 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280537 got 94489280538 for osd.15 2024-04-26T03:40:30.469 DEBUG:teuthology.parallel:result is None 2024-04-26T03:40:30.485 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247837 got 98784247837 for osd.1 2024-04-26T03:40:30.486 DEBUG:teuthology.parallel:result is None 2024-04-26T03:40:30.494 INFO:teuthology.orchestra.run.smithi028.stdout:94489280538 2024-04-26T03:40:30.508 INFO:teuthology.orchestra.run.smithi028.stdout:98784247838 2024-04-26T03:40:30.514 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280537 got 94489280538 for osd.9 2024-04-26T03:40:30.514 DEBUG:teuthology.parallel:result is None 2024-04-26T03:40:30.524 INFO:teuthology.orchestra.run.smithi028.stdout:94489280538 2024-04-26T03:40:30.530 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247838 got 98784247838 for osd.4 2024-04-26T03:40:30.531 DEBUG:teuthology.parallel:result is None 2024-04-26T03:40:30.539 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280537 got 94489280538 for osd.14 2024-04-26T03:40:30.539 DEBUG:teuthology.parallel:result is None 2024-04-26T03:40:30.544 INFO:teuthology.orchestra.run.smithi028.stdout:94489280539 2024-04-26T03:40:30.557 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280538 got 94489280539 for osd.7 2024-04-26T03:40:30.557 DEBUG:teuthology.parallel:result is None 2024-04-26T03:40:30.559 INFO:teuthology.orchestra.run.smithi028.stdout:94489280538 2024-04-26T03:40:30.564 INFO:teuthology.orchestra.run.smithi028.stdout:98784247835 2024-04-26T03:40:30.573 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280537 got 94489280538 for osd.12 2024-04-26T03:40:30.574 DEBUG:teuthology.parallel:result is None 2024-04-26T03:40:30.582 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247835 got 98784247835 for osd.0 2024-04-26T03:40:30.583 DEBUG:teuthology.parallel:result is None 2024-04-26T03:40:30.595 INFO:teuthology.orchestra.run.smithi028.stdout:94489280539 2024-04-26T03:40:30.608 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280538 got 94489280539 for osd.6 2024-04-26T03:40:30.608 DEBUG:teuthology.parallel:result is None 2024-04-26T03:40:30.608 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-04-26T03:40:30.608 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T03:40:30.946 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:40:30.946 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-04-26T03:40:30.958 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":174,"stamp":"2024-04-26T03:40:30.073044+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":13,"kb":1509949440,"kb_used":585092,"kb_used_data":23660,"kb_used_omap":24,"kb_used_meta":561127,"kb_avail":1509364348,"statfs":{"total":1546188226560,"available":1545589092352,"internally_reserved":0,"allocated":24227840,"data_stored":19139989,"data_compressed":3452762,"data_compressed_allocated":6991872,"data_compressed_original":14876672,"omap_allocated":25522,"internal_metadata":574594126},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.029762"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":220,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:26.632577+0000","last_change":"2024-04-26T03:40:08.073279+0000","last_active":"2024-04-26T03:40:26.632577+0000","last_peered":"2024-04-26T03:40:26.632577+0000","last_clean":"2024-04-26T03:40:26.632577+0000","last_became_active":"2024-04-26T03:38:37.358910+0000","last_became_peered":"2024-04-26T03:38:37.358910+0000","last_unstale":"2024-04-26T03:40:26.632577+0000","last_undegraded":"2024-04-26T03:40:26.632577+0000","last_fullsized":"2024-04-26T03:40:26.632577+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:40:08.073132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:40:08.073132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:17.208416+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":217,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:26.055670+0000","last_change":"2024-04-26T03:39:45.393243+0000","last_active":"2024-04-26T03:40:26.055670+0000","last_peered":"2024-04-26T03:40:26.055670+0000","last_clean":"2024-04-26T03:40:26.055670+0000","last_became_active":"2024-04-26T03:38:36.484388+0000","last_became_peered":"2024-04-26T03:38:36.484388+0000","last_unstale":"2024-04-26T03:40:26.055670+0000","last_undegraded":"2024-04-26T03:40:26.055670+0000","last_fullsized":"2024-04-26T03:40:26.055670+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:39:45.393105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:39:45.393105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:50.701189+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,13],"acting":[8,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":217,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:26.632834+0000","last_change":"2024-04-26T03:40:07.106714+0000","last_active":"2024-04-26T03:40:26.632834+0000","last_peered":"2024-04-26T03:40:26.632834+0000","last_clean":"2024-04-26T03:40:26.632834+0000","last_became_active":"2024-04-26T03:38:38.187383+0000","last_became_peered":"2024-04-26T03:38:38.187383+0000","last_unstale":"2024-04-26T03:40:26.632834+0000","last_undegraded":"2024-04-26T03:40:26.632834+0000","last_fullsized":"2024-04-26T03:40:26.632834+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:40:07.106612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:40:07.106612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:16.998505+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4],"acting":[9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":218,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:26.632157+0000","last_change":"2024-04-26T03:39:46.090838+0000","last_active":"2024-04-26T03:40:26.632157+0000","last_peered":"2024-04-26T03:40:26.632157+0000","last_clean":"2024-04-26T03:40:26.632157+0000","last_became_active":"2024-04-26T03:38:37.255066+0000","last_became_peered":"2024-04-26T03:38:37.255066+0000","last_unstale":"2024-04-26T03:40:26.632157+0000","last_undegraded":"2024-04-26T03:40:26.632157+0000","last_fullsized":"2024-04-26T03:40:26.632157+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:39:46.090747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:39:46.090747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:48.933362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":207,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:27.004238+0000","last_change":"2024-04-26T03:39:45.538938+0000","last_active":"2024-04-26T03:40:27.004238+0000","last_peered":"2024-04-26T03:40:27.004238+0000","last_clean":"2024-04-26T03:40:27.004238+0000","last_became_active":"2024-04-26T03:38:48.475155+0000","last_became_peered":"2024-04-26T03:38:48.475155+0000","last_unstale":"2024-04-26T03:40:27.004238+0000","last_undegraded":"2024-04-26T03:40:27.004238+0000","last_fullsized":"2024-04-26T03:40:27.004238+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:39:45.538842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:39:45.538842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:48.125744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,12],"acting":[11,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":217,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:26.631774+0000","last_change":"2024-04-26T03:40:01.434482+0000","last_active":"2024-04-26T03:40:26.631774+0000","last_peered":"2024-04-26T03:40:26.631774+0000","last_clean":"2024-04-26T03:40:26.631774+0000","last_became_active":"2024-04-26T03:38:37.255181+0000","last_became_peered":"2024-04-26T03:38:37.255181+0000","last_unstale":"2024-04-26T03:40:26.631774+0000","last_undegraded":"2024-04-26T03:40:26.631774+0000","last_fullsized":"2024-04-26T03:40:26.631774+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:40:01.434355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:40:01.434355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:19.401059+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0],"acting":[14,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":218,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:26.055830+0000","last_change":"2024-04-26T03:39:54.682639+0000","last_active":"2024-04-26T03:40:26.055830+0000","last_peered":"2024-04-26T03:40:26.055830+0000","last_clean":"2024-04-26T03:40:26.055830+0000","last_became_active":"2024-04-26T03:38:36.431121+0000","last_became_peered":"2024-04-26T03:38:36.431121+0000","last_unstale":"2024-04-26T03:40:26.055830+0000","last_undegraded":"2024-04-26T03:40:26.055830+0000","last_fullsized":"2024-04-26T03:40:26.055830+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:39:54.682516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:39:54.682516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:12.279328+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":217,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:26.631789+0000","last_change":"2024-04-26T03:39:48.469712+0000","last_active":"2024-04-26T03:40:26.631789+0000","last_peered":"2024-04-26T03:40:26.631789+0000","last_clean":"2024-04-26T03:40:26.631789+0000","last_became_active":"2024-04-26T03:38:36.431449+0000","last_became_peered":"2024-04-26T03:38:36.431449+0000","last_unstale":"2024-04-26T03:40:26.631789+0000","last_undegraded":"2024-04-26T03:40:26.631789+0000","last_fullsized":"2024-04-26T03:40:26.631789+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:39:48.469598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:39:48.469598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:02.722236+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"27'157","reported_seq":418,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:26.234535+0000","last_change":"2024-04-26T03:39:42.110436+0000","last_active":"2024-04-26T03:40:26.234535+0000","last_peered":"2024-04-26T03:40:26.234535+0000","last_clean":"2024-04-26T03:40:26.234535+0000","last_became_active":"2024-04-26T03:38:34.389099+0000","last_became_peered":"2024-04-26T03:38:34.389099+0000","last_unstale":"2024-04-26T03:40:26.234535+0000","last_undegraded":"2024-04-26T03:40:26.234535+0000","last_fullsized":"2024-04-26T03:40:26.234535+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'157","last_scrub_stamp":"2024-04-26T03:39:42.110300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:32.306810+0000","last_clean_scrub_stamp":"2024-04-26T03:39:42.110300+0000","objects_scrubbed":2,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:11.925418+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":13852,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":24,"seq":103079215133,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36876,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334964,"statfs":{"total":96636764160,"available":96599003136,"internally_reserved":0,"allocated":1564672,"data_stored":1662071,"data_compressed":222496,"data_compressed_allocated":466944,"data_compressed_original":1380352,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,4,6,7,8,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215133,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32660,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,4,5,6,7,9,10,11,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":23,"seq":98784247839,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36824,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335016,"statfs":{"total":96636764160,"available":96599056384,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,7,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280539,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32716,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339124,"statfs":{"total":96636764160,"available":96603262976,"internally_reserved":0,"allocated":1564672,"data_stored":1662071,"data_compressed":222496,"data_compressed_allocated":466944,"data_compressed_original":1380352,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,4,5,7,8,9,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280539,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32656,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,3,4,5,6,8,9,10,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":22,"seq":94489280540,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,4,5,6,7,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280538,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32652,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339188,"statfs":{"total":96636764160,"available":96603328512,"internally_reserved":0,"allocated":1499136,"data_stored":1123512,"data_compressed":213528,"data_compressed_allocated":430080,"data_compressed_original":860160,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,2,4,7,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280539,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[1,2,3,4,7,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313244,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50704,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321136,"statfs":{"total":96636764160,"available":96584843264,"internally_reserved":0,"allocated":1499136,"data_stored":1123512,"data_compressed":213528,"data_compressed_allocated":430080,"data_compressed_original":860160,"omap_allocated":1596,"internal_metadata":50395588},"hb_peers":[4,5,6,7,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":22,"seq":94489280538,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28500,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343340,"statfs":{"total":96636764160,"available":96607580160,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,7,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280538,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1499136,"data_stored":1123512,"data_compressed":213528,"data_compressed_allocated":430080,"data_compressed_original":860160,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[4,6,7,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247836,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54856,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316984,"statfs":{"total":96636764160,"available":96580591616,"internally_reserved":0,"allocated":1499136,"data_stored":1123512,"data_compressed":213528,"data_compressed_allocated":430080,"data_compressed_original":860160,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,6,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280538,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1499136,"data_stored":1123512,"data_compressed":213528,"data_compressed_allocated":430080,"data_compressed_original":860160,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,3,5,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":23,"seq":98784247838,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":54868,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316972,"statfs":{"total":96636764160,"available":96580579328,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,4,6,7,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280538,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28496,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343344,"statfs":{"total":96636764160,"available":96607584256,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,4,5,7,8,9,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247836,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32660,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T03:40:30.959 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T03:40:31.253 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:40:31.254 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-04-26T03:40:31.265 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":174,"stamp":"2024-04-26T03:40:30.073044+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":13,"kb":1509949440,"kb_used":585092,"kb_used_data":23660,"kb_used_omap":24,"kb_used_meta":561127,"kb_avail":1509364348,"statfs":{"total":1546188226560,"available":1545589092352,"internally_reserved":0,"allocated":24227840,"data_stored":19139989,"data_compressed":3452762,"data_compressed_allocated":6991872,"data_compressed_original":14876672,"omap_allocated":25522,"internal_metadata":574594126},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.029762"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":220,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:26.632577+0000","last_change":"2024-04-26T03:40:08.073279+0000","last_active":"2024-04-26T03:40:26.632577+0000","last_peered":"2024-04-26T03:40:26.632577+0000","last_clean":"2024-04-26T03:40:26.632577+0000","last_became_active":"2024-04-26T03:38:37.358910+0000","last_became_peered":"2024-04-26T03:38:37.358910+0000","last_unstale":"2024-04-26T03:40:26.632577+0000","last_undegraded":"2024-04-26T03:40:26.632577+0000","last_fullsized":"2024-04-26T03:40:26.632577+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:40:08.073132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:40:08.073132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:17.208416+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":217,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:26.055670+0000","last_change":"2024-04-26T03:39:45.393243+0000","last_active":"2024-04-26T03:40:26.055670+0000","last_peered":"2024-04-26T03:40:26.055670+0000","last_clean":"2024-04-26T03:40:26.055670+0000","last_became_active":"2024-04-26T03:38:36.484388+0000","last_became_peered":"2024-04-26T03:38:36.484388+0000","last_unstale":"2024-04-26T03:40:26.055670+0000","last_undegraded":"2024-04-26T03:40:26.055670+0000","last_fullsized":"2024-04-26T03:40:26.055670+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:39:45.393105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:39:45.393105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:50.701189+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,13],"acting":[8,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":217,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:26.632834+0000","last_change":"2024-04-26T03:40:07.106714+0000","last_active":"2024-04-26T03:40:26.632834+0000","last_peered":"2024-04-26T03:40:26.632834+0000","last_clean":"2024-04-26T03:40:26.632834+0000","last_became_active":"2024-04-26T03:38:38.187383+0000","last_became_peered":"2024-04-26T03:38:38.187383+0000","last_unstale":"2024-04-26T03:40:26.632834+0000","last_undegraded":"2024-04-26T03:40:26.632834+0000","last_fullsized":"2024-04-26T03:40:26.632834+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:40:07.106612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:40:07.106612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:16.998505+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4],"acting":[9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":218,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:26.632157+0000","last_change":"2024-04-26T03:39:46.090838+0000","last_active":"2024-04-26T03:40:26.632157+0000","last_peered":"2024-04-26T03:40:26.632157+0000","last_clean":"2024-04-26T03:40:26.632157+0000","last_became_active":"2024-04-26T03:38:37.255066+0000","last_became_peered":"2024-04-26T03:38:37.255066+0000","last_unstale":"2024-04-26T03:40:26.632157+0000","last_undegraded":"2024-04-26T03:40:26.632157+0000","last_fullsized":"2024-04-26T03:40:26.632157+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:39:46.090747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:39:46.090747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:48.933362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":207,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:27.004238+0000","last_change":"2024-04-26T03:39:45.538938+0000","last_active":"2024-04-26T03:40:27.004238+0000","last_peered":"2024-04-26T03:40:27.004238+0000","last_clean":"2024-04-26T03:40:27.004238+0000","last_became_active":"2024-04-26T03:38:48.475155+0000","last_became_peered":"2024-04-26T03:38:48.475155+0000","last_unstale":"2024-04-26T03:40:27.004238+0000","last_undegraded":"2024-04-26T03:40:27.004238+0000","last_fullsized":"2024-04-26T03:40:27.004238+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:39:45.538842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:39:45.538842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:48.125744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,12],"acting":[11,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":217,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:26.631774+0000","last_change":"2024-04-26T03:40:01.434482+0000","last_active":"2024-04-26T03:40:26.631774+0000","last_peered":"2024-04-26T03:40:26.631774+0000","last_clean":"2024-04-26T03:40:26.631774+0000","last_became_active":"2024-04-26T03:38:37.255181+0000","last_became_peered":"2024-04-26T03:38:37.255181+0000","last_unstale":"2024-04-26T03:40:26.631774+0000","last_undegraded":"2024-04-26T03:40:26.631774+0000","last_fullsized":"2024-04-26T03:40:26.631774+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:40:01.434355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:40:01.434355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:19.401059+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0],"acting":[14,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":218,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:26.055830+0000","last_change":"2024-04-26T03:39:54.682639+0000","last_active":"2024-04-26T03:40:26.055830+0000","last_peered":"2024-04-26T03:40:26.055830+0000","last_clean":"2024-04-26T03:40:26.055830+0000","last_became_active":"2024-04-26T03:38:36.431121+0000","last_became_peered":"2024-04-26T03:38:36.431121+0000","last_unstale":"2024-04-26T03:40:26.055830+0000","last_undegraded":"2024-04-26T03:40:26.055830+0000","last_fullsized":"2024-04-26T03:40:26.055830+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:39:54.682516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:39:54.682516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:12.279328+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":217,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:26.631789+0000","last_change":"2024-04-26T03:39:48.469712+0000","last_active":"2024-04-26T03:40:26.631789+0000","last_peered":"2024-04-26T03:40:26.631789+0000","last_clean":"2024-04-26T03:40:26.631789+0000","last_became_active":"2024-04-26T03:38:36.431449+0000","last_became_peered":"2024-04-26T03:38:36.431449+0000","last_unstale":"2024-04-26T03:40:26.631789+0000","last_undegraded":"2024-04-26T03:40:26.631789+0000","last_fullsized":"2024-04-26T03:40:26.631789+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:39:48.469598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:39:48.469598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:02.722236+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"27'157","reported_seq":418,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:26.234535+0000","last_change":"2024-04-26T03:39:42.110436+0000","last_active":"2024-04-26T03:40:26.234535+0000","last_peered":"2024-04-26T03:40:26.234535+0000","last_clean":"2024-04-26T03:40:26.234535+0000","last_became_active":"2024-04-26T03:38:34.389099+0000","last_became_peered":"2024-04-26T03:38:34.389099+0000","last_unstale":"2024-04-26T03:40:26.234535+0000","last_undegraded":"2024-04-26T03:40:26.234535+0000","last_fullsized":"2024-04-26T03:40:26.234535+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'157","last_scrub_stamp":"2024-04-26T03:39:42.110300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:32.306810+0000","last_clean_scrub_stamp":"2024-04-26T03:39:42.110300+0000","objects_scrubbed":2,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:11.925418+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":13852,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":24,"seq":103079215133,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36876,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334964,"statfs":{"total":96636764160,"available":96599003136,"internally_reserved":0,"allocated":1564672,"data_stored":1662071,"data_compressed":222496,"data_compressed_allocated":466944,"data_compressed_original":1380352,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,4,6,7,8,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215133,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32660,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,4,5,6,7,9,10,11,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":23,"seq":98784247839,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36824,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335016,"statfs":{"total":96636764160,"available":96599056384,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,7,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280539,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32716,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339124,"statfs":{"total":96636764160,"available":96603262976,"internally_reserved":0,"allocated":1564672,"data_stored":1662071,"data_compressed":222496,"data_compressed_allocated":466944,"data_compressed_original":1380352,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,4,5,7,8,9,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280539,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32656,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,3,4,5,6,8,9,10,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":22,"seq":94489280540,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,4,5,6,7,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280538,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32652,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339188,"statfs":{"total":96636764160,"available":96603328512,"internally_reserved":0,"allocated":1499136,"data_stored":1123512,"data_compressed":213528,"data_compressed_allocated":430080,"data_compressed_original":860160,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,2,4,7,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280539,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[1,2,3,4,7,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313244,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50704,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321136,"statfs":{"total":96636764160,"available":96584843264,"internally_reserved":0,"allocated":1499136,"data_stored":1123512,"data_compressed":213528,"data_compressed_allocated":430080,"data_compressed_original":860160,"omap_allocated":1596,"internal_metadata":50395588},"hb_peers":[4,5,6,7,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":22,"seq":94489280538,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28500,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343340,"statfs":{"total":96636764160,"available":96607580160,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,7,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280538,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1499136,"data_stored":1123512,"data_compressed":213528,"data_compressed_allocated":430080,"data_compressed_original":860160,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[4,6,7,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247836,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54856,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316984,"statfs":{"total":96636764160,"available":96580591616,"internally_reserved":0,"allocated":1499136,"data_stored":1123512,"data_compressed":213528,"data_compressed_allocated":430080,"data_compressed_original":860160,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,6,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280538,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1499136,"data_stored":1123512,"data_compressed":213528,"data_compressed_allocated":430080,"data_compressed_original":860160,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,3,5,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":23,"seq":98784247838,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":54868,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316972,"statfs":{"total":96636764160,"available":96580579328,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,4,6,7,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280538,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28496,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343344,"statfs":{"total":96636764160,"available":96607584256,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,4,5,7,8,9,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247836,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32660,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T03:40:31.266 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-04-26T03:40:31.267 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-26T03:40:31.277 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-26T03:40:31.277 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T03:40:31.571 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:40:31.571 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-04-26T03:40:31.583 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":174,"stamp":"2024-04-26T03:40:30.073044+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":13,"kb":1509949440,"kb_used":585092,"kb_used_data":23660,"kb_used_omap":24,"kb_used_meta":561127,"kb_avail":1509364348,"statfs":{"total":1546188226560,"available":1545589092352,"internally_reserved":0,"allocated":24227840,"data_stored":19139989,"data_compressed":3452762,"data_compressed_allocated":6991872,"data_compressed_original":14876672,"omap_allocated":25522,"internal_metadata":574594126},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.029762"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":220,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:26.632577+0000","last_change":"2024-04-26T03:40:08.073279+0000","last_active":"2024-04-26T03:40:26.632577+0000","last_peered":"2024-04-26T03:40:26.632577+0000","last_clean":"2024-04-26T03:40:26.632577+0000","last_became_active":"2024-04-26T03:38:37.358910+0000","last_became_peered":"2024-04-26T03:38:37.358910+0000","last_unstale":"2024-04-26T03:40:26.632577+0000","last_undegraded":"2024-04-26T03:40:26.632577+0000","last_fullsized":"2024-04-26T03:40:26.632577+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:40:08.073132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:40:08.073132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:17.208416+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":217,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:26.055670+0000","last_change":"2024-04-26T03:39:45.393243+0000","last_active":"2024-04-26T03:40:26.055670+0000","last_peered":"2024-04-26T03:40:26.055670+0000","last_clean":"2024-04-26T03:40:26.055670+0000","last_became_active":"2024-04-26T03:38:36.484388+0000","last_became_peered":"2024-04-26T03:38:36.484388+0000","last_unstale":"2024-04-26T03:40:26.055670+0000","last_undegraded":"2024-04-26T03:40:26.055670+0000","last_fullsized":"2024-04-26T03:40:26.055670+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:39:45.393105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:39:45.393105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:50.701189+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,13],"acting":[8,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":217,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:26.632834+0000","last_change":"2024-04-26T03:40:07.106714+0000","last_active":"2024-04-26T03:40:26.632834+0000","last_peered":"2024-04-26T03:40:26.632834+0000","last_clean":"2024-04-26T03:40:26.632834+0000","last_became_active":"2024-04-26T03:38:38.187383+0000","last_became_peered":"2024-04-26T03:38:38.187383+0000","last_unstale":"2024-04-26T03:40:26.632834+0000","last_undegraded":"2024-04-26T03:40:26.632834+0000","last_fullsized":"2024-04-26T03:40:26.632834+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:40:07.106612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:40:07.106612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:16.998505+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4],"acting":[9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":218,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:26.632157+0000","last_change":"2024-04-26T03:39:46.090838+0000","last_active":"2024-04-26T03:40:26.632157+0000","last_peered":"2024-04-26T03:40:26.632157+0000","last_clean":"2024-04-26T03:40:26.632157+0000","last_became_active":"2024-04-26T03:38:37.255066+0000","last_became_peered":"2024-04-26T03:38:37.255066+0000","last_unstale":"2024-04-26T03:40:26.632157+0000","last_undegraded":"2024-04-26T03:40:26.632157+0000","last_fullsized":"2024-04-26T03:40:26.632157+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:39:46.090747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:39:46.090747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:48.933362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":207,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:27.004238+0000","last_change":"2024-04-26T03:39:45.538938+0000","last_active":"2024-04-26T03:40:27.004238+0000","last_peered":"2024-04-26T03:40:27.004238+0000","last_clean":"2024-04-26T03:40:27.004238+0000","last_became_active":"2024-04-26T03:38:48.475155+0000","last_became_peered":"2024-04-26T03:38:48.475155+0000","last_unstale":"2024-04-26T03:40:27.004238+0000","last_undegraded":"2024-04-26T03:40:27.004238+0000","last_fullsized":"2024-04-26T03:40:27.004238+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:39:45.538842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:39:45.538842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:48.125744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,12],"acting":[11,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":217,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:26.631774+0000","last_change":"2024-04-26T03:40:01.434482+0000","last_active":"2024-04-26T03:40:26.631774+0000","last_peered":"2024-04-26T03:40:26.631774+0000","last_clean":"2024-04-26T03:40:26.631774+0000","last_became_active":"2024-04-26T03:38:37.255181+0000","last_became_peered":"2024-04-26T03:38:37.255181+0000","last_unstale":"2024-04-26T03:40:26.631774+0000","last_undegraded":"2024-04-26T03:40:26.631774+0000","last_fullsized":"2024-04-26T03:40:26.631774+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:40:01.434355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:40:01.434355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:19.401059+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0],"acting":[14,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":218,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:26.055830+0000","last_change":"2024-04-26T03:39:54.682639+0000","last_active":"2024-04-26T03:40:26.055830+0000","last_peered":"2024-04-26T03:40:26.055830+0000","last_clean":"2024-04-26T03:40:26.055830+0000","last_became_active":"2024-04-26T03:38:36.431121+0000","last_became_peered":"2024-04-26T03:38:36.431121+0000","last_unstale":"2024-04-26T03:40:26.055830+0000","last_undegraded":"2024-04-26T03:40:26.055830+0000","last_fullsized":"2024-04-26T03:40:26.055830+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:39:54.682516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:39:54.682516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:12.279328+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":217,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:26.631789+0000","last_change":"2024-04-26T03:39:48.469712+0000","last_active":"2024-04-26T03:40:26.631789+0000","last_peered":"2024-04-26T03:40:26.631789+0000","last_clean":"2024-04-26T03:40:26.631789+0000","last_became_active":"2024-04-26T03:38:36.431449+0000","last_became_peered":"2024-04-26T03:38:36.431449+0000","last_unstale":"2024-04-26T03:40:26.631789+0000","last_undegraded":"2024-04-26T03:40:26.631789+0000","last_fullsized":"2024-04-26T03:40:26.631789+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:39:48.469598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:39:48.469598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:02.722236+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"27'157","reported_seq":418,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:26.234535+0000","last_change":"2024-04-26T03:39:42.110436+0000","last_active":"2024-04-26T03:40:26.234535+0000","last_peered":"2024-04-26T03:40:26.234535+0000","last_clean":"2024-04-26T03:40:26.234535+0000","last_became_active":"2024-04-26T03:38:34.389099+0000","last_became_peered":"2024-04-26T03:38:34.389099+0000","last_unstale":"2024-04-26T03:40:26.234535+0000","last_undegraded":"2024-04-26T03:40:26.234535+0000","last_fullsized":"2024-04-26T03:40:26.234535+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'157","last_scrub_stamp":"2024-04-26T03:39:42.110300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:32.306810+0000","last_clean_scrub_stamp":"2024-04-26T03:39:42.110300+0000","objects_scrubbed":2,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:11.925418+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":13852,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":24,"seq":103079215133,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36876,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334964,"statfs":{"total":96636764160,"available":96599003136,"internally_reserved":0,"allocated":1564672,"data_stored":1662071,"data_compressed":222496,"data_compressed_allocated":466944,"data_compressed_original":1380352,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,4,6,7,8,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215133,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32660,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,4,5,6,7,9,10,11,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":23,"seq":98784247839,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36824,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335016,"statfs":{"total":96636764160,"available":96599056384,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,7,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280539,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32716,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339124,"statfs":{"total":96636764160,"available":96603262976,"internally_reserved":0,"allocated":1564672,"data_stored":1662071,"data_compressed":222496,"data_compressed_allocated":466944,"data_compressed_original":1380352,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,4,5,7,8,9,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280539,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32656,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,3,4,5,6,8,9,10,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":22,"seq":94489280540,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,4,5,6,7,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280538,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32652,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339188,"statfs":{"total":96636764160,"available":96603328512,"internally_reserved":0,"allocated":1499136,"data_stored":1123512,"data_compressed":213528,"data_compressed_allocated":430080,"data_compressed_original":860160,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,2,4,7,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280539,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[1,2,3,4,7,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313244,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50704,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321136,"statfs":{"total":96636764160,"available":96584843264,"internally_reserved":0,"allocated":1499136,"data_stored":1123512,"data_compressed":213528,"data_compressed_allocated":430080,"data_compressed_original":860160,"omap_allocated":1596,"internal_metadata":50395588},"hb_peers":[4,5,6,7,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":22,"seq":94489280538,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28500,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343340,"statfs":{"total":96636764160,"available":96607580160,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,7,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280538,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1499136,"data_stored":1123512,"data_compressed":213528,"data_compressed_allocated":430080,"data_compressed_original":860160,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[4,6,7,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247836,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54856,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316984,"statfs":{"total":96636764160,"available":96580591616,"internally_reserved":0,"allocated":1499136,"data_stored":1123512,"data_compressed":213528,"data_compressed_allocated":430080,"data_compressed_original":860160,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,6,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280538,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1499136,"data_stored":1123512,"data_compressed":213528,"data_compressed_allocated":430080,"data_compressed_original":860160,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,3,5,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":23,"seq":98784247838,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":54868,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316972,"statfs":{"total":96636764160,"available":96580579328,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,4,6,7,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280538,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28496,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343344,"statfs":{"total":96636764160,"available":96607584256,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,4,5,7,8,9,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247836,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32660,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T03:40:31.585 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T03:40:31.879 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:40:31.880 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-04-26T03:40:31.892 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":174,"stamp":"2024-04-26T03:40:30.073044+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":13,"kb":1509949440,"kb_used":585092,"kb_used_data":23660,"kb_used_omap":24,"kb_used_meta":561127,"kb_avail":1509364348,"statfs":{"total":1546188226560,"available":1545589092352,"internally_reserved":0,"allocated":24227840,"data_stored":19139989,"data_compressed":3452762,"data_compressed_allocated":6991872,"data_compressed_original":14876672,"omap_allocated":25522,"internal_metadata":574594126},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.029762"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":220,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:26.632577+0000","last_change":"2024-04-26T03:40:08.073279+0000","last_active":"2024-04-26T03:40:26.632577+0000","last_peered":"2024-04-26T03:40:26.632577+0000","last_clean":"2024-04-26T03:40:26.632577+0000","last_became_active":"2024-04-26T03:38:37.358910+0000","last_became_peered":"2024-04-26T03:38:37.358910+0000","last_unstale":"2024-04-26T03:40:26.632577+0000","last_undegraded":"2024-04-26T03:40:26.632577+0000","last_fullsized":"2024-04-26T03:40:26.632577+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:40:08.073132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:40:08.073132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:17.208416+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":217,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:26.055670+0000","last_change":"2024-04-26T03:39:45.393243+0000","last_active":"2024-04-26T03:40:26.055670+0000","last_peered":"2024-04-26T03:40:26.055670+0000","last_clean":"2024-04-26T03:40:26.055670+0000","last_became_active":"2024-04-26T03:38:36.484388+0000","last_became_peered":"2024-04-26T03:38:36.484388+0000","last_unstale":"2024-04-26T03:40:26.055670+0000","last_undegraded":"2024-04-26T03:40:26.055670+0000","last_fullsized":"2024-04-26T03:40:26.055670+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:39:45.393105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:39:45.393105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:50.701189+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,13],"acting":[8,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":217,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:26.632834+0000","last_change":"2024-04-26T03:40:07.106714+0000","last_active":"2024-04-26T03:40:26.632834+0000","last_peered":"2024-04-26T03:40:26.632834+0000","last_clean":"2024-04-26T03:40:26.632834+0000","last_became_active":"2024-04-26T03:38:38.187383+0000","last_became_peered":"2024-04-26T03:38:38.187383+0000","last_unstale":"2024-04-26T03:40:26.632834+0000","last_undegraded":"2024-04-26T03:40:26.632834+0000","last_fullsized":"2024-04-26T03:40:26.632834+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:40:07.106612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:40:07.106612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:16.998505+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4],"acting":[9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":218,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:26.632157+0000","last_change":"2024-04-26T03:39:46.090838+0000","last_active":"2024-04-26T03:40:26.632157+0000","last_peered":"2024-04-26T03:40:26.632157+0000","last_clean":"2024-04-26T03:40:26.632157+0000","last_became_active":"2024-04-26T03:38:37.255066+0000","last_became_peered":"2024-04-26T03:38:37.255066+0000","last_unstale":"2024-04-26T03:40:26.632157+0000","last_undegraded":"2024-04-26T03:40:26.632157+0000","last_fullsized":"2024-04-26T03:40:26.632157+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:39:46.090747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:39:46.090747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:48.933362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":207,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:27.004238+0000","last_change":"2024-04-26T03:39:45.538938+0000","last_active":"2024-04-26T03:40:27.004238+0000","last_peered":"2024-04-26T03:40:27.004238+0000","last_clean":"2024-04-26T03:40:27.004238+0000","last_became_active":"2024-04-26T03:38:48.475155+0000","last_became_peered":"2024-04-26T03:38:48.475155+0000","last_unstale":"2024-04-26T03:40:27.004238+0000","last_undegraded":"2024-04-26T03:40:27.004238+0000","last_fullsized":"2024-04-26T03:40:27.004238+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:39:45.538842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:39:45.538842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:48.125744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,12],"acting":[11,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":217,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:26.631774+0000","last_change":"2024-04-26T03:40:01.434482+0000","last_active":"2024-04-26T03:40:26.631774+0000","last_peered":"2024-04-26T03:40:26.631774+0000","last_clean":"2024-04-26T03:40:26.631774+0000","last_became_active":"2024-04-26T03:38:37.255181+0000","last_became_peered":"2024-04-26T03:38:37.255181+0000","last_unstale":"2024-04-26T03:40:26.631774+0000","last_undegraded":"2024-04-26T03:40:26.631774+0000","last_fullsized":"2024-04-26T03:40:26.631774+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:40:01.434355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:40:01.434355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:19.401059+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0],"acting":[14,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":218,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:26.055830+0000","last_change":"2024-04-26T03:39:54.682639+0000","last_active":"2024-04-26T03:40:26.055830+0000","last_peered":"2024-04-26T03:40:26.055830+0000","last_clean":"2024-04-26T03:40:26.055830+0000","last_became_active":"2024-04-26T03:38:36.431121+0000","last_became_peered":"2024-04-26T03:38:36.431121+0000","last_unstale":"2024-04-26T03:40:26.055830+0000","last_undegraded":"2024-04-26T03:40:26.055830+0000","last_fullsized":"2024-04-26T03:40:26.055830+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:39:54.682516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:39:54.682516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:12.279328+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":217,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:26.631789+0000","last_change":"2024-04-26T03:39:48.469712+0000","last_active":"2024-04-26T03:40:26.631789+0000","last_peered":"2024-04-26T03:40:26.631789+0000","last_clean":"2024-04-26T03:40:26.631789+0000","last_became_active":"2024-04-26T03:38:36.431449+0000","last_became_peered":"2024-04-26T03:38:36.431449+0000","last_unstale":"2024-04-26T03:40:26.631789+0000","last_undegraded":"2024-04-26T03:40:26.631789+0000","last_fullsized":"2024-04-26T03:40:26.631789+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:39:48.469598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:39:48.469598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:02.722236+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"27'157","reported_seq":418,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:26.234535+0000","last_change":"2024-04-26T03:39:42.110436+0000","last_active":"2024-04-26T03:40:26.234535+0000","last_peered":"2024-04-26T03:40:26.234535+0000","last_clean":"2024-04-26T03:40:26.234535+0000","last_became_active":"2024-04-26T03:38:34.389099+0000","last_became_peered":"2024-04-26T03:38:34.389099+0000","last_unstale":"2024-04-26T03:40:26.234535+0000","last_undegraded":"2024-04-26T03:40:26.234535+0000","last_fullsized":"2024-04-26T03:40:26.234535+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'157","last_scrub_stamp":"2024-04-26T03:39:42.110300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:32.306810+0000","last_clean_scrub_stamp":"2024-04-26T03:39:42.110300+0000","objects_scrubbed":2,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:11.925418+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":13852,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":24,"seq":103079215133,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36876,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334964,"statfs":{"total":96636764160,"available":96599003136,"internally_reserved":0,"allocated":1564672,"data_stored":1662071,"data_compressed":222496,"data_compressed_allocated":466944,"data_compressed_original":1380352,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,4,6,7,8,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215133,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32660,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,4,5,6,7,9,10,11,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":23,"seq":98784247839,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36824,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335016,"statfs":{"total":96636764160,"available":96599056384,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,7,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280539,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32716,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339124,"statfs":{"total":96636764160,"available":96603262976,"internally_reserved":0,"allocated":1564672,"data_stored":1662071,"data_compressed":222496,"data_compressed_allocated":466944,"data_compressed_original":1380352,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,4,5,7,8,9,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280539,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32656,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,3,4,5,6,8,9,10,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":22,"seq":94489280540,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,4,5,6,7,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280538,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32652,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339188,"statfs":{"total":96636764160,"available":96603328512,"internally_reserved":0,"allocated":1499136,"data_stored":1123512,"data_compressed":213528,"data_compressed_allocated":430080,"data_compressed_original":860160,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,2,4,7,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280539,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[1,2,3,4,7,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313244,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50704,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321136,"statfs":{"total":96636764160,"available":96584843264,"internally_reserved":0,"allocated":1499136,"data_stored":1123512,"data_compressed":213528,"data_compressed_allocated":430080,"data_compressed_original":860160,"omap_allocated":1596,"internal_metadata":50395588},"hb_peers":[4,5,6,7,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":22,"seq":94489280538,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28500,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343340,"statfs":{"total":96636764160,"available":96607580160,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,7,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280538,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1499136,"data_stored":1123512,"data_compressed":213528,"data_compressed_allocated":430080,"data_compressed_original":860160,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[4,6,7,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247836,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54856,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316984,"statfs":{"total":96636764160,"available":96580591616,"internally_reserved":0,"allocated":1499136,"data_stored":1123512,"data_compressed":213528,"data_compressed_allocated":430080,"data_compressed_original":860160,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,6,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280538,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1499136,"data_stored":1123512,"data_compressed":213528,"data_compressed_allocated":430080,"data_compressed_original":860160,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,3,5,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":23,"seq":98784247838,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":54868,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316972,"statfs":{"total":96636764160,"available":96580579328,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,4,6,7,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280538,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28496,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343344,"statfs":{"total":96636764160,"available":96607584256,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,4,5,7,8,9,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247836,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32660,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T03:40:31.894 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-26T03:40:31.894 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T03:40:32.191 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:40:32.192 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-04-26T03:40:32.204 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":175,"stamp":"2024-04-26T03:40:32.073554+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":13,"kb":1509949440,"kb_used":585092,"kb_used_data":23672,"kb_used_omap":24,"kb_used_meta":561127,"kb_avail":1509364348,"statfs":{"total":1546188226560,"available":1545589092352,"internally_reserved":0,"allocated":24240128,"data_stored":19149620,"data_compressed":3454804,"data_compressed_allocated":6995968,"data_compressed_original":14884864,"omap_allocated":25522,"internal_metadata":574594126},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001801"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":220,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:26.632577+0000","last_change":"2024-04-26T03:40:08.073279+0000","last_active":"2024-04-26T03:40:26.632577+0000","last_peered":"2024-04-26T03:40:26.632577+0000","last_clean":"2024-04-26T03:40:26.632577+0000","last_became_active":"2024-04-26T03:38:37.358910+0000","last_became_peered":"2024-04-26T03:38:37.358910+0000","last_unstale":"2024-04-26T03:40:26.632577+0000","last_undegraded":"2024-04-26T03:40:26.632577+0000","last_fullsized":"2024-04-26T03:40:26.632577+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:40:08.073132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:40:08.073132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:17.208416+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":217,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:26.055670+0000","last_change":"2024-04-26T03:39:45.393243+0000","last_active":"2024-04-26T03:40:26.055670+0000","last_peered":"2024-04-26T03:40:26.055670+0000","last_clean":"2024-04-26T03:40:26.055670+0000","last_became_active":"2024-04-26T03:38:36.484388+0000","last_became_peered":"2024-04-26T03:38:36.484388+0000","last_unstale":"2024-04-26T03:40:26.055670+0000","last_undegraded":"2024-04-26T03:40:26.055670+0000","last_fullsized":"2024-04-26T03:40:26.055670+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:39:45.393105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:39:45.393105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:50.701189+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,13],"acting":[8,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":217,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:26.632834+0000","last_change":"2024-04-26T03:40:07.106714+0000","last_active":"2024-04-26T03:40:26.632834+0000","last_peered":"2024-04-26T03:40:26.632834+0000","last_clean":"2024-04-26T03:40:26.632834+0000","last_became_active":"2024-04-26T03:38:38.187383+0000","last_became_peered":"2024-04-26T03:38:38.187383+0000","last_unstale":"2024-04-26T03:40:26.632834+0000","last_undegraded":"2024-04-26T03:40:26.632834+0000","last_fullsized":"2024-04-26T03:40:26.632834+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:40:07.106612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:40:07.106612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:16.998505+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4],"acting":[9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":218,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:26.632157+0000","last_change":"2024-04-26T03:39:46.090838+0000","last_active":"2024-04-26T03:40:26.632157+0000","last_peered":"2024-04-26T03:40:26.632157+0000","last_clean":"2024-04-26T03:40:26.632157+0000","last_became_active":"2024-04-26T03:38:37.255066+0000","last_became_peered":"2024-04-26T03:38:37.255066+0000","last_unstale":"2024-04-26T03:40:26.632157+0000","last_undegraded":"2024-04-26T03:40:26.632157+0000","last_fullsized":"2024-04-26T03:40:26.632157+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:39:46.090747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:39:46.090747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:48.933362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":207,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:27.004238+0000","last_change":"2024-04-26T03:39:45.538938+0000","last_active":"2024-04-26T03:40:27.004238+0000","last_peered":"2024-04-26T03:40:27.004238+0000","last_clean":"2024-04-26T03:40:27.004238+0000","last_became_active":"2024-04-26T03:38:48.475155+0000","last_became_peered":"2024-04-26T03:38:48.475155+0000","last_unstale":"2024-04-26T03:40:27.004238+0000","last_undegraded":"2024-04-26T03:40:27.004238+0000","last_fullsized":"2024-04-26T03:40:27.004238+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:39:45.538842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:39:45.538842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:48.125744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,12],"acting":[11,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":217,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:26.631774+0000","last_change":"2024-04-26T03:40:01.434482+0000","last_active":"2024-04-26T03:40:26.631774+0000","last_peered":"2024-04-26T03:40:26.631774+0000","last_clean":"2024-04-26T03:40:26.631774+0000","last_became_active":"2024-04-26T03:38:37.255181+0000","last_became_peered":"2024-04-26T03:38:37.255181+0000","last_unstale":"2024-04-26T03:40:26.631774+0000","last_undegraded":"2024-04-26T03:40:26.631774+0000","last_fullsized":"2024-04-26T03:40:26.631774+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:40:01.434355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:40:01.434355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:19.401059+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0],"acting":[14,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":218,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:26.055830+0000","last_change":"2024-04-26T03:39:54.682639+0000","last_active":"2024-04-26T03:40:26.055830+0000","last_peered":"2024-04-26T03:40:26.055830+0000","last_clean":"2024-04-26T03:40:26.055830+0000","last_became_active":"2024-04-26T03:38:36.431121+0000","last_became_peered":"2024-04-26T03:38:36.431121+0000","last_unstale":"2024-04-26T03:40:26.055830+0000","last_undegraded":"2024-04-26T03:40:26.055830+0000","last_fullsized":"2024-04-26T03:40:26.055830+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:39:54.682516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:39:54.682516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:12.279328+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":217,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:26.631789+0000","last_change":"2024-04-26T03:39:48.469712+0000","last_active":"2024-04-26T03:40:26.631789+0000","last_peered":"2024-04-26T03:40:26.631789+0000","last_clean":"2024-04-26T03:40:26.631789+0000","last_became_active":"2024-04-26T03:38:36.431449+0000","last_became_peered":"2024-04-26T03:38:36.431449+0000","last_unstale":"2024-04-26T03:40:26.631789+0000","last_undegraded":"2024-04-26T03:40:26.631789+0000","last_fullsized":"2024-04-26T03:40:26.631789+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:39:48.469598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:39:48.469598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:02.722236+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"27'157","reported_seq":418,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:26.234535+0000","last_change":"2024-04-26T03:39:42.110436+0000","last_active":"2024-04-26T03:40:26.234535+0000","last_peered":"2024-04-26T03:40:26.234535+0000","last_clean":"2024-04-26T03:40:26.234535+0000","last_became_active":"2024-04-26T03:38:34.389099+0000","last_became_peered":"2024-04-26T03:38:34.389099+0000","last_unstale":"2024-04-26T03:40:26.234535+0000","last_undegraded":"2024-04-26T03:40:26.234535+0000","last_fullsized":"2024-04-26T03:40:26.234535+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'157","last_scrub_stamp":"2024-04-26T03:39:42.110300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:32.306810+0000","last_clean_scrub_stamp":"2024-04-26T03:39:42.110300+0000","objects_scrubbed":2,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:11.925418+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":13852,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":24,"seq":103079215133,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36876,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334964,"statfs":{"total":96636764160,"available":96599003136,"internally_reserved":0,"allocated":1564672,"data_stored":1662071,"data_compressed":222496,"data_compressed_allocated":466944,"data_compressed_original":1380352,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,4,6,7,8,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215133,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32660,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,4,5,6,7,9,10,11,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":23,"seq":98784247839,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36824,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335016,"statfs":{"total":96636764160,"available":96599056384,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,7,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280539,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32716,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339124,"statfs":{"total":96636764160,"available":96603262976,"internally_reserved":0,"allocated":1564672,"data_stored":1662071,"data_compressed":222496,"data_compressed_allocated":466944,"data_compressed_original":1380352,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,4,5,7,8,9,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280539,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32656,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,3,4,5,6,8,9,10,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":22,"seq":94489280540,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,4,5,6,7,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280538,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32652,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339188,"statfs":{"total":96636764160,"available":96603328512,"internally_reserved":0,"allocated":1499136,"data_stored":1123512,"data_compressed":213528,"data_compressed_allocated":430080,"data_compressed_original":860160,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,2,4,7,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280539,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[1,2,3,4,7,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313245,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50704,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321136,"statfs":{"total":96636764160,"available":96584843264,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1596,"internal_metadata":50395588},"hb_peers":[4,5,6,7,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":12,"up_from":22,"seq":94489280538,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28500,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343340,"statfs":{"total":96636764160,"available":96607580160,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,7,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280538,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1499136,"data_stored":1123512,"data_compressed":213528,"data_compressed_allocated":430080,"data_compressed_original":860160,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[4,6,7,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247836,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54856,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316984,"statfs":{"total":96636764160,"available":96580591616,"internally_reserved":0,"allocated":1499136,"data_stored":1123512,"data_compressed":213528,"data_compressed_allocated":430080,"data_compressed_original":860160,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,6,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280538,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1499136,"data_stored":1123512,"data_compressed":213528,"data_compressed_allocated":430080,"data_compressed_original":860160,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,3,5,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":23,"seq":98784247838,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":54868,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316972,"statfs":{"total":96636764160,"available":96580579328,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,4,6,7,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280538,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28496,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343344,"statfs":{"total":96636764160,"available":96607584256,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,4,5,7,8,9,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247836,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32660,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T03:40:32.206 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T03:40:32.505 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:40:32.505 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":129,"fsid":"c5a13c99-5c9a-4b8b-a893-4f6f4211677a","created":"2024-04-26T03:38:19.896648+0000","modified":"2024-04-26T03:40:26.051686+0000","last_up_change":"2024-04-26T03:38:33.308561+0000","last_in_change":"2024-04-26T03:38:24.150336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T03:38:32.306696+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-26T03:38:34.592633+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.75,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"31b0e7a9-455d-4874-9c60-26efac8d0b1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2377656090},{"type":"v1","addr":"172.21.15.28:6801","nonce":2377656090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2377656090},{"type":"v1","addr":"172.21.15.28:6803","nonce":2377656090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2377656090},{"type":"v1","addr":"172.21.15.28:6807","nonce":2377656090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2377656090},{"type":"v1","addr":"172.21.15.28:6805","nonce":2377656090}]},"public_addr":"172.21.15.28:6801/2377656090","cluster_addr":"172.21.15.28:6803/2377656090","heartbeat_back_addr":"172.21.15.28:6807/2377656090","heartbeat_front_addr":"172.21.15.28:6805/2377656090","state":["exists","up"]},{"osd":1,"uuid":"52819ace-a070-4822-b33e-3c54466e61ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":1678150206},{"type":"v1","addr":"172.21.15.42:6801","nonce":1678150206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1678150206},{"type":"v1","addr":"172.21.15.42:6803","nonce":1678150206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1678150206},{"type":"v1","addr":"172.21.15.42:6807","nonce":1678150206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1678150206},{"type":"v1","addr":"172.21.15.42:6805","nonce":1678150206}]},"public_addr":"172.21.15.42:6801/1678150206","cluster_addr":"172.21.15.42:6803/1678150206","heartbeat_back_addr":"172.21.15.42:6807/1678150206","heartbeat_front_addr":"172.21.15.42:6805/1678150206","state":["exists","up"]},{"osd":2,"uuid":"858f8ac1-2edc-43d5-914c-c1914bdc0c9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":714535415},{"type":"v1","addr":"172.21.15.66:6801","nonce":714535415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":714535415},{"type":"v1","addr":"172.21.15.66:6803","nonce":714535415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":714535415},{"type":"v1","addr":"172.21.15.66:6807","nonce":714535415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":714535415},{"type":"v1","addr":"172.21.15.66:6805","nonce":714535415}]},"public_addr":"172.21.15.66:6801/714535415","cluster_addr":"172.21.15.66:6803/714535415","heartbeat_back_addr":"172.21.15.66:6807/714535415","heartbeat_front_addr":"172.21.15.66:6805/714535415","state":["exists","up"]},{"osd":3,"uuid":"531a2e30-3351-4f84-804b-5fb5050b84ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2377983625},{"type":"v1","addr":"172.21.15.92:6811","nonce":2377983625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2377983625},{"type":"v1","addr":"172.21.15.92:6813","nonce":2377983625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2377983625},{"type":"v1","addr":"172.21.15.92:6817","nonce":2377983625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2377983625},{"type":"v1","addr":"172.21.15.92:6815","nonce":2377983625}]},"public_addr":"172.21.15.92:6811/2377983625","cluster_addr":"172.21.15.92:6813/2377983625","heartbeat_back_addr":"172.21.15.92:6817/2377983625","heartbeat_front_addr":"172.21.15.92:6815/2377983625","state":["exists","up"]},{"osd":4,"uuid":"6d38ffda-40eb-4fd7-9300-2a09f2673e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":183752455},{"type":"v1","addr":"172.21.15.28:6809","nonce":183752455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":183752455},{"type":"v1","addr":"172.21.15.28:6811","nonce":183752455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":183752455},{"type":"v1","addr":"172.21.15.28:6815","nonce":183752455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":183752455},{"type":"v1","addr":"172.21.15.28:6813","nonce":183752455}]},"public_addr":"172.21.15.28:6809/183752455","cluster_addr":"172.21.15.28:6811/183752455","heartbeat_back_addr":"172.21.15.28:6815/183752455","heartbeat_front_addr":"172.21.15.28:6813/183752455","state":["exists","up"]},{"osd":5,"uuid":"361aa40d-e53b-42af-afd4-81fbcac28725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2012261939},{"type":"v1","addr":"172.21.15.42:6809","nonce":2012261939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2012261939},{"type":"v1","addr":"172.21.15.42:6811","nonce":2012261939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2012261939},{"type":"v1","addr":"172.21.15.42:6815","nonce":2012261939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2012261939},{"type":"v1","addr":"172.21.15.42:6813","nonce":2012261939}]},"public_addr":"172.21.15.42:6809/2012261939","cluster_addr":"172.21.15.42:6811/2012261939","heartbeat_back_addr":"172.21.15.42:6815/2012261939","heartbeat_front_addr":"172.21.15.42:6813/2012261939","state":["exists","up"]},{"osd":6,"uuid":"f610ad3e-1fd8-42bd-af4b-0d79c3bb9cf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":3533460882},{"type":"v1","addr":"172.21.15.66:6809","nonce":3533460882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":3533460882},{"type":"v1","addr":"172.21.15.66:6811","nonce":3533460882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":3533460882},{"type":"v1","addr":"172.21.15.66:6815","nonce":3533460882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":3533460882},{"type":"v1","addr":"172.21.15.66:6813","nonce":3533460882}]},"public_addr":"172.21.15.66:6809/3533460882","cluster_addr":"172.21.15.66:6811/3533460882","heartbeat_back_addr":"172.21.15.66:6815/3533460882","heartbeat_front_addr":"172.21.15.66:6813/3533460882","state":["exists","up"]},{"osd":7,"uuid":"a4bae6fb-e087-4563-a129-6938e18eed8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":1760163933},{"type":"v1","addr":"172.21.15.92:6803","nonce":1760163933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":1760163933},{"type":"v1","addr":"172.21.15.92:6805","nonce":1760163933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1760163933},{"type":"v1","addr":"172.21.15.92:6809","nonce":1760163933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":1760163933},{"type":"v1","addr":"172.21.15.92:6807","nonce":1760163933}]},"public_addr":"172.21.15.92:6803/1760163933","cluster_addr":"172.21.15.92:6805/1760163933","heartbeat_back_addr":"172.21.15.92:6809/1760163933","heartbeat_front_addr":"172.21.15.92:6807/1760163933","state":["exists","up"]},{"osd":8,"uuid":"479f3885-452e-4231-bdc2-b79668384ba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":4127602407},{"type":"v1","addr":"172.21.15.28:6817","nonce":4127602407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":4127602407},{"type":"v1","addr":"172.21.15.28:6819","nonce":4127602407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":4127602407},{"type":"v1","addr":"172.21.15.28:6823","nonce":4127602407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":4127602407},{"type":"v1","addr":"172.21.15.28:6821","nonce":4127602407}]},"public_addr":"172.21.15.28:6817/4127602407","cluster_addr":"172.21.15.28:6819/4127602407","heartbeat_back_addr":"172.21.15.28:6823/4127602407","heartbeat_front_addr":"172.21.15.28:6821/4127602407","state":["exists","up"]},{"osd":9,"uuid":"9a603a47-7c5a-47d4-a298-56b3ac991892","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":4168729845},{"type":"v1","addr":"172.21.15.42:6818","nonce":4168729845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":4168729845},{"type":"v1","addr":"172.21.15.42:6822","nonce":4168729845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":4168729845},{"type":"v1","addr":"172.21.15.42:6830","nonce":4168729845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":4168729845},{"type":"v1","addr":"172.21.15.42:6826","nonce":4168729845}]},"public_addr":"172.21.15.42:6818/4168729845","cluster_addr":"172.21.15.42:6822/4168729845","heartbeat_back_addr":"172.21.15.42:6830/4168729845","heartbeat_front_addr":"172.21.15.42:6826/4168729845","state":["exists","up"]},{"osd":10,"uuid":"1cd42945-6ee1-49f0-b17b-4093dd36b4ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4180572718},{"type":"v1","addr":"172.21.15.66:6817","nonce":4180572718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":4180572718},{"type":"v1","addr":"172.21.15.66:6819","nonce":4180572718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":4180572718},{"type":"v1","addr":"172.21.15.66:6823","nonce":4180572718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":4180572718},{"type":"v1","addr":"172.21.15.66:6821","nonce":4180572718}]},"public_addr":"172.21.15.66:6817/4180572718","cluster_addr":"172.21.15.66:6819/4180572718","heartbeat_back_addr":"172.21.15.66:6823/4180572718","heartbeat_front_addr":"172.21.15.66:6821/4180572718","state":["exists","up"]},{"osd":11,"uuid":"b41e8100-1d4f-4b7a-991a-7a27bc29feec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":295713815},{"type":"v1","addr":"172.21.15.92:6819","nonce":295713815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":295713815},{"type":"v1","addr":"172.21.15.92:6821","nonce":295713815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":295713815},{"type":"v1","addr":"172.21.15.92:6825","nonce":295713815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":295713815},{"type":"v1","addr":"172.21.15.92:6823","nonce":295713815}]},"public_addr":"172.21.15.92:6819/295713815","cluster_addr":"172.21.15.92:6821/295713815","heartbeat_back_addr":"172.21.15.92:6825/295713815","heartbeat_front_addr":"172.21.15.92:6823/295713815","state":["exists","up"]},{"osd":12,"uuid":"458cf150-32e1-449e-b533-ac6ab5226f85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":2081141953},{"type":"v1","addr":"172.21.15.28:6825","nonce":2081141953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2081141953},{"type":"v1","addr":"172.21.15.28:6827","nonce":2081141953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2081141953},{"type":"v1","addr":"172.21.15.28:6831","nonce":2081141953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":2081141953},{"type":"v1","addr":"172.21.15.28:6829","nonce":2081141953}]},"public_addr":"172.21.15.28:6825/2081141953","cluster_addr":"172.21.15.28:6827/2081141953","heartbeat_back_addr":"172.21.15.28:6831/2081141953","heartbeat_front_addr":"172.21.15.28:6829/2081141953","state":["exists","up"]},{"osd":13,"uuid":"f008462e-a735-4845-9ecf-dd7d4a6941af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6817","nonce":301360893},{"type":"v1","addr":"172.21.15.42:6819","nonce":301360893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6821","nonce":301360893},{"type":"v1","addr":"172.21.15.42:6823","nonce":301360893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6829","nonce":301360893},{"type":"v1","addr":"172.21.15.42:6831","nonce":301360893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6825","nonce":301360893},{"type":"v1","addr":"172.21.15.42:6827","nonce":301360893}]},"public_addr":"172.21.15.42:6819/301360893","cluster_addr":"172.21.15.42:6823/301360893","heartbeat_back_addr":"172.21.15.42:6831/301360893","heartbeat_front_addr":"172.21.15.42:6827/301360893","state":["exists","up"]},{"osd":14,"uuid":"6f66ae97-eb8a-48eb-9bcc-20f65fbf690b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":1534905602},{"type":"v1","addr":"172.21.15.66:6825","nonce":1534905602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":1534905602},{"type":"v1","addr":"172.21.15.66:6827","nonce":1534905602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":1534905602},{"type":"v1","addr":"172.21.15.66:6831","nonce":1534905602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":1534905602},{"type":"v1","addr":"172.21.15.66:6829","nonce":1534905602}]},"public_addr":"172.21.15.66:6825/1534905602","cluster_addr":"172.21.15.66:6827/1534905602","heartbeat_back_addr":"172.21.15.66:6831/1534905602","heartbeat_front_addr":"172.21.15.66:6829/1534905602","state":["exists","up"]},{"osd":15,"uuid":"b856bfc8-253b-4112-be5c-f67d61d01768","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3982634323},{"type":"v1","addr":"172.21.15.92:6827","nonce":3982634323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3982634323},{"type":"v1","addr":"172.21.15.92:6829","nonce":3982634323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3982634323},{"type":"v1","addr":"172.21.15.92:6833","nonce":3982634323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3982634323},{"type":"v1","addr":"172.21.15.92:6831","nonce":3982634323}]},"public_addr":"172.21.15.92:6827/3982634323","cluster_addr":"172.21.15.92:6829/3982634323","heartbeat_back_addr":"172.21.15.92:6833/3982634323","heartbeat_front_addr":"172.21.15.92:6831/3982634323","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:40.180887+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:34.060445+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:35.302133+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:34.066992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:52.275741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:33.955241+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:40.990020+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:53.221513+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:32.243723+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:49.481065+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:55.555809+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:35.430680+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:32.221433+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:34.790817+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:57.884579+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T03:39:36.409075+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.2","mappings":[{"from":3,"to":11}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"shecprofile":{"c":"2","crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","k":"4","m":"3","name":"shecprofile","plugin":"shec","technique":"multiple"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":25,"length":1},{"begin":28,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T03:40:33.521 INFO:tasks.ceph:Scrubbing osd.0 2024-04-26T03:40:33.521 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-26T03:40:33.661 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:40:33.662 INFO:teuthology.orchestra.run.smithi028.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T03:40:33.662 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:40:33.673 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-26T03:40:33.972 INFO:teuthology.orchestra.run.smithi028.stderr:instructed osd(s) 0 to deep-scrub 2024-04-26T03:40:33.985 INFO:tasks.ceph:Scrubbing osd.4 2024-04-26T03:40:33.985 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-26T03:40:34.111 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:40:34.111 INFO:teuthology.orchestra.run.smithi028.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T03:40:34.111 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:40:34.122 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-26T03:40:34.418 INFO:teuthology.orchestra.run.smithi028.stderr:instructed osd(s) 4 to deep-scrub 2024-04-26T03:40:34.431 INFO:tasks.ceph:Scrubbing osd.8 2024-04-26T03:40:34.431 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 config set osd_debug_deep_scrub_sleep 0 2024-04-26T03:40:34.558 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:40:34.558 INFO:teuthology.orchestra.run.smithi028.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T03:40:34.558 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:40:34.568 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 8 2024-04-26T03:40:34.866 INFO:teuthology.orchestra.run.smithi028.stderr:instructed osd(s) 8 to deep-scrub 2024-04-26T03:40:34.879 INFO:tasks.ceph:Scrubbing osd.12 2024-04-26T03:40:34.879 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.12 config set osd_debug_deep_scrub_sleep 0 2024-04-26T03:40:35.006 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:40:35.006 INFO:teuthology.orchestra.run.smithi028.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T03:40:35.006 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:40:35.016 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 12 2024-04-26T03:40:35.314 INFO:teuthology.orchestra.run.smithi028.stderr:instructed osd(s) 12 to deep-scrub 2024-04-26T03:40:35.327 INFO:tasks.ceph:Scrubbing osd.1 2024-04-26T03:40:35.328 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-26T03:40:35.455 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:40:35.455 INFO:teuthology.orchestra.run.smithi028.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T03:40:35.455 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:40:35.466 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-26T03:40:35.763 INFO:teuthology.orchestra.run.smithi028.stderr:instructed osd(s) 1 to deep-scrub 2024-04-26T03:40:35.776 INFO:tasks.ceph:Scrubbing osd.5 2024-04-26T03:40:35.777 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-26T03:40:35.904 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:40:35.904 INFO:teuthology.orchestra.run.smithi028.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T03:40:35.904 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:40:35.915 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-26T03:40:36.213 INFO:teuthology.orchestra.run.smithi028.stderr:instructed osd(s) 5 to deep-scrub 2024-04-26T03:40:36.226 INFO:tasks.ceph:Scrubbing osd.9 2024-04-26T03:40:36.226 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 config set osd_debug_deep_scrub_sleep 0 2024-04-26T03:40:36.354 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:40:36.354 INFO:teuthology.orchestra.run.smithi028.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T03:40:36.354 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:40:36.365 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 9 2024-04-26T03:40:36.661 INFO:teuthology.orchestra.run.smithi028.stderr:instructed osd(s) 9 to deep-scrub 2024-04-26T03:40:36.674 INFO:tasks.ceph:Scrubbing osd.13 2024-04-26T03:40:36.675 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.13 config set osd_debug_deep_scrub_sleep 0 2024-04-26T03:40:36.802 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:40:36.802 INFO:teuthology.orchestra.run.smithi028.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T03:40:36.803 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:40:36.813 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 13 2024-04-26T03:40:37.113 INFO:teuthology.orchestra.run.smithi028.stderr:instructed osd(s) 13 to deep-scrub 2024-04-26T03:40:37.126 INFO:tasks.ceph:Scrubbing osd.2 2024-04-26T03:40:37.127 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-26T03:40:37.254 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:40:37.254 INFO:teuthology.orchestra.run.smithi028.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T03:40:37.254 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:40:37.265 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-26T03:40:37.564 INFO:teuthology.orchestra.run.smithi028.stderr:instructed osd(s) 2 to deep-scrub 2024-04-26T03:40:37.577 INFO:tasks.ceph:Scrubbing osd.6 2024-04-26T03:40:37.578 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-26T03:40:37.705 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:40:37.705 INFO:teuthology.orchestra.run.smithi028.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T03:40:37.706 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:40:37.716 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-26T03:40:38.013 INFO:teuthology.orchestra.run.smithi028.stderr:instructed osd(s) 6 to deep-scrub 2024-04-26T03:40:38.026 INFO:tasks.ceph:Scrubbing osd.10 2024-04-26T03:40:38.026 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 config set osd_debug_deep_scrub_sleep 0 2024-04-26T03:40:38.154 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:40:38.155 INFO:teuthology.orchestra.run.smithi028.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T03:40:38.155 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:40:38.165 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 10 2024-04-26T03:40:38.462 INFO:teuthology.orchestra.run.smithi028.stderr:instructed osd(s) 10 to deep-scrub 2024-04-26T03:40:38.474 INFO:tasks.ceph:Scrubbing osd.14 2024-04-26T03:40:38.475 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.14 config set osd_debug_deep_scrub_sleep 0 2024-04-26T03:40:38.602 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:40:38.602 INFO:teuthology.orchestra.run.smithi028.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T03:40:38.602 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:40:38.613 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 14 2024-04-26T03:40:38.912 INFO:teuthology.orchestra.run.smithi028.stderr:instructed osd(s) 14 to deep-scrub 2024-04-26T03:40:38.925 INFO:tasks.ceph:Scrubbing osd.3 2024-04-26T03:40:38.925 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-26T03:40:39.052 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:40:39.053 INFO:teuthology.orchestra.run.smithi028.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T03:40:39.053 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:40:39.063 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-26T03:40:39.360 INFO:teuthology.orchestra.run.smithi028.stderr:instructed osd(s) 3 to deep-scrub 2024-04-26T03:40:39.373 INFO:tasks.ceph:Scrubbing osd.7 2024-04-26T03:40:39.373 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-26T03:40:39.500 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:40:39.501 INFO:teuthology.orchestra.run.smithi028.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T03:40:39.501 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:40:39.511 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-26T03:40:39.808 INFO:teuthology.orchestra.run.smithi028.stderr:instructed osd(s) 7 to deep-scrub 2024-04-26T03:40:39.821 INFO:tasks.ceph:Scrubbing osd.11 2024-04-26T03:40:39.821 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 config set osd_debug_deep_scrub_sleep 0 2024-04-26T03:40:39.948 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:40:39.948 INFO:teuthology.orchestra.run.smithi028.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T03:40:39.948 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:40:39.959 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 11 2024-04-26T03:40:40.263 INFO:teuthology.orchestra.run.smithi028.stderr:instructed osd(s) 11 to deep-scrub 2024-04-26T03:40:40.275 INFO:tasks.ceph:Scrubbing osd.15 2024-04-26T03:40:40.275 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.15 config set osd_debug_deep_scrub_sleep 0 2024-04-26T03:40:40.403 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T03:40:40.403 INFO:teuthology.orchestra.run.smithi028.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T03:40:40.403 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T03:40:40.414 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 15 2024-04-26T03:40:40.713 INFO:teuthology.orchestra.run.smithi028.stderr:instructed osd(s) 15 to deep-scrub 2024-04-26T03:40:40.726 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T03:40:41.026 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:40:41.026 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-04-26T03:40:41.039 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":181,"stamp":"2024-04-26T03:40:40.075474+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":13,"kb":1509949440,"kb_used":585100,"kb_used_data":23744,"kb_used_omap":24,"kb_used_meta":561127,"kb_avail":1509364340,"statfs":{"total":1546188226560,"available":1545589084160,"internally_reserved":0,"allocated":24313856,"data_stored":19207490,"data_compressed":3467058,"data_compressed_allocated":7020544,"data_compressed_original":14934016,"omap_allocated":25522,"internal_metadata":574594126},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001920"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":220,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:26.632577+0000","last_change":"2024-04-26T03:40:08.073279+0000","last_active":"2024-04-26T03:40:26.632577+0000","last_peered":"2024-04-26T03:40:26.632577+0000","last_clean":"2024-04-26T03:40:26.632577+0000","last_became_active":"2024-04-26T03:38:37.358910+0000","last_became_peered":"2024-04-26T03:38:37.358910+0000","last_unstale":"2024-04-26T03:40:26.632577+0000","last_undegraded":"2024-04-26T03:40:26.632577+0000","last_fullsized":"2024-04-26T03:40:26.632577+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:40:08.073132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:40:08.073132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:17.208416+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":226,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:35.366891+0000","last_change":"2024-04-26T03:40:35.366911+0000","last_active":"2024-04-26T03:40:35.366891+0000","last_peered":"2024-04-26T03:40:35.366891+0000","last_clean":"2024-04-26T03:40:35.366891+0000","last_became_active":"2024-04-26T03:38:36.484388+0000","last_became_peered":"2024-04-26T03:38:36.484388+0000","last_unstale":"2024-04-26T03:40:35.366891+0000","last_undegraded":"2024-04-26T03:40:35.366891+0000","last_fullsized":"2024-04-26T03:40:35.366891+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:40:35.366825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:40:35.366825+0000","last_clean_scrub_stamp":"2024-04-26T03:40:35.366825+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:38.637072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,13],"acting":[8,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":218,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:26.632875+0000","last_change":"2024-04-26T03:40:07.106714+0000","last_active":"2024-04-26T03:40:26.632875+0000","last_peered":"2024-04-26T03:40:26.632875+0000","last_clean":"2024-04-26T03:40:26.632875+0000","last_became_active":"2024-04-26T03:38:38.187383+0000","last_became_peered":"2024-04-26T03:38:38.187383+0000","last_unstale":"2024-04-26T03:40:26.632875+0000","last_undegraded":"2024-04-26T03:40:26.632875+0000","last_fullsized":"2024-04-26T03:40:26.632875+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:40:07.106612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:40:07.106612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for deep scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4],"acting":[9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":227,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:35.098937+0000","last_change":"2024-04-26T03:40:35.098976+0000","last_active":"2024-04-26T03:40:35.098937+0000","last_peered":"2024-04-26T03:40:35.098937+0000","last_clean":"2024-04-26T03:40:35.098937+0000","last_became_active":"2024-04-26T03:38:37.255066+0000","last_became_peered":"2024-04-26T03:38:37.255066+0000","last_unstale":"2024-04-26T03:40:35.098937+0000","last_undegraded":"2024-04-26T03:40:35.098937+0000","last_fullsized":"2024-04-26T03:40:35.098937+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:40:35.098832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:40:35.098832+0000","last_clean_scrub_stamp":"2024-04-26T03:40:35.098832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:38.838380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":207,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:27.004238+0000","last_change":"2024-04-26T03:39:45.538938+0000","last_active":"2024-04-26T03:40:27.004238+0000","last_peered":"2024-04-26T03:40:27.004238+0000","last_clean":"2024-04-26T03:40:27.004238+0000","last_became_active":"2024-04-26T03:38:48.475155+0000","last_became_peered":"2024-04-26T03:38:48.475155+0000","last_unstale":"2024-04-26T03:40:27.004238+0000","last_undegraded":"2024-04-26T03:40:27.004238+0000","last_fullsized":"2024-04-26T03:40:27.004238+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:39:45.538842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:39:45.538842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:48.125744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,12],"acting":[11,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":217,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:26.631774+0000","last_change":"2024-04-26T03:40:01.434482+0000","last_active":"2024-04-26T03:40:26.631774+0000","last_peered":"2024-04-26T03:40:26.631774+0000","last_clean":"2024-04-26T03:40:26.631774+0000","last_became_active":"2024-04-26T03:38:37.255181+0000","last_became_peered":"2024-04-26T03:38:37.255181+0000","last_unstale":"2024-04-26T03:40:26.631774+0000","last_undegraded":"2024-04-26T03:40:26.631774+0000","last_fullsized":"2024-04-26T03:40:26.631774+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:40:01.434355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:40:01.434355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:19.401059+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0],"acting":[14,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":221,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-26T03:40:38.603975+0000","last_change":"2024-04-26T03:39:54.682639+0000","last_active":"2024-04-26T03:40:38.603975+0000","last_peered":"2024-04-26T03:40:38.603975+0000","last_clean":"2024-04-26T03:40:38.603975+0000","last_became_active":"2024-04-26T03:38:36.431121+0000","last_became_peered":"2024-04-26T03:38:36.431121+0000","last_unstale":"2024-04-26T03:40:38.603975+0000","last_undegraded":"2024-04-26T03:40:38.603975+0000","last_fullsized":"2024-04-26T03:40:38.603975+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:39:54.682516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:39:54.682516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:12.279328+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":217,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:26.631789+0000","last_change":"2024-04-26T03:39:48.469712+0000","last_active":"2024-04-26T03:40:26.631789+0000","last_peered":"2024-04-26T03:40:26.631789+0000","last_clean":"2024-04-26T03:40:26.631789+0000","last_became_active":"2024-04-26T03:38:36.431449+0000","last_became_peered":"2024-04-26T03:38:36.431449+0000","last_unstale":"2024-04-26T03:40:26.631789+0000","last_undegraded":"2024-04-26T03:40:26.631789+0000","last_fullsized":"2024-04-26T03:40:26.631789+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:39:48.469598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:35.319452+0000","last_clean_scrub_stamp":"2024-04-26T03:39:48.469598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:02.722236+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"27'157","reported_seq":419,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-26T03:40:26.234582+0000","last_change":"2024-04-26T03:39:42.110436+0000","last_active":"2024-04-26T03:40:26.234582+0000","last_peered":"2024-04-26T03:40:26.234582+0000","last_clean":"2024-04-26T03:40:26.234582+0000","last_became_active":"2024-04-26T03:38:34.389099+0000","last_became_peered":"2024-04-26T03:38:34.389099+0000","last_unstale":"2024-04-26T03:40:26.234582+0000","last_undegraded":"2024-04-26T03:40:26.234582+0000","last_fullsized":"2024-04-26T03:40:26.234582+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'157","last_scrub_stamp":"2024-04-26T03:39:42.110300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:38:32.306810+0000","last_clean_scrub_stamp":"2024-04-26T03:39:42.110300+0000","objects_scrubbed":2,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:11.925418+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":13852,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":24,"seq":103079215135,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36880,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334960,"statfs":{"total":96636764160,"available":96598999040,"internally_reserved":0,"allocated":1576960,"data_stored":1671744,"data_compressed":224539,"data_compressed_allocated":471040,"data_compressed_original":1388544,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,4,6,7,8,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215135,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32668,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339172,"statfs":{"total":96636764160,"available":96603312128,"internally_reserved":0,"allocated":1523712,"data_stored":1142816,"data_compressed":217613,"data_compressed_allocated":438272,"data_compressed_original":876544,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,4,5,6,7,9,10,11,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":23,"seq":98784247841,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36812,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335028,"statfs":{"total":96636764160,"available":96599068672,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,7,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280541,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32716,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339124,"statfs":{"total":96636764160,"available":96603262976,"internally_reserved":0,"allocated":1564672,"data_stored":1662071,"data_compressed":222496,"data_compressed_allocated":466944,"data_compressed_original":1380352,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,4,5,7,8,9,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280541,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32656,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,3,4,5,6,8,9,10,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":22,"seq":94489280542,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32652,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339188,"statfs":{"total":96636764160,"available":96603328512,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,4,5,6,7,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280540,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,2,4,7,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280541,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[1,2,3,4,7,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313246,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50704,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321136,"statfs":{"total":96636764160,"available":96584843264,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1596,"internal_metadata":50395588},"hb_peers":[4,5,6,7,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":12,"up_from":22,"seq":94489280540,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28500,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343340,"statfs":{"total":96636764160,"available":96607580160,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,7,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280540,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[4,6,7,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247838,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54864,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316976,"statfs":{"total":96636764160,"available":96580583424,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,6,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280540,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,3,5,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":23,"seq":98784247840,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":54868,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316972,"statfs":{"total":96636764160,"available":96580579328,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,4,6,7,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280540,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28496,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343344,"statfs":{"total":96636764160,"available":96607584256,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,4,5,7,8,9,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247838,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32660,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1511424,"data_stored":1133143,"data_compressed":215570,"data_compressed_allocated":434176,"data_compressed_original":868352,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T03:40:41.042 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-04-26T03:40:08.073132+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=3, tm_min=40, tm_sec=8, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=3, tm_min=40, tm_sec=32, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T03:40:41.043 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-04-26T03:40:07.106612+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=3, tm_min=40, tm_sec=7, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=3, tm_min=40, tm_sec=32, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T03:40:41.043 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-04-26T03:39:45.538842+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=3, tm_min=39, tm_sec=45, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=3, tm_min=40, tm_sec=32, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T03:40:41.043 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-04-26T03:40:01.434355+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=3, tm_min=40, tm_sec=1, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=3, tm_min=40, tm_sec=32, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T03:40:41.043 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-26T03:39:54.682516+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=3, tm_min=39, tm_sec=54, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=3, tm_min=40, tm_sec=32, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T03:40:41.043 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-04-26T03:39:48.469598+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=3, tm_min=39, tm_sec=48, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=3, tm_min=40, tm_sec=32, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T03:40:41.043 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-26T03:39:42.110300+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=3, tm_min=39, tm_sec=42, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=3, tm_min=40, tm_sec=32, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T03:40:41.043 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-26T03:41:01.044 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T03:41:01.353 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T03:41:01.353 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-04-26T03:41:01.365 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2024-04-26T03:41:00.080555+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":13,"kb":1509949440,"kb_used":586844,"kb_used_data":25448,"kb_used_omap":24,"kb_used_meta":561127,"kb_avail":1509362596,"statfs":{"total":1546188226560,"available":1545587298304,"internally_reserved":0,"allocated":26058752,"data_stored":20581056,"data_compressed":3757932,"data_compressed_allocated":7602176,"data_compressed_original":16097280,"omap_allocated":25522,"internal_metadata":574594126},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.936006"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":247,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-26T03:40:51.739694+0000","last_change":"2024-04-26T03:40:41.116361+0000","last_active":"2024-04-26T03:40:51.739694+0000","last_peered":"2024-04-26T03:40:51.739694+0000","last_clean":"2024-04-26T03:40:51.739694+0000","last_became_active":"2024-04-26T03:38:37.358910+0000","last_became_peered":"2024-04-26T03:38:37.358910+0000","last_unstale":"2024-04-26T03:40:51.739694+0000","last_undegraded":"2024-04-26T03:40:51.739694+0000","last_fullsized":"2024-04-26T03:40:51.739694+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:40:41.116252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:40:41.116252+0000","last_clean_scrub_stamp":"2024-04-26T03:40:41.116252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:43.906570+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":245,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-26T03:40:51.145774+0000","last_change":"2024-04-26T03:40:35.366911+0000","last_active":"2024-04-26T03:40:51.145774+0000","last_peered":"2024-04-26T03:40:51.145774+0000","last_clean":"2024-04-26T03:40:51.145774+0000","last_became_active":"2024-04-26T03:38:36.484388+0000","last_became_peered":"2024-04-26T03:38:36.484388+0000","last_unstale":"2024-04-26T03:40:51.145774+0000","last_undegraded":"2024-04-26T03:40:51.145774+0000","last_fullsized":"2024-04-26T03:40:51.145774+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:40:35.366825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:40:35.366825+0000","last_clean_scrub_stamp":"2024-04-26T03:40:35.366825+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:38.637072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,13],"acting":[8,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":245,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-26T03:40:51.739777+0000","last_change":"2024-04-26T03:40:37.371723+0000","last_active":"2024-04-26T03:40:51.739777+0000","last_peered":"2024-04-26T03:40:51.739777+0000","last_clean":"2024-04-26T03:40:51.739777+0000","last_became_active":"2024-04-26T03:38:38.187383+0000","last_became_peered":"2024-04-26T03:38:38.187383+0000","last_unstale":"2024-04-26T03:40:51.739777+0000","last_undegraded":"2024-04-26T03:40:51.739777+0000","last_fullsized":"2024-04-26T03:40:51.739777+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:40:37.371576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:40:37.371576+0000","last_clean_scrub_stamp":"2024-04-26T03:40:37.371576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:45.567695+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4],"acting":[9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":245,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-26T03:40:51.587099+0000","last_change":"2024-04-26T03:40:35.098976+0000","last_active":"2024-04-26T03:40:51.587099+0000","last_peered":"2024-04-26T03:40:51.587099+0000","last_clean":"2024-04-26T03:40:51.587099+0000","last_became_active":"2024-04-26T03:38:37.255066+0000","last_became_peered":"2024-04-26T03:38:37.255066+0000","last_unstale":"2024-04-26T03:40:51.587099+0000","last_undegraded":"2024-04-26T03:40:51.587099+0000","last_fullsized":"2024-04-26T03:40:51.587099+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:40:35.098832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:40:35.098832+0000","last_clean_scrub_stamp":"2024-04-26T03:40:35.098832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:38.838380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":233,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-26T03:40:51.739422+0000","last_change":"2024-04-26T03:40:40.406898+0000","last_active":"2024-04-26T03:40:51.739422+0000","last_peered":"2024-04-26T03:40:51.739422+0000","last_clean":"2024-04-26T03:40:51.739422+0000","last_became_active":"2024-04-26T03:38:48.475155+0000","last_became_peered":"2024-04-26T03:38:48.475155+0000","last_unstale":"2024-04-26T03:40:51.739422+0000","last_undegraded":"2024-04-26T03:40:51.739422+0000","last_fullsized":"2024-04-26T03:40:51.739422+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:40:40.406792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:40:40.406792+0000","last_clean_scrub_stamp":"2024-04-26T03:40:40.406792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:51.979432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,12],"acting":[11,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":245,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-26T03:40:51.234388+0000","last_change":"2024-04-26T03:40:40.436584+0000","last_active":"2024-04-26T03:40:51.234388+0000","last_peered":"2024-04-26T03:40:51.234388+0000","last_clean":"2024-04-26T03:40:51.234388+0000","last_became_active":"2024-04-26T03:38:37.255181+0000","last_became_peered":"2024-04-26T03:38:37.255181+0000","last_unstale":"2024-04-26T03:40:51.234388+0000","last_undegraded":"2024-04-26T03:40:51.234388+0000","last_fullsized":"2024-04-26T03:40:51.234388+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:40:40.436466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:40:40.436466+0000","last_clean_scrub_stamp":"2024-04-26T03:40:40.436466+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:44.738949+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0],"acting":[14,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":246,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-26T03:40:59.257960+0000","last_change":"2024-04-26T03:40:40.562611+0000","last_active":"2024-04-26T03:40:59.257960+0000","last_peered":"2024-04-26T03:40:59.257960+0000","last_clean":"2024-04-26T03:40:59.257960+0000","last_became_active":"2024-04-26T03:38:36.431121+0000","last_became_peered":"2024-04-26T03:38:36.431121+0000","last_unstale":"2024-04-26T03:40:59.257960+0000","last_undegraded":"2024-04-26T03:40:59.257960+0000","last_fullsized":"2024-04-26T03:40:59.257960+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:40:40.562473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:40:40.562473+0000","last_clean_scrub_stamp":"2024-04-26T03:40:40.562473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:52.663120+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":245,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-26T03:40:51.234391+0000","last_change":"2024-04-26T03:40:39.452185+0000","last_active":"2024-04-26T03:40:51.234391+0000","last_peered":"2024-04-26T03:40:51.234391+0000","last_clean":"2024-04-26T03:40:51.234391+0000","last_became_active":"2024-04-26T03:38:36.431449+0000","last_became_peered":"2024-04-26T03:38:36.431449+0000","last_unstale":"2024-04-26T03:40:51.234391+0000","last_undegraded":"2024-04-26T03:40:51.234391+0000","last_fullsized":"2024-04-26T03:40:51.234391+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T03:40:39.452075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T03:40:39.452075+0000","last_clean_scrub_stamp":"2024-04-26T03:40:39.452075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:56.638696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"130'159","reported_seq":447,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-26T03:40:51.146877+0000","last_change":"2024-04-26T03:40:38.656674+0000","last_active":"2024-04-26T03:40:51.146877+0000","last_peered":"2024-04-26T03:40:51.146877+0000","last_clean":"2024-04-26T03:40:51.146877+0000","last_became_active":"2024-04-26T03:38:34.389099+0000","last_became_peered":"2024-04-26T03:38:34.389099+0000","last_unstale":"2024-04-26T03:40:51.146877+0000","last_undegraded":"2024-04-26T03:40:51.146877+0000","last_fullsized":"2024-04-26T03:40:51.146877+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"130'159","last_scrub_stamp":"2024-04-26T03:40:38.656559+0000","last_deep_scrub":"130'159","last_deep_scrub_stamp":"2024-04-26T03:40:38.656559+0000","last_clean_scrub_stamp":"2024-04-26T03:40:38.656559+0000","objects_scrubbed":2,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:42:05.197692+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":13852,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":24,"seq":103079215139,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36976,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334864,"statfs":{"total":96636764160,"available":96598900736,"internally_reserved":0,"allocated":1675264,"data_stored":1749128,"data_compressed":240931,"data_compressed_allocated":503808,"data_compressed_original":1454080,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,4,6,7,8,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215139,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32768,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339072,"statfs":{"total":96636764160,"available":96603209728,"internally_reserved":0,"allocated":1622016,"data_stored":1220200,"data_compressed":234005,"data_compressed_allocated":471040,"data_compressed_original":942080,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,4,5,6,7,9,10,11,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":23,"seq":98784247845,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36928,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334912,"statfs":{"total":96636764160,"available":96598949888,"internally_reserved":0,"allocated":1622016,"data_stored":1220200,"data_compressed":234005,"data_compressed_allocated":471040,"data_compressed_original":942080,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,7,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280545,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32816,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339024,"statfs":{"total":96636764160,"available":96603160576,"internally_reserved":0,"allocated":1675264,"data_stored":1749128,"data_compressed":240931,"data_compressed_allocated":503808,"data_compressed_original":1454080,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,4,5,7,8,9,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280545,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32768,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339072,"statfs":{"total":96636764160,"available":96603209728,"internally_reserved":0,"allocated":1622016,"data_stored":1220200,"data_compressed":234005,"data_compressed_allocated":471040,"data_compressed_original":942080,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,3,4,5,6,8,9,10,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":22,"seq":94489280546,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32768,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339072,"statfs":{"total":96636764160,"available":96603209728,"internally_reserved":0,"allocated":1622016,"data_stored":1220200,"data_compressed":234005,"data_compressed_allocated":471040,"data_compressed_original":942080,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,4,5,6,7,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280544,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32768,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339072,"statfs":{"total":96636764160,"available":96603209728,"internally_reserved":0,"allocated":1622016,"data_stored":1220200,"data_compressed":234005,"data_compressed_allocated":471040,"data_compressed_original":942080,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,2,4,7,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280545,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32768,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339072,"statfs":{"total":96636764160,"available":96603209728,"internally_reserved":0,"allocated":1622016,"data_stored":1220200,"data_compressed":234005,"data_compressed_allocated":471040,"data_compressed_original":942080,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[1,2,3,4,7,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313250,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50816,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321024,"statfs":{"total":96636764160,"available":96584728576,"internally_reserved":0,"allocated":1622016,"data_stored":1220200,"data_compressed":234005,"data_compressed_allocated":471040,"data_compressed_original":942080,"omap_allocated":1596,"internal_metadata":50395588},"hb_peers":[4,5,6,7,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":22,"seq":94489280544,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28608,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343232,"statfs":{"total":96636764160,"available":96607469568,"internally_reserved":0,"allocated":1622016,"data_stored":1220200,"data_compressed":234005,"data_compressed_allocated":471040,"data_compressed_original":942080,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,7,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280544,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32768,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339072,"statfs":{"total":96636764160,"available":96603209728,"internally_reserved":0,"allocated":1622016,"data_stored":1220200,"data_compressed":234005,"data_compressed_allocated":471040,"data_compressed_original":942080,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[4,6,7,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247842,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54976,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316864,"statfs":{"total":96636764160,"available":96580468736,"internally_reserved":0,"allocated":1622016,"data_stored":1220200,"data_compressed":234005,"data_compressed_allocated":471040,"data_compressed_original":942080,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,6,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280544,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32768,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339072,"statfs":{"total":96636764160,"available":96603209728,"internally_reserved":0,"allocated":1622016,"data_stored":1220200,"data_compressed":234005,"data_compressed_allocated":471040,"data_compressed_original":942080,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,3,5,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":23,"seq":98784247844,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":54976,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316864,"statfs":{"total":96636764160,"available":96580468736,"internally_reserved":0,"allocated":1622016,"data_stored":1220200,"data_compressed":234005,"data_compressed_allocated":471040,"data_compressed_original":942080,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,4,6,7,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280544,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28608,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343232,"statfs":{"total":96636764160,"available":96607469568,"internally_reserved":0,"allocated":1622016,"data_stored":1220200,"data_compressed":234005,"data_compressed_allocated":471040,"data_compressed_original":942080,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,4,5,7,8,9,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247842,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32764,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339076,"statfs":{"total":96636764160,"available":96603213824,"internally_reserved":0,"allocated":1622016,"data_stored":1220200,"data_compressed":234005,"data_compressed_allocated":471040,"data_compressed_original":942080,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6926,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T03:41:01.367 DEBUG:teuthology.orchestra.run.smithi028:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-26T03:41:01.678 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-26T03:41:01.679 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-26T03:41:01.679 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-26T03:41:01.679 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T03:41:01.715 INFO:tasks.ceph.osd.0:Stopped 2024-04-26T03:41:01.715 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-26T03:41:01.715 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T03:41:01.800 INFO:tasks.ceph.osd.4:Stopped 2024-04-26T03:41:01.801 DEBUG:tasks.ceph.osd.8:waiting for process to exit 2024-04-26T03:41:01.801 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T03:41:01.839 INFO:tasks.ceph.osd.8:Stopped 2024-04-26T03:41:01.839 DEBUG:tasks.ceph.osd.12:waiting for process to exit 2024-04-26T03:41:01.839 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T03:41:01.908 INFO:tasks.ceph.osd.12:Stopped 2024-04-26T03:41:01.908 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-26T03:41:01.908 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T03:41:01.947 INFO:tasks.ceph.osd.1:Stopped 2024-04-26T03:41:01.947 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-26T03:41:01.947 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T03:41:02.022 INFO:tasks.ceph.osd.5:Stopped 2024-04-26T03:41:02.022 DEBUG:tasks.ceph.osd.9:waiting for process to exit 2024-04-26T03:41:02.022 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T03:41:02.096 INFO:tasks.ceph.osd.9:Stopped 2024-04-26T03:41:02.097 DEBUG:tasks.ceph.osd.13:waiting for process to exit 2024-04-26T03:41:02.097 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T03:41:02.167 INFO:tasks.ceph.osd.13:Stopped 2024-04-26T03:41:02.168 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-26T03:41:02.168 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T03:41:02.197 INFO:tasks.ceph.osd.2:Stopped 2024-04-26T03:41:02.197 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-26T03:41:02.197 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T03:41:02.277 INFO:tasks.ceph.osd.6:Stopped 2024-04-26T03:41:02.277 DEBUG:tasks.ceph.osd.10:waiting for process to exit 2024-04-26T03:41:02.277 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T03:41:02.356 INFO:tasks.ceph.osd.10:Stopped 2024-04-26T03:41:02.357 DEBUG:tasks.ceph.osd.14:waiting for process to exit 2024-04-26T03:41:02.357 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T03:41:02.431 INFO:tasks.ceph.osd.14:Stopped 2024-04-26T03:41:02.431 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-26T03:41:02.431 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T03:41:02.464 INFO:tasks.ceph.osd.3:Stopped 2024-04-26T03:41:02.464 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-26T03:41:02.464 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T03:41:02.536 INFO:tasks.ceph.osd.7:Stopped 2024-04-26T03:41:02.537 DEBUG:tasks.ceph.osd.11:waiting for process to exit 2024-04-26T03:41:02.537 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T03:41:02.605 INFO:tasks.ceph.osd.11:Stopped 2024-04-26T03:41:02.605 DEBUG:tasks.ceph.osd.15:waiting for process to exit 2024-04-26T03:41:02.606 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T03:41:02.673 INFO:tasks.ceph.osd.15:Stopped 2024-04-26T03:41:02.673 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-26T03:41:02.673 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-26T03:41:02.674 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T03:41:02.699 INFO:tasks.ceph.mgr.y:Stopped 2024-04-26T03:41:02.700 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-26T03:41:02.700 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T03:41:02.746 INFO:tasks.ceph.mgr.x:Stopped 2024-04-26T03:41:02.747 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-26T03:41:02.748 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-26T03:41:02.749 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T03:41:02.760 INFO:tasks.ceph.mon.a:Stopped 2024-04-26T03:41:02.761 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-26T03:41:02.761 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T03:41:02.774 INFO:tasks.ceph.mon.b:Stopped 2024-04-26T03:41:02.774 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-26T03:41:02.774 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T03:41:02.789 INFO:tasks.ceph.mon.c:Stopped 2024-04-26T03:41:02.789 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-26T03:41:02.789 DEBUG:teuthology.orchestra.run.smithi028:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'slow request' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | head -n 1 2024-04-26T03:41:02.838 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi028.front.sepia.ceph.com 2024-04-26T03:41:02.838 DEBUG:teuthology.orchestra.run.smithi028:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-26T03:41:03.030 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi028.front.sepia.ceph.com 2024-04-26T03:41:03.030 DEBUG:teuthology.orchestra.run.smithi028:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-26T03:41:03.111 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-8 on ubuntu@smithi028.front.sepia.ceph.com 2024-04-26T03:41:03.111 DEBUG:teuthology.orchestra.run.smithi028:> sync && sudo umount -f /var/lib/ceph/osd/ceph-8 2024-04-26T03:41:03.178 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-12 on ubuntu@smithi028.front.sepia.ceph.com 2024-04-26T03:41:03.178 DEBUG:teuthology.orchestra.run.smithi028:> sync && sudo umount -f /var/lib/ceph/osd/ceph-12 2024-04-26T03:41:03.236 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi042.front.sepia.ceph.com 2024-04-26T03:41:03.236 DEBUG:teuthology.orchestra.run.smithi042:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-26T03:41:03.378 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi042.front.sepia.ceph.com 2024-04-26T03:41:03.378 DEBUG:teuthology.orchestra.run.smithi042:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-26T03:41:03.439 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-9 on ubuntu@smithi042.front.sepia.ceph.com 2024-04-26T03:41:03.439 DEBUG:teuthology.orchestra.run.smithi042:> sync && sudo umount -f /var/lib/ceph/osd/ceph-9 2024-04-26T03:41:03.490 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-13 on ubuntu@smithi042.front.sepia.ceph.com 2024-04-26T03:41:03.490 DEBUG:teuthology.orchestra.run.smithi042:> sync && sudo umount -f /var/lib/ceph/osd/ceph-13 2024-04-26T03:41:03.555 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi066.front.sepia.ceph.com 2024-04-26T03:41:03.555 DEBUG:teuthology.orchestra.run.smithi066:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-26T03:41:03.680 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi066.front.sepia.ceph.com 2024-04-26T03:41:03.680 DEBUG:teuthology.orchestra.run.smithi066:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-26T03:41:03.761 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-10 on ubuntu@smithi066.front.sepia.ceph.com 2024-04-26T03:41:03.761 DEBUG:teuthology.orchestra.run.smithi066:> sync && sudo umount -f /var/lib/ceph/osd/ceph-10 2024-04-26T03:41:03.820 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-14 on ubuntu@smithi066.front.sepia.ceph.com 2024-04-26T03:41:03.820 DEBUG:teuthology.orchestra.run.smithi066:> sync && sudo umount -f /var/lib/ceph/osd/ceph-14 2024-04-26T03:41:03.887 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi092.front.sepia.ceph.com 2024-04-26T03:41:03.887 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-26T03:41:04.207 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi092.front.sepia.ceph.com 2024-04-26T03:41:04.207 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-26T03:41:04.293 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-11 on ubuntu@smithi092.front.sepia.ceph.com 2024-04-26T03:41:04.293 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-11 2024-04-26T03:41:04.352 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-15 on ubuntu@smithi092.front.sepia.ceph.com 2024-04-26T03:41:04.352 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-15 2024-04-26T03:41:04.415 INFO:tasks.ceph:Archiving mon data... 2024-04-26T03:41:04.416 DEBUG:teuthology.misc:Transferring archived files from smithi028:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi/7674443/data/mon.a.tgz 2024-04-26T03:41:04.416 DEBUG:teuthology.orchestra.run.smithi028:> mktemp 2024-04-26T03:41:04.435 INFO:teuthology.orchestra.run.smithi028.stdout:/tmp/tmp.nD22uAfg8A 2024-04-26T03:41:04.435 DEBUG:teuthology.orchestra.run.smithi028:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.nD22uAfg8A 2024-04-26T03:41:04.600 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 0666 /tmp/tmp.nD22uAfg8A 2024-04-26T03:41:04.685 DEBUG:teuthology.orchestra.remote:smithi028:/tmp/tmp.nD22uAfg8A is 408KB 2024-04-26T03:41:04.784 DEBUG:teuthology.orchestra.run.smithi028:> rm -fr /tmp/tmp.nD22uAfg8A 2024-04-26T03:41:04.800 DEBUG:teuthology.misc:Transferring archived files from smithi042:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi/7674443/data/mon.b.tgz 2024-04-26T03:41:04.801 DEBUG:teuthology.orchestra.run.smithi042:> mktemp 2024-04-26T03:41:04.819 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/tmp.Dbz7vT8pko 2024-04-26T03:41:04.820 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.Dbz7vT8pko 2024-04-26T03:41:05.018 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 0666 /tmp/tmp.Dbz7vT8pko 2024-04-26T03:41:05.101 DEBUG:teuthology.orchestra.remote:smithi042:/tmp/tmp.Dbz7vT8pko is 579KB 2024-04-26T03:41:05.204 DEBUG:teuthology.orchestra.run.smithi042:> rm -fr /tmp/tmp.Dbz7vT8pko 2024-04-26T03:41:05.219 DEBUG:teuthology.misc:Transferring archived files from smithi066:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi/7674443/data/mon.c.tgz 2024-04-26T03:41:05.219 DEBUG:teuthology.orchestra.run.smithi066:> mktemp 2024-04-26T03:41:05.238 INFO:teuthology.orchestra.run.smithi066.stdout:/tmp/tmp.QfECkAeIpp 2024-04-26T03:41:05.238 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.QfECkAeIpp 2024-04-26T03:41:05.440 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 0666 /tmp/tmp.QfECkAeIpp 2024-04-26T03:41:05.524 DEBUG:teuthology.orchestra.remote:smithi066:/tmp/tmp.QfECkAeIpp is 578KB 2024-04-26T03:41:05.624 DEBUG:teuthology.orchestra.run.smithi066:> rm -fr /tmp/tmp.QfECkAeIpp 2024-04-26T03:41:05.639 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-26T03:41:05.640 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-26T03:41:05.642 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-26T03:41:05.644 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-26T03:41:05.681 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-26T03:41:05.729 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-26T03:41:05.746 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-26T03:41:05.761 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-26T03:41:05.778 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-26T03:41:05.778 INFO:tasks.ceph:Archiving crash dumps... 2024-04-26T03:41:05.779 DEBUG:teuthology.misc:Transferring archived files from smithi028:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi/7674443/remote/smithi028/crash 2024-04-26T03:41:05.780 DEBUG:teuthology.orchestra.run.smithi028:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-26T03:41:05.808 DEBUG:teuthology.misc:Transferring archived files from smithi042:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi/7674443/remote/smithi042/crash 2024-04-26T03:41:05.809 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-26T03:41:05.838 DEBUG:teuthology.misc:Transferring archived files from smithi066:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi/7674443/remote/smithi066/crash 2024-04-26T03:41:05.840 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-26T03:41:05.866 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi/7674443/remote/smithi092/crash 2024-04-26T03:41:05.868 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-26T03:41:05.998 INFO:tasks.ceph:Compressing logs... 2024-04-26T03:41:05.999 DEBUG:teuthology.orchestra.run.smithi028:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T03:41:06.002 DEBUG:teuthology.orchestra.run.smithi042:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T03:41:06.004 DEBUG:teuthology.orchestra.run.smithi066:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T03:41:06.006 DEBUG:teuthology.orchestra.run.smithi092:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T03:41:06.028 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.13.log 2024-04-26T03:41:06.028 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-26T03:41:06.029 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-osd.13.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-26T03:41:06.029 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-26T03:41:06.029 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-26T03:41:06.029 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-26T03:41:06.029 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-osd.5.log: /var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.9.log 2024-04-26T03:41:06.029 INFO:teuthology.orchestra.run.smithi042.stderr: 85.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-26T03:41:06.030 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-osd.9.log: 89.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-26T03:41:06.036 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-26T03:41:06.036 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-26T03:41:06.036 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-26T03:41:06.036 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-26T03:41:06.036 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-26T03:41:06.036 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.10.log 2024-04-26T03:41:06.037 INFO:teuthology.orchestra.run.smithi066.stderr: 85.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-26T03:41:06.037 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.14.log 2024-04-26T03:41:06.037 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-osd.10.log: /var/log/ceph/ceph-osd.14.log: 89.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-26T03:41:06.060 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.15.log 2024-04-26T03:41:06.060 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31479.log 2024-04-26T03:41:06.061 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-osd.15.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31159.log 2024-04-26T03:41:06.061 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.31479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31719.log 2024-04-26T03:41:06.061 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31479.log.gz 2024-04-26T03:41:06.061 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.31159.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-26T03:41:06.061 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31159.log.gz 2024-04-26T03:41:06.061 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.31719.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-26T03:41:06.061 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31719.log.gz 2024-04-26T03:41:06.061 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31399.log 2024-04-26T03:41:06.061 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.35267.log 2024-04-26T03:41:06.062 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.31399.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31399.log.gz 2024-04-26T03:41:06.062 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31199.log 2024-04-26T03:41:06.062 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31519.log 2024-04-26T03:41:06.062 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.0.35267.log: /var/log/ceph/ceph-client.admin.31199.log: 76.4% -- replaced with /var/log/ceph/ceph-client.0.35267.log.gz 2024-04-26T03:41:06.062 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31199.log.gz 2024-04-26T03:41:06.062 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-26T03:41:06.062 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31439.log 2024-04-26T03:41:06.062 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.31519.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31519.log.gz 2024-04-26T03:41:06.062 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31279.log 2024-04-26T03:41:06.063 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.31439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31112.log 2024-04-26T03:41:06.063 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31439.log.gz 2024-04-26T03:41:06.063 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.31279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31559.log 2024-04-26T03:41:06.063 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31279.log.gz 2024-04-26T03:41:06.063 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.31112.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.11.log 2024-04-26T03:41:06.063 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31112.log.gz 2024-04-26T03:41:06.063 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.31559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31679.log 2024-04-26T03:41:06.063 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31559.log.gz 2024-04-26T03:41:06.064 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31359.log 2024-04-26T03:41:06.064 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.31679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31599.log 2024-04-26T03:41:06.064 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31679.log.gz 2024-04-26T03:41:06.064 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.31359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31319.log 2024-04-26T03:41:06.064 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31359.log.gz 2024-04-26T03:41:06.065 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.31599.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31599.log.gz 2024-04-26T03:41:06.065 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31639.log 2024-04-26T03:41:06.065 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.31319.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31319.log.gz 2024-04-26T03:41:06.065 INFO:teuthology.orchestra.run.smithi092.stderr: --verbose -- /var/log/ceph/ceph-client.admin.31239.log 2024-04-26T03:41:06.065 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.31639.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31639.log.gz 2024-04-26T03:41:06.065 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.31239.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31239.log.gz 2024-04-26T03:41:06.087 INFO:teuthology.orchestra.run.smithi092.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-26T03:41:06.130 INFO:teuthology.orchestra.run.smithi042.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-26T03:41:06.169 INFO:teuthology.orchestra.run.smithi066.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-26T03:41:06.173 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32111.log 2024-04-26T03:41:06.173 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40644.log 2024-04-26T03:41:06.173 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.32111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42945.log 2024-04-26T03:41:06.173 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32111.log.gz 2024-04-26T03:41:06.173 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.40644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46410.log 2024-04-26T03:41:06.173 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40644.log.gz 2024-04-26T03:41:06.173 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47180.log 2024-04-26T03:41:06.173 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42945.log.gz 2024-04-26T03:41:06.173 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45468.log 2024-04-26T03:41:06.173 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46410.log.gz 2024-04-26T03:41:06.174 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47344.log 2024-04-26T03:41:06.174 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47180.log.gz 2024-04-26T03:41:06.174 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40681.log 2024-04-26T03:41:06.174 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45468.log.gz 2024-04-26T03:41:06.174 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37591.log 2024-04-26T03:41:06.174 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47344.log.gz 2024-04-26T03:41:06.174 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.40681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36640.log 2024-04-26T03:41:06.174 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40681.log.gz 2024-04-26T03:41:06.174 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.37591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37592.log 2024-04-26T03:41:06.174 INFO:teuthology.orchestra.run.smithi028.stderr: 24.6% -- replaced with /var/log/ceph/ceph-client.admin.37591.log.gz 2024-04-26T03:41:06.174 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.36640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46542.log 2024-04-26T03:41:06.174 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36640.log.gz 2024-04-26T03:41:06.174 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.37592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36612.log 2024-04-26T03:41:06.174 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37592.log.gz 2024-04-26T03:41:06.174 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35968.log 2024-04-26T03:41:06.175 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46542.log.gz 2024-04-26T03:41:06.175 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.36612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41386.log 2024-04-26T03:41:06.175 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36612.log.gz 2024-04-26T03:41:06.175 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.35968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48198.log 2024-04-26T03:41:06.175 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35968.log.gz 2024-04-26T03:41:06.175 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.41386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35963.log 2024-04-26T03:41:06.175 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41386.log.gz 2024-04-26T03:41:06.175 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.48198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44502.log 2024-04-26T03:41:06.175 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48198.log.gz 2024-04-26T03:41:06.175 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.35963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44628.log 2024-04-26T03:41:06.175 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35963.log.gz 2024-04-26T03:41:06.175 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47257.log 2024-04-26T03:41:06.175 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44502.log.gz 2024-04-26T03:41:06.175 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44628.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.25896.log 2024-04-26T03:41:06.176 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44628.log.gz 2024-04-26T03:41:06.176 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42745.log 2024-04-26T03:41:06.176 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47257.log.gz 2024-04-26T03:41:06.176 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph.tmp-client.admin.25896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47610.log 2024-04-26T03:41:06.176 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.25896.log.gz 2024-04-26T03:41:06.176 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36596.log 2024-04-26T03:41:06.176 INFO:teuthology.orchestra.run.smithi028.stderr: 24.2% -- replaced with /var/log/ceph/ceph-client.admin.42745.log.gz 2024-04-26T03:41:06.176 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36381.log 2024-04-26T03:41:06.176 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47610.log.gz 2024-04-26T03:41:06.176 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.36596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44249.log 2024-04-26T03:41:06.176 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36596.log.gz 2024-04-26T03:41:06.176 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.36381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45300.log 2024-04-26T03:41:06.176 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36381.log.gz 2024-04-26T03:41:06.176 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35936.log 0.0% 2024-04-26T03:41:06.176 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44249.log.gz 2024-04-26T03:41:06.177 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46566.log 2024-04-26T03:41:06.177 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45300.log.gz 2024-04-26T03:41:06.177 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.35936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48996.log 2024-04-26T03:41:06.177 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35936.log.gz 2024-04-26T03:41:06.177 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43743.log 2024-04-26T03:41:06.177 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46566.log.gz 2024-04-26T03:41:06.177 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.48996.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.38198.log 2024-04-26T03:41:06.177 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48996.log.gz 2024-04-26T03:41:06.177 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.43743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46353.log 2024-04-26T03:41:06.177 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43743.log.gz 2024-04-26T03:41:06.177 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.38198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40602.log 2024-04-26T03:41:06.177 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38198.log.gz 2024-04-26T03:41:06.177 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46353.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-26T03:41:06.177 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46353.log.gz 2024-04-26T03:41:06.177 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.40602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48492.log 2024-04-26T03:41:06.177 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40602.log.gz 2024-04-26T03:41:06.178 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48450.log 2024-04-26T03:41:06.178 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.48492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41743.log 2024-04-26T03:41:06.178 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48492.log.gz 2024-04-26T03:41:06.178 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.48450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45884.log 2024-04-26T03:41:06.178 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48450.log.gz 2024-04-26T03:41:06.178 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.41743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45132.log 2024-04-26T03:41:06.178 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41743.log.gz 2024-04-26T03:41:06.178 INFO:teuthology.orchestra.run.smithi028.stderr: 85.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-26T03:41:06.178 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48156.log 2024-04-26T03:41:06.178 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45884.log.gz 2024-04-26T03:41:06.178 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46522.log 2024-04-26T03:41:06.178 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45132.log.gz 2024-04-26T03:41:06.178 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.48156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45859.log 2024-04-26T03:41:06.178 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48156.log.gz 2024-04-26T03:41:06.179 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37138.log 2024-04-26T03:41:06.179 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46522.log.gz 2024-04-26T03:41:06.179 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5/var/log/ceph/ceph-client.admin.45859.log: --verbose -- /var/log/ceph/ceph-client.admin.39421.log 2024-04-26T03:41:06.179 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45859.log.gz 2024-04-26T03:41:06.179 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.37138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38909.log 2024-04-26T03:41:06.179 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37138.log.gz 2024-04-26T03:41:06.179 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.39421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36597.log 2024-04-26T03:41:06.179 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39421.log.gz 2024-04-26T03:41:06.179 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.38909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40560.log 2024-04-26T03:41:06.179 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38909.log.gz 2024-04-26T03:41:06.179 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.36597.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36597.log.gz -5 2024-04-26T03:41:06.179 INFO:teuthology.orchestra.run.smithi028.stderr: --verbose -- /var/log/ceph/ceph-client.admin.44291.log 2024-04-26T03:41:06.179 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.40560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47736.log 2024-04-26T03:41:06.179 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40560.log.gz 2024-04-26T03:41:06.180 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43666.log 2024-04-26T03:41:06.180 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44291.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44291.log.gz 2024-04-26T03:41:06.180 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38710.log 2024-04-26T03:41:06.180 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47736.log.gz 2024-04-26T03:41:06.180 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.43666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47652.log 2024-04-26T03:41:06.180 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43666.log.gz 2024-04-26T03:41:06.180 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.38710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48324.log 2024-04-26T03:41:06.180 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38710.log.gz 2024-04-26T03:41:06.180 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41867.log 2024-04-26T03:41:06.180 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47652.log.gz 2024-04-26T03:41:06.180 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.48324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47778.log 2024-04-26T03:41:06.180 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48324.log.gz 2024-04-26T03:41:06.180 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.41867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42044.log 2024-04-26T03:41:06.180 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41867.log.gz 2024-04-26T03:41:06.180 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45006.log 2024-04-26T03:41:06.181 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47778.log.gz 2024-04-26T03:41:06.181 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43866.log 2024-04-26T03:41:06.181 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42044.log.gz 2024-04-26T03:41:06.181 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45888.log 2024-04-26T03:41:06.181 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45006.log.gz 2024-04-26T03:41:06.181 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.43866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46406.log 2024-04-26T03:41:06.181 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43866.log.gz 2024-04-26T03:41:06.181 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36570.log 2024-04-26T03:41:06.181 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45888.log.gz 2024-04-26T03:41:06.181 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46406.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-26T03:41:06.181 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46406.log.gz 2024-04-26T03:41:06.181 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.36570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44964.log 2024-04-26T03:41:06.181 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36570.log.gz 2024-04-26T03:41:06.181 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46520.log 2024-04-26T03:41:06.181 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45877.log 2024-04-26T03:41:06.182 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44964.log.gz 2024-04-26T03:41:06.182 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35958.log 2024-04-26T03:41:06.182 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46520.log.gz 2024-04-26T03:41:06.182 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45342.log 2024-04-26T03:41:06.182 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45877.log.gz 2024-04-26T03:41:06.182 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.35958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43095.log 2024-04-26T03:41:06.182 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35958.log.gz 2024-04-26T03:41:06.182 INFO:teuthology.orchestra.run.smithi028.stderr: 89.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-26T03:41:06.182 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48576.log 2024-04-26T03:41:06.182 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45342.log.gz 2024-04-26T03:41:06.182 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.43095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44880.log 2024-04-26T03:41:06.182 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43095.log.gz 2024-04-26T03:41:06.182 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.48576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39284.log 2024-04-26T03:41:06.182 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48576.log.gz 2024-04-26T03:41:06.182 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43224.log 2024-04-26T03:41:06.183 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44880.log.gz 2024-04-26T03:41:06.183 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.39284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37916.log 2024-04-26T03:41:06.183 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39284.log.gz 2024-04-26T03:41:06.183 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.43224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35418.log 2024-04-26T03:41:06.183 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43224.log.gz 2024-04-26T03:41:06.183 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.37916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36620.log 2024-04-26T03:41:06.183 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37916.log.gz 2024-04-26T03:41:06.183 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.35418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46562.log 2024-04-26T03:41:06.183 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35418.log.gz 2024-04-26T03:41:06.183 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.36620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43948.log 2024-04-26T03:41:06.183 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36620.log.gz 2024-04-26T03:41:06.183 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46938.log 2024-04-26T03:41:06.183 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46562.log.gz 2024-04-26T03:41:06.183 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.43948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41982.log 2024-04-26T03:41:06.183 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43948.log.gz 2024-04-26T03:41:06.184 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46938.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.47988.log -- replaced with /var/log/ceph/ceph-client.admin.46938.log.gz 2024-04-26T03:41:06.184 INFO:teuthology.orchestra.run.smithi028.stderr: 2024-04-26T03:41:06.184 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.41982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40090.log 2024-04-26T03:41:06.184 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41982.log.gz 2024-04-26T03:41:06.184 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40285.log 2024-04-26T03:41:06.184 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47988.log.gz 2024-04-26T03:41:06.184 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.40090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39264.log 2024-04-26T03:41:06.184 INFO:teuthology.orchestra.run.smithi028.stderr: 28.5% -- replaced with /var/log/ceph/ceph-client.admin.40090.log.gz 2024-04-26T03:41:06.184 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.40285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44375.log 2024-04-26T03:41:06.184 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40285.log.gz 2024-04-26T03:41:06.184 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.39264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43707.log 2024-04-26T03:41:06.185 INFO:teuthology.orchestra.run.smithi028.stderr: 24.2% -- replaced with /var/log/ceph/ceph-client.admin.39264.log.gz 2024-04-26T03:41:06.185 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36608.log 2024-04-26T03:41:06.185 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44375.log.gz 2024-04-26T03:41:06.185 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.43707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41908.log 2024-04-26T03:41:06.185 INFO:teuthology.orchestra.run.smithi028.stderr: 24.1% -- replaced with /var/log/ceph/ceph-client.admin.43707.log.gz 2024-04-26T03:41:06.185 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.36608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41763.log 2024-04-26T03:41:06.185 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36608.log.gz 2024-04-26T03:41:06.185 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.41908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49288.log 2024-04-26T03:41:06.185 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41908.log.gz 2024-04-26T03:41:06.185 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.41763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40000.log 2024-04-26T03:41:06.185 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41763.log.gz 2024-04-26T03:41:06.185 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.49288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44333.log 2024-04-26T03:41:06.185 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49288.log.gz 2024-04-26T03:41:06.185 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.40000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45090.log 2024-04-26T03:41:06.186 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40000.log.gz 2024-04-26T03:41:06.186 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39461.log 2024-04-26T03:41:06.186 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44333.log.gz 2024-04-26T03:41:06.186 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41239.log 2024-04-26T03:41:06.186 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45090.log.gz 2024-04-26T03:41:06.186 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.39461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35542.log 2024-04-26T03:41:06.186 INFO:teuthology.orchestra.run.smithi028.stderr: 23.9% -- replaced with /var/log/ceph/ceph-client.admin.39461.log.gz 2024-04-26T03:41:06.186 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.41239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45510.log 2024-04-26T03:41:06.186 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41239.log.gz 2024-04-26T03:41:06.186 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.35542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40132.log 2024-04-26T03:41:06.186 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35542.log.gz 2024-04-26T03:41:06.186 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47820.log 2024-04-26T03:41:06.186 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45510.log.gz 2024-04-26T03:41:06.186 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.40132.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-26T03:41:06.187 INFO:teuthology.orchestra.run.smithi028.stderr: 24.3% -- replaced with /var/log/ceph/ceph-client.admin.40132.log.gz 2024-04-26T03:41:06.187 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37390.log 2024-04-26T03:41:06.187 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47820.log.gz 2024-04-26T03:41:06.187 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45874.log 2024-04-26T03:41:06.187 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.37390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44796.log 2024-04-26T03:41:06.187 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37390.log.gz 2024-04-26T03:41:06.187 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43059.log 2024-04-26T03:41:06.187 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45874.log.gz 2024-04-26T03:41:06.187 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44796.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.12.log 2024-04-26T03:41:06.187 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44796.log.gz 2024-04-26T03:41:06.187 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.43059.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.43059.log.gz /var/log/ceph/ceph-client.admin.46543.log 2024-04-26T03:41:06.187 INFO:teuthology.orchestra.run.smithi028.stderr: 2024-04-26T03:41:06.187 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-osd.12.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46959.log 2024-04-26T03:41:06.187 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46455.log 2024-04-26T03:41:06.187 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46543.log.gz 2024-04-26T03:41:06.188 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40048.log 2024-04-26T03:41:06.188 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46959.log.gz 2024-04-26T03:41:06.188 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46455.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46455.log.gz 2024-04-26T03:41:06.188 INFO:teuthology.orchestra.run.smithi028.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.35476.log 2024-04-26T03:41:06.188 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.40048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37593.log 2024-04-26T03:41:06.188 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40048.log.gz 2024-04-26T03:41:06.188 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.35476.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.35476.log.gz 2024-04-26T03:41:06.188 INFO:teuthology.orchestra.run.smithi028.stderr: -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-26T03:41:06.188 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.37593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37012.log 2024-04-26T03:41:06.188 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37593.log.gz 2024-04-26T03:41:06.188 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48408.log 2024-04-26T03:41:06.188 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.37012.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37012.log.gzgzip 2024-04-26T03:41:06.188 INFO:teuthology.orchestra.run.smithi028.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.37054.log 2024-04-26T03:41:06.188 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42364.log 2024-04-26T03:41:06.188 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.48408.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48408.log.gz 2024-04-26T03:41:06.189 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.37054.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37054.log.gz -5 2024-04-26T03:41:06.189 INFO:teuthology.orchestra.run.smithi028.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43992.log 2024-04-26T03:41:06.189 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35957.log 2024-04-26T03:41:06.189 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42364.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42364.log.gz 2024-04-26T03:41:06.189 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.43992.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43992.log.gz 2024-04-26T03:41:06.189 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35935.log 2024-04-26T03:41:06.189 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-26T03:41:06.189 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.35957.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35957.log.gz 2024-04-26T03:41:06.189 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.35935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35956.log 2024-04-26T03:41:06.189 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35935.log.gz 2024-04-26T03:41:06.189 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42163.log 2024-04-26T03:41:06.189 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.35956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42830.log 2024-04-26T03:41:06.189 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35956.log.gz 2024-04-26T03:41:06.189 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35362.log 2024-04-26T03:41:06.189 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42163.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42163.log.gz 2024-04-26T03:41:06.190 INFO:teuthology.orchestra.run.smithi028.stderr: 92.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-26T03:41:06.190 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44460.log 2024-04-26T03:41:06.190 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42830.log.gz 2024-04-26T03:41:06.190 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40426.log 2024-04-26T03:41:06.190 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.35362.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35362.log.gz 2024-04-26T03:41:06.190 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44460.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.44460.log.gz --verbose 2024-04-26T03:41:06.190 INFO:teuthology.orchestra.run.smithi028.stderr: -- /var/log/ceph/ceph-client.admin.41621.log 2024-04-26T03:41:06.190 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.40426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48744.log 2024-04-26T03:41:06.190 INFO:teuthology.orchestra.run.smithi028.stderr: 23.2% -- replaced with /var/log/ceph/ceph-client.admin.40426.log.gz 2024-04-26T03:41:06.190 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.41621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47263.log 2024-04-26T03:41:06.190 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41621.log.gz 2024-04-26T03:41:06.190 INFO:teuthology.orchestra.run.smithi028.stderr:gzip/var/log/ceph/ceph-client.admin.48744.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.44922.log 2024-04-26T03:41:06.190 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48744.log.gz 2024-04-26T03:41:06.190 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47263.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.47263.log.gz --verbose 2024-04-26T03:41:06.191 INFO:teuthology.orchestra.run.smithi028.stderr: -- /var/log/ceph/ceph-client.admin.41468.log 2024-04-26T03:41:06.191 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48912.log 2024-04-26T03:41:06.191 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44922.log.gz 2024-04-26T03:41:06.191 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.41468.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41468.log.gz 2024-04-26T03:41:06.191 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47904.log 2024-04-26T03:41:06.191 INFO:teuthology.orchestra.run.smithi028.stderr:gzip/var/log/ceph/ceph-client.admin.48912.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.40821.log 2024-04-26T03:41:06.191 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48912.log.gz 2024-04-26T03:41:06.191 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47904.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.47904.log.gz 2024-04-26T03:41:06.191 INFO:teuthology.orchestra.run.smithi028.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.45839.log 2024-04-26T03:41:06.191 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38637.log 2024-04-26T03:41:06.191 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.40821.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40821.log.gz 2024-04-26T03:41:06.191 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45893.log 2024-04-26T03:41:06.191 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45839.log.gz 2024-04-26T03:41:06.191 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.38637.log: -- /var/log/ceph/ceph-client.admin.36611.log 2024-04-26T03:41:06.191 INFO:teuthology.orchestra.run.smithi028.stderr: 23.5% -- replaced with /var/log/ceph/ceph-client.admin.38637.log.gz 2024-04-26T03:41:06.192 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47341.log 2024-04-26T03:41:06.192 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45893.log.gz 2024-04-26T03:41:06.192 INFO:teuthology.orchestra.run.smithi028.stderr:gzip/var/log/ceph/ceph-client.admin.36611.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.45873.log 2024-04-26T03:41:06.192 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36611.log.gz 2024-04-26T03:41:06.192 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47018.log 2024-04-26T03:41:06.192 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47341.log.gz 2024-04-26T03:41:06.192 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45873.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45873.log.gz 2024-04-26T03:41:06.192 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35626.log 2024-04-26T03:41:06.192 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44544.log 2024-04-26T03:41:06.192 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47018.log.gz 2024-04-26T03:41:06.192 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.35626.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35626.log.gz 2024-04-26T03:41:06.192 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42223.log 2024-04-26T03:41:06.192 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45552.log 2024-04-26T03:41:06.192 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44544.log.gz 2024-04-26T03:41:06.193 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42223.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.42223.log.gz 2024-04-26T03:41:06.193 INFO:teuthology.orchestra.run.smithi028.stderr: /var/log/ceph/ceph-client.admin.49206.log 2024-04-26T03:41:06.193 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48954.log 2024-04-26T03:41:06.193 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45552.log.gz 2024-04-26T03:41:06.193 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.49206.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.49206.log.gz 2024-04-26T03:41:06.193 INFO:teuthology.orchestra.run.smithi028.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44118.log 2024-04-26T03:41:06.193 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5/var/log/ceph/ceph-client.admin.48954.log: --verbose -- /var/log/ceph/ceph-client.admin.47694.log 2024-04-26T03:41:06.193 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48954.log.gz 2024-04-26T03:41:06.193 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45870.log 2024-04-26T03:41:06.193 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44118.log.gz 2024-04-26T03:41:06.193 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48282.log 2024-04-26T03:41:06.193 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47694.log.gz 2024-04-26T03:41:06.193 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45870.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.45870.log.gz 2024-04-26T03:41:06.193 INFO:teuthology.orchestra.run.smithi028.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.39851.log 2024-04-26T03:41:06.193 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41215.log/var/log/ceph/ceph-client.admin.48282.log: 2024-04-26T03:41:06.194 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48282.log.gz 2024-04-26T03:41:06.194 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.39851.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39851.log.gz 2024-04-26T03:41:06.194 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44076.log 2024-04-26T03:41:06.194 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.41215.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-26T03:41:06.194 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41215.log.gz 2024-04-26T03:41:06.194 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44076.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44076.log.gz 2024-04-26T03:41:06.194 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48828.log 2024-04-26T03:41:06.194 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48786.log 2024-04-26T03:41:06.194 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.48828.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.48828.log.gz -5 2024-04-26T03:41:06.194 INFO:teuthology.orchestra.run.smithi028.stderr: --verbose -- /var/log/ceph/ceph-client.admin.37804.log 2024-04-26T03:41:06.194 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.48786.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.48786.log.gz -5 2024-04-26T03:41:06.194 INFO:teuthology.orchestra.run.smithi028.stderr: --verbose -- /var/log/ceph/ceph-client.admin.38751.log 2024-04-26T03:41:06.194 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38950.log 2024-04-26T03:41:06.194 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.37804.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37804.log.gz 2024-04-26T03:41:06.194 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.38751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35934.log 2024-04-26T03:41:06.195 INFO:teuthology.orchestra.run.smithi028.stderr: 23.7% -- replaced with /var/log/ceph/ceph-client.admin.38751.log.gz 2024-04-26T03:41:06.195 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.38950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42903.log 2024-04-26T03:41:06.195 INFO:teuthology.orchestra.run.smithi028.stderr: 23.9% -- replaced with /var/log/ceph/ceph-client.admin.38950.log.gz 2024-04-26T03:41:06.195 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44165.log/var/log/ceph/ceph-client.admin.35934.log: 2024-04-26T03:41:06.195 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35934.log.gz 2024-04-26T03:41:06.195 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37096.log 2024-04-26T03:41:06.195 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42903.log.gz 2024-04-26T03:41:06.195 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47116.log 2024-04-26T03:41:06.195 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44165.log.gz 2024-04-26T03:41:06.195 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.37096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45426.log 2024-04-26T03:41:06.195 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37096.log.gz 2024-04-26T03:41:06.195 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44034.log 2024-04-26T03:41:06.195 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47116.log.gz 2024-04-26T03:41:06.195 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39379.log 2024-04-26T03:41:06.195 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45426.log.gz 2024-04-26T03:41:06.196 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48618.log 2024-04-26T03:41:06.196 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44034.log.gz 2024-04-26T03:41:06.196 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.39379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48870.log 2024-04-26T03:41:06.196 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39379.log.gz 2024-04-26T03:41:06.196 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.48618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37222.log 2024-04-26T03:41:06.196 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48618.log.gz 2024-04-26T03:41:06.196 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.48870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35955.log 2024-04-26T03:41:06.196 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48870.log.gz 2024-04-26T03:41:06.196 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.37222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41156.log 2024-04-26T03:41:06.196 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37222.log.gz 2024-04-26T03:41:06.196 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.35955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42704.log 2024-04-26T03:41:06.196 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35955.log.gz 2024-04-26T03:41:06.196 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.41156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49080.log 2024-04-26T03:41:06.196 INFO:teuthology.orchestra.run.smithi028.stderr: 23.7% -- replaced with /var/log/ceph/ceph-client.admin.41156.log.gz 2024-04-26T03:41:06.196 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40801.log 2024-04-26T03:41:06.197 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42704.log.gz 2024-04-26T03:41:06.197 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36619.log 2024-04-26T03:41:06.197 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.49080.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49080.log.gz 2024-04-26T03:41:06.197 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.40801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47327.log 2024-04-26T03:41:06.197 INFO:teuthology.orchestra.run.smithi028.stderr: 23.9% -- replaced with /var/log/ceph/ceph-client.admin.40801.log.gz 2024-04-26T03:41:06.197 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.36619.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36619.log.gz 2024-04-26T03:41:06.197 INFO:teuthology.orchestra.run.smithi028.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.39618.log 2024-04-26T03:41:06.197 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43561.log 2024-04-26T03:41:06.197 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47327.log.gz 2024-04-26T03:41:06.197 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.39618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37306.log 2024-04-26T03:41:06.197 INFO:teuthology.orchestra.run.smithi028.stderr: 23.9% -- replaced with /var/log/ceph/ceph-client.admin.39618.log.gz 2024-04-26T03:41:06.197 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.43561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47176.log 2024-04-26T03:41:06.197 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43561.log.gz 2024-04-26T03:41:06.197 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.37306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36429.log 2024-04-26T03:41:06.197 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37306.log.gz 2024-04-26T03:41:06.198 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47176.log: gzip -5 0.0% --verbose -- /var/log/ceph/ceph-client.admin.39534.log 2024-04-26T03:41:06.198 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47176.log.gz 2024-04-26T03:41:06.198 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.36429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45384.log 2024-04-26T03:41:06.198 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36429.log.gz 2024-04-26T03:41:06.198 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43824.log 2024-04-26T03:41:06.198 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.39534.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39534.log.gz 2024-04-26T03:41:06.198 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45384.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45384.log.gz 2024-04-26T03:41:06.198 INFO:teuthology.orchestra.run.smithi028.stderr: --verbose -- /var/log/ceph/ceph-client.admin.42106.log 2024-04-26T03:41:06.198 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.43824.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43824.log.gz 2024-04-26T03:41:06.198 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46514.log 2024-04-26T03:41:06.198 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42365.log 2024-04-26T03:41:06.198 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42106.log: 24.6% -- replaced with /var/log/ceph/ceph-client.admin.42106.log.gz 2024-04-26T03:41:06.198 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48072.log 2024-04-26T03:41:06.198 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46514.log.gz 2024-04-26T03:41:06.199 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42365.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42365.log.gz 2024-04-26T03:41:06.199 INFO:teuthology.orchestra.run.smithi028.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.36600.log 2024-04-26T03:41:06.199 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37180.log 2024-04-26T03:41:06.199 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.48072.log: /var/log/ceph/ceph-client.admin.36600.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48072.log.gz 2024-04-26T03:41:06.199 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36600.log.gz 2024-04-26T03:41:06.199 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45875.log 2024-04-26T03:41:06.199 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36579.log 2024-04-26T03:41:06.199 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.37180.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37180.log.gz 2024-04-26T03:41:06.199 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38240.log 2024-04-26T03:41:06.199 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45875.log.gz 2024-04-26T03:41:06.199 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.36579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38124.log 2024-04-26T03:41:06.199 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36579.log.gz 2024-04-26T03:41:06.199 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.38240.log: /var/log/ceph/ceph-client.admin.48534.log 2024-04-26T03:41:06.199 INFO:teuthology.orchestra.run.smithi028.stderr: 23.5% -- replaced with /var/log/ceph/ceph-client.admin.38240.log.gz 2024-04-26T03:41:06.200 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.38124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47862.log 2024-04-26T03:41:06.200 INFO:teuthology.orchestra.run.smithi028.stderr: 54.7% -- replaced with /var/log/ceph/ceph-client.admin.38124.log.gz 2024-04-26T03:41:06.200 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.48534.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48534.log.gz 2024-04-26T03:41:06.200 INFO:teuthology.orchestra.run.smithi028.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47946.log 2024-04-26T03:41:06.200 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44586.log 2024-04-26T03:41:06.200 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47862.log.gz 2024-04-26T03:41:06.200 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46465.log 2024-04-26T03:41:06.200 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47946.log.gz 2024-04-26T03:41:06.200 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45866.log 2024-04-26T03:41:06.200 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44586.log.gz 2024-04-26T03:41:06.200 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38083.log 2024-04-26T03:41:06.200 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46465.log.gz 2024-04-26T03:41:06.200 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45866.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45866.log.gz 2024-04-26T03:41:06.200 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45876.log 2024-04-26T03:41:06.200 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.38083.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38083.log.gz 2024-04-26T03:41:06.201 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47318.log 2024-04-26T03:41:06.201 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43459.log 2024-04-26T03:41:06.201 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45876.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45876.log.gz 2024-04-26T03:41:06.201 INFO:teuthology.orchestra.run.smithi028.stderr:gzip/var/log/ceph/ceph-client.admin.47318.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.42445.log 2024-04-26T03:41:06.201 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47318.log.gz 2024-04-26T03:41:06.201 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.43459.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.43459.log.gz -5 2024-04-26T03:41:06.201 INFO:teuthology.orchestra.run.smithi028.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48702.log 2024-04-26T03:41:06.201 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42835.log 2024-04-26T03:41:06.201 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42445.log: 24.0% -- replaced with /var/log/ceph/ceph-client.admin.42445.log.gz 2024-04-26T03:41:06.201 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.48702.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48702.log.gz 2024-04-26T03:41:06.201 INFO:teuthology.orchestra.run.smithi028.stderr: --verbose -- /var/log/ceph/ceph-client.admin.44670.log 2024-04-26T03:41:06.201 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42835.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42835.log.gz 2024-04-26T03:41:06.201 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45174.log 2024-04-26T03:41:06.201 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49248.log 2024-04-26T03:41:06.201 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44670.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44670.log.gz 2024-04-26T03:41:06.202 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45894.log 2024-04-26T03:41:06.202 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45174.log.gz 2024-04-26T03:41:06.202 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.49248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42622.log 2024-04-26T03:41:06.202 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49248.log.gz 2024-04-26T03:41:06.202 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48240.log 2024-04-26T03:41:06.202 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45894.log.gz 2024-04-26T03:41:06.202 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42622.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.8.log 2024-04-26T03:41:06.202 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42622.log.gz 2024-04-26T03:41:06.202 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.48240.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48240.log.gz 2024-04-26T03:41:06.202 INFO:teuthology.orchestra.run.smithi028.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.36396.log 2024-04-26T03:41:06.202 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44417.log 2024-04-26T03:41:06.202 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.36396.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36396.log.gz 2024-04-26T03:41:06.202 INFO:teuthology.orchestra.run.smithi028.stderr: --verbose -- /var/log/ceph/ceph-client.admin.46547.log 2024-04-26T03:41:06.202 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44417.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44417.log.gz 2024-04-26T03:41:06.202 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35960.log 2024-04-26T03:41:06.203 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46547.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46547.log.gz 2024-04-26T03:41:06.203 INFO:teuthology.orchestra.run.smithi028.stderr: --verbose -- /var/log/ceph/ceph-client.admin.45891.log 2024-04-26T03:41:06.203 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.35960.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35960.log.gz 2024-04-26T03:41:06.203 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42043.log 2024-04-26T03:41:06.203 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45891.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45891.log.gz 2024-04-26T03:41:06.203 INFO:teuthology.orchestra.run.smithi028.stderr: --verbose -- /var/log/ceph/ceph-client.admin.44207.log 2024-04-26T03:41:06.203 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42043.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42043.log.gz 2024-04-26T03:41:06.203 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45879.log 2024-04-26T03:41:06.203 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44207.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44207.log.gz 2024-04-26T03:41:06.203 INFO:teuthology.orchestra.run.smithi028.stderr: --verbose -- /var/log/ceph/ceph-client.admin.45216.log 2024-04-26T03:41:06.203 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45879.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45879.log.gz 2024-04-26T03:41:06.203 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43541.log 2024-04-26T03:41:06.203 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37348.log 2024-04-26T03:41:06.203 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45216.log.gz 2024-04-26T03:41:06.203 INFO:teuthology.orchestra.run.smithi028.stderr:gzip/var/log/ceph/ceph-client.admin.43541.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.43325.log 2024-04-26T03:41:06.204 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43541.log.gz 2024-04-26T03:41:06.204 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48366.log 2024-04-26T03:41:06.204 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.37348.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37348.log.gz 2024-04-26T03:41:06.204 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43908.log 2024-04-26T03:41:06.204 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.43325.log: 24.6% -- replaced with /var/log/ceph/ceph-client.admin.43325.log.gz 2024-04-26T03:41:06.204 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.48366.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48366.log.gz 2024-04-26T03:41:06.204 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37264.log 2024-04-26T03:41:06.204 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.43908.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43908.log.gz 2024-04-26T03:41:06.204 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35967.log 2024-04-26T03:41:06.204 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.37264.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37264.log.gz 2024-04-26T03:41:06.204 INFO:teuthology.orchestra.run.smithi028.stderr: --verbose -- /var/log/ceph/ceph-client.admin.39933.log 2024-04-26T03:41:06.204 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.35967.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35967.log.gz 2024-04-26T03:41:06.204 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38825.log 2024-04-26T03:41:06.204 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.39933.log: 24.2% -- replaced with /var/log/ceph/ceph-client.admin.39933.log.gz 2024-04-26T03:41:06.204 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47058.log 2024-04-26T03:41:06.205 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.38825.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38825.log.gz 2024-04-26T03:41:06.205 INFO:teuthology.orchestra.run.smithi028.stderr: --verbose -- /var/log/ceph/ceph-client.admin.45871.log 2024-04-26T03:41:06.205 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47058.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47058.log.gz 2024-04-26T03:41:06.205 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36442.log 2024-04-26T03:41:06.205 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45871.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45871.log.gz 2024-04-26T03:41:06.205 INFO:teuthology.orchestra.run.smithi028.stderr: --verbose -- /var/log/ceph/ceph-client.admin.35937.log 2024-04-26T03:41:06.205 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.36442.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36442.log.gz 2024-04-26T03:41:06.205 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44754.log 2024-04-26T03:41:06.205 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.35937.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35937.log.gz 2024-04-26T03:41:06.205 INFO:teuthology.orchestra.run.smithi028.stderr: --verbose -- /var/log/ceph/ceph-client.admin.38313.log 2024-04-26T03:41:06.205 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44754.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44754.log.gz 2024-04-26T03:41:06.205 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40726.log 2024-04-26T03:41:06.205 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.38313.log: gzip -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38313.log.gz 2024-04-26T03:41:06.205 INFO:teuthology.orchestra.run.smithi028.stderr: -- /var/log/ceph/ceph-client.admin.46531.log 2024-04-26T03:41:06.206 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.40726.log: 23.2% -- replaced with /var/log/ceph/ceph-client.admin.40726.log.gz 2024-04-26T03:41:06.206 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35933.log 2024-04-26T03:41:06.206 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46531.log: gzip 0.0% -5 --verbose -- /var/log/ceph/ceph-client.admin.42965.log 2024-04-26T03:41:06.206 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46531.log.gz 2024-04-26T03:41:06.206 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.35933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49038.log 2024-04-26T03:41:06.206 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35933.log.gz 2024-04-26T03:41:06.206 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31992.log 2024-04-26T03:41:06.206 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41313.log 2024-04-26T03:41:06.206 INFO:teuthology.orchestra.run.smithi028.stderr: 23.9% -- replaced with /var/log/ceph/ceph-client.admin.42965.log.gz 2024-04-26T03:41:06.206 INFO:teuthology.orchestra.run.smithi028.stderr:gzip/var/log/ceph/ceph-client.admin.31992.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.37805.log 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31992.log.gz 2024-04-26T03:41:06.206 INFO:teuthology.orchestra.run.smithi028.stderr: 2024-04-26T03:41:06.206 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.49038.log: /var/log/ceph/ceph-client.admin.41313.log: 0.0% 23.9% -- replaced with /var/log/ceph/ceph-client.admin.41313.log.gz 2024-04-26T03:41:06.206 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49038.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45048.log 2024-04-26T03:41:06.206 INFO:teuthology.orchestra.run.smithi028.stderr: 2024-04-26T03:41:06.206 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37452.log 2024-04-26T03:41:06.207 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.37805.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37805.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47117.log 2024-04-26T03:41:06.207 INFO:teuthology.orchestra.run.smithi028.stderr: 2024-04-26T03:41:06.207 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45048.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45048.log.gz 2024-04-26T03:41:06.207 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.37452.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37452.log.gz 2024-04-26T03:41:06.207 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49164.log 2024-04-26T03:41:06.207 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47117.log: gzip -5 0.0% --verbose -- /var/log/ceph/ceph-client.admin.44838.log 2024-04-26T03:41:06.207 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47117.log.gz 2024-04-26T03:41:06.207 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.49164.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47190.log 2024-04-26T03:41:06.207 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49164.log.gz 2024-04-26T03:41:06.207 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37594.log 2024-04-26T03:41:06.207 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44838.log.gz 2024-04-26T03:41:06.207 INFO:teuthology.orchestra.run.smithi028.stderr:gzip/var/log/ceph/ceph-client.admin.47190.log: -5 --verbose -- 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47190.log.gz 2024-04-26T03:41:06.207 INFO:teuthology.orchestra.run.smithi028.stderr: /var/log/ceph/ceph-client.admin.35951.log 2024-04-26T03:41:06.207 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.37594.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37594.log.gz 2024-04-26T03:41:06.207 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47118.log 2024-04-26T03:41:06.208 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.35951.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35951.log.gz 2024-04-26T03:41:06.208 INFO:teuthology.orchestra.run.smithi028.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.38867.log 2024-04-26T03:41:06.208 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47118.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35940.log 2024-04-26T03:41:06.208 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47118.log.gz 2024-04-26T03:41:06.208 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.38867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45258.log 2024-04-26T03:41:06.208 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38867.log.gz 2024-04-26T03:41:06.208 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38352.log 2024-04-26T03:41:06.208 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.35940.log: /var/log/ceph/ceph-client.admin.45258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48030.log 2024-04-26T03:41:06.208 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35940.log.gz 2024-04-26T03:41:06.208 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45258.log.gz 2024-04-26T03:41:06.208 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48114.log 2024-04-26T03:41:06.208 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.38352.log: /var/log/ceph/ceph-client.admin.48030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44712.log 2024-04-26T03:41:06.208 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38352.log.gz 2024-04-26T03:41:06.208 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48030.log.gz 2024-04-26T03:41:06.208 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.48114.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46301.log 2024-04-26T03:41:06.209 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48114.log.gz 2024-04-26T03:41:06.209 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49122.log 2024-04-26T03:41:06.209 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44712.log.gz 2024-04-26T03:41:06.209 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.46301.log: -- /var/log/ceph/ceph-client.admin.35584.log 2024-04-26T03:41:06.209 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46301.log.gz 2024-04-26T03:41:06.209 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.49122.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49122.log.gz 2024-04-26T03:41:06.209 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43624.log 2024-04-26T03:41:06.209 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.35584.log: gzip 0.0% -5 --verbose -- /var/log/ceph/ceph-client.admin.46481.log 2024-04-26T03:41:06.209 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35584.log.gz 2024-04-26T03:41:06.209 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.43624.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43624.log.gz 2024-04-26T03:41:06.209 INFO:teuthology.orchestra.run.smithi028.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48660.log 2024-04-26T03:41:06.209 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46481.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46481.log.gz 2024-04-26T03:41:06.209 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36614.log 2024-04-26T03:41:06.209 INFO:teuthology.orchestra.run.smithi028.stderr:gzip/var/log/ceph/ceph-client.admin.48660.log: -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48660.log.gz 2024-04-26T03:41:06.209 INFO:teuthology.orchestra.run.smithi028.stderr: -- /var/log/ceph/ceph-client.admin.31726.log 2024-04-26T03:41:06.210 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.36614.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36614.log.gz 2024-04-26T03:41:06.210 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35919.log 2024-04-26T03:41:06.210 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.31726.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31726.log.gz 2024-04-26T03:41:06.210 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.35919.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35919.log.gz 2024-04-26T03:41:06.327 INFO:teuthology.orchestra.run.smithi028.stderr: 91.0% -- replaced with /var/log/ceph/ceph-osd.12.log.gz 2024-04-26T03:41:06.400 INFO:teuthology.orchestra.run.smithi028.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-26T03:41:06.403 INFO:teuthology.orchestra.run.smithi042.stderr: 91.2% -- replaced with /var/log/ceph/ceph-osd.9.log.gz 2024-04-26T03:41:06.433 INFO:teuthology.orchestra.run.smithi092.stderr: 91.5% -- replaced with /var/log/ceph/ceph-osd.15.log.gz 2024-04-26T03:41:06.445 INFO:teuthology.orchestra.run.smithi092.stderr: 91.3% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-26T03:41:06.463 INFO:teuthology.orchestra.run.smithi042.stderr: 91.6% -- replaced with /var/log/ceph/ceph-osd.13.log.gz 2024-04-26T03:41:06.481 INFO:teuthology.orchestra.run.smithi066.stderr: 91.6% -- replaced with /var/log/ceph/ceph-osd.14.log.gz 2024-04-26T03:41:06.577 INFO:teuthology.orchestra.run.smithi092.stderr: 91.8% -- replaced with /var/log/ceph/ceph-osd.11.log.gz 2024-04-26T03:41:06.582 INFO:teuthology.orchestra.run.smithi028.stderr: 91.5% -- replaced with /var/log/ceph/ceph-osd.8.log.gz 2024-04-26T03:41:06.588 INFO:teuthology.orchestra.run.smithi066.stderr: 92.0% -- replaced with /var/log/ceph/ceph-osd.10.log.gz 2024-04-26T03:41:06.614 INFO:teuthology.orchestra.run.smithi066.stderr: 91.7% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-26T03:41:06.765 INFO:teuthology.orchestra.run.smithi092.stderr: 92.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-26T03:41:06.765 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-04-26T03:41:06.765 INFO:teuthology.orchestra.run.smithi092.stderr:real 0m0.739s 2024-04-26T03:41:06.765 INFO:teuthology.orchestra.run.smithi092.stderr:user 0m2.064s 2024-04-26T03:41:06.765 INFO:teuthology.orchestra.run.smithi092.stderr:sys 0m0.095s 2024-04-26T03:41:06.770 INFO:teuthology.orchestra.run.smithi066.stderr: 92.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-26T03:41:06.772 INFO:teuthology.orchestra.run.smithi066.stderr: 2024-04-26T03:41:06.772 INFO:teuthology.orchestra.run.smithi066.stderr:real 0m0.753s 2024-04-26T03:41:06.772 INFO:teuthology.orchestra.run.smithi066.stderr:user 0m2.405s 2024-04-26T03:41:06.772 INFO:teuthology.orchestra.run.smithi066.stderr:sys 0m0.090s 2024-04-26T03:41:06.850 INFO:teuthology.orchestra.run.smithi042.stderr: 92.9% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-26T03:41:06.931 INFO:teuthology.orchestra.run.smithi028.stderr: 92.9% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-26T03:41:06.956 INFO:teuthology.orchestra.run.smithi042.stderr: 92.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-26T03:41:06.957 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-04-26T03:41:06.957 INFO:teuthology.orchestra.run.smithi042.stderr:real 0m0.940s 2024-04-26T03:41:06.957 INFO:teuthology.orchestra.run.smithi042.stderr:user 0m2.592s 2024-04-26T03:41:06.957 INFO:teuthology.orchestra.run.smithi042.stderr:sys 0m0.094s 2024-04-26T03:41:07.047 INFO:teuthology.orchestra.run.smithi028.stderr: 92.8% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-26T03:41:07.049 INFO:teuthology.orchestra.run.smithi028.stderr: 2024-04-26T03:41:07.049 INFO:teuthology.orchestra.run.smithi028.stderr:real 0m1.032s 2024-04-26T03:41:07.049 INFO:teuthology.orchestra.run.smithi028.stderr:user 0m2.936s 2024-04-26T03:41:07.049 INFO:teuthology.orchestra.run.smithi028.stderr:sys 0m0.238s 2024-04-26T03:41:07.049 INFO:tasks.ceph:Archiving logs... 2024-04-26T03:41:07.049 DEBUG:teuthology.misc:Transferring archived files from smithi028:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi/7674443/remote/smithi028/log 2024-04-26T03:41:07.051 DEBUG:teuthology.orchestra.run.smithi028:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T03:41:07.772 DEBUG:teuthology.misc:Transferring archived files from smithi042:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi/7674443/remote/smithi042/log 2024-04-26T03:41:07.773 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T03:41:08.214 DEBUG:teuthology.misc:Transferring archived files from smithi066:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi/7674443/remote/smithi066/log 2024-04-26T03:41:08.215 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T03:41:08.653 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi/7674443/remote/smithi092/log 2024-04-26T03:41:08.654 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T03:41:09.045 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-26T03:41:09.055 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-26T03:41:09.055 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T03:41:09.058 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T03:41:09.061 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T03:41:09.064 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T03:41:09.186 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-26T03:41:09.198 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-26T03:41:09.198 DEBUG:teuthology.orchestra.run.smithi028:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T03:41:09.202 DEBUG:teuthology.orchestra.run.smithi042:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T03:41:09.205 DEBUG:teuthology.orchestra.run.smithi066:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T03:41:09.207 DEBUG:teuthology.orchestra.run.smithi092:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T03:41:09.216 INFO:teuthology.orchestra.run.smithi028.stderr:bash: line 1: ntpq: command not found 2024-04-26T03:41:09.219 INFO:teuthology.orchestra.run.smithi042.stderr:bash: line 1: ntpq: command not found 2024-04-26T03:41:09.221 INFO:teuthology.orchestra.run.smithi028.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T03:41:09.221 INFO:teuthology.orchestra.run.smithi028.stdout:=============================================================================== 2024-04-26T03:41:09.222 INFO:teuthology.orchestra.run.smithi028.stdout:^* hv01.front.sepia.ceph.com 3 6 377 58 +1505us[+1512us] +/- 52ms 2024-04-26T03:41:09.222 INFO:teuthology.orchestra.run.smithi028.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 58 -1715us[-1708us] +/- 56ms 2024-04-26T03:41:09.222 INFO:teuthology.orchestra.run.smithi028.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 58 +98us[ +105us] +/- 83ms 2024-04-26T03:41:09.222 INFO:teuthology.orchestra.run.smithi028.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T03:41:09.222 INFO:teuthology.orchestra.run.smithi066.stderr:bash: line 1: ntpq: command not found 2024-04-26T03:41:09.223 INFO:teuthology.orchestra.run.smithi092.stderr:bash: line 1: ntpq: command not found 2024-04-26T03:41:09.224 INFO:teuthology.orchestra.run.smithi042.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T03:41:09.224 INFO:teuthology.orchestra.run.smithi042.stdout:=============================================================================== 2024-04-26T03:41:09.224 INFO:teuthology.orchestra.run.smithi042.stdout:^* hv01.front.sepia.ceph.com 3 6 377 58 +1502us[+1503us] +/- 52ms 2024-04-26T03:41:09.225 INFO:teuthology.orchestra.run.smithi042.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 57 -1698us[-1698us] +/- 56ms 2024-04-26T03:41:09.225 INFO:teuthology.orchestra.run.smithi042.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 59 +98us[ +99us] +/- 82ms 2024-04-26T03:41:09.225 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T03:41:09.227 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T03:41:09.227 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2024-04-26T03:41:09.227 INFO:teuthology.orchestra.run.smithi092.stdout:^* hv01.front.sepia.ceph.com 3 6 377 59 +1501us[+1502us] +/- 52ms 2024-04-26T03:41:09.227 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 58 -1703us[-1703us] +/- 56ms 2024-04-26T03:41:09.227 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 59 +96us[ +97us] +/- 82ms 2024-04-26T03:41:09.227 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T03:41:09.228 INFO:teuthology.orchestra.run.smithi066.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T03:41:09.228 INFO:teuthology.orchestra.run.smithi066.stdout:=============================================================================== 2024-04-26T03:41:09.228 INFO:teuthology.orchestra.run.smithi066.stdout:^* hv01.front.sepia.ceph.com 3 6 377 58 +1503us[+1505us] +/- 52ms 2024-04-26T03:41:09.229 INFO:teuthology.orchestra.run.smithi066.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 57 -1695us[-1695us] +/- 56ms 2024-04-26T03:41:09.229 INFO:teuthology.orchestra.run.smithi066.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 59 +100us[ +101us] +/- 82ms 2024-04-26T03:41:09.229 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T03:41:09.229 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-26T03:41:09.239 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-26T03:41:09.239 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-26T03:41:09.248 DEBUG:teuthology.orchestra.run.smithi028:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T03:41:09.340 DEBUG:teuthology.orchestra.run.smithi042:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T03:41:09.437 DEBUG:teuthology.orchestra.run.smithi066:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T03:41:09.505 DEBUG:teuthology.orchestra.run.smithi092:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T03:41:09.570 DEBUG:teuthology.orchestra.run.smithi028:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T03:41:09.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T03:41:09.602 DEBUG:teuthology.orchestra.run.smithi042:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T03:41:09.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T03:41:09.633 DEBUG:teuthology.orchestra.run.smithi066:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T03:41:09.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T03:41:09.663 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T03:41:09.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T03:41:09.691 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-26T03:41:09.701 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-26T03:41:09.745 INFO:teuthology.task.internal:Duration was 607.910732 seconds 2024-04-26T03:41:09.745 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-26T03:41:09.754 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-26T03:41:09.754 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T03:41:09.757 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T03:41:09.759 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T03:41:09.762 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T03:41:09.799 INFO:teuthology.orchestra.run.smithi028.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T03:41:09.800 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T03:41:09.800 INFO:teuthology.orchestra.run.smithi042.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T03:41:09.803 INFO:teuthology.orchestra.run.smithi066.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T03:41:10.307 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-26T03:41:10.307 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi028.front.sepia.ceph.com 2024-04-26T03:41:10.307 DEBUG:teuthology.orchestra.run.smithi028:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T03:41:10.332 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi042.front.sepia.ceph.com 2024-04-26T03:41:10.333 DEBUG:teuthology.orchestra.run.smithi042:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T03:41:10.378 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi066.front.sepia.ceph.com 2024-04-26T03:41:10.378 DEBUG:teuthology.orchestra.run.smithi066:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T03:41:10.406 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi092.front.sepia.ceph.com 2024-04-26T03:41:10.406 DEBUG:teuthology.orchestra.run.smithi092:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T03:41:10.441 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-26T03:41:10.442 DEBUG:teuthology.orchestra.run.smithi028:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T03:41:10.445 DEBUG:teuthology.orchestra.run.smithi042:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T03:41:10.447 DEBUG:teuthology.orchestra.run.smithi066:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T03:41:10.450 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T03:41:10.508 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-26T03:41:10.508 DEBUG:teuthology.orchestra.run.smithi028:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T03:41:10.516 DEBUG:teuthology.orchestra.run.smithi042:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T03:41:10.519 DEBUG:teuthology.orchestra.run.smithi066:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T03:41:10.521 DEBUG:teuthology.orchestra.run.smithi092:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T03:41:10.647 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-26T03:41:10.660 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-26T03:41:10.660 DEBUG:teuthology.orchestra.run.smithi028:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T03:41:10.686 DEBUG:teuthology.orchestra.run.smithi042:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T03:41:10.711 DEBUG:teuthology.orchestra.run.smithi066:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T03:41:10.737 DEBUG:teuthology.orchestra.run.smithi092:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T03:41:10.760 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-26T03:41:10.771 DEBUG:teuthology.orchestra.run.smithi028:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T03:41:10.774 DEBUG:teuthology.orchestra.run.smithi042:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T03:41:10.776 DEBUG:teuthology.orchestra.run.smithi066:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T03:41:10.779 DEBUG:teuthology.orchestra.run.smithi092:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T03:41:10.796 INFO:teuthology.orchestra.run.smithi028.stdout:kernel.core_pattern = core 2024-04-26T03:41:10.797 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern = core 2024-04-26T03:41:10.801 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern = core 2024-04-26T03:41:10.822 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = core 2024-04-26T03:41:10.854 DEBUG:teuthology.orchestra.run.smithi028:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T03:41:10.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T03:41:10.882 DEBUG:teuthology.orchestra.run.smithi042:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T03:41:10.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T03:41:10.896 DEBUG:teuthology.orchestra.run.smithi066:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T03:41:10.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T03:41:10.911 DEBUG:teuthology.orchestra.run.smithi092:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T03:41:10.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T03:41:10.925 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-26T03:41:10.940 INFO:teuthology.task.internal:Transferring archived files... 2024-04-26T03:41:10.941 DEBUG:teuthology.misc:Transferring archived files from smithi028:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi/7674443/remote/smithi028 2024-04-26T03:41:10.941 DEBUG:teuthology.orchestra.run.smithi028:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T03:41:10.975 DEBUG:teuthology.misc:Transferring archived files from smithi042:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi/7674443/remote/smithi042 2024-04-26T03:41:10.976 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T03:41:11.008 DEBUG:teuthology.misc:Transferring archived files from smithi066:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi/7674443/remote/smithi066 2024-04-26T03:41:11.008 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T03:41:11.042 DEBUG:teuthology.misc:Transferring archived files from smithi092:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi/7674443/remote/smithi092 2024-04-26T03:41:11.042 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T03:41:11.074 INFO:teuthology.task.internal:Removing archive directory... 2024-04-26T03:41:11.075 DEBUG:teuthology.orchestra.run.smithi028:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T03:41:11.077 DEBUG:teuthology.orchestra.run.smithi042:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T03:41:11.079 DEBUG:teuthology.orchestra.run.smithi066:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T03:41:11.082 DEBUG:teuthology.orchestra.run.smithi092:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T03:41:11.124 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-26T03:41:11.141 INFO:teuthology.task.internal:Not uploading archives. 2024-04-26T03:41:11.141 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-26T03:41:11.151 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-26T03:41:11.151 DEBUG:teuthology.orchestra.run.smithi028:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T03:41:11.154 DEBUG:teuthology.orchestra.run.smithi042:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T03:41:11.156 DEBUG:teuthology.orchestra.run.smithi066:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T03:41:11.158 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T03:41:11.167 INFO:teuthology.orchestra.run.smithi028.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 03:41 /home/ubuntu/cephtest 2024-04-26T03:41:11.168 INFO:teuthology.orchestra.run.smithi042.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 03:41 /home/ubuntu/cephtest 2024-04-26T03:41:11.170 INFO:teuthology.orchestra.run.smithi066.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 03:41 /home/ubuntu/cephtest 2024-04-26T03:41:11.178 INFO:teuthology.orchestra.run.smithi092.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 03:41 /home/ubuntu/cephtest 2024-04-26T03:41:11.180 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-26T03:41:11.191 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-26T03:41:11.215 INFO:teuthology.run:Summary data: description: rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{default} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} duration: 607.9107315540314 flavor: default owner: scheduled_yuriw@teuthology success: true 2024-04-26T03:41:11.215 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T03:41:11.295 INFO:teuthology.run:pass